Learn About: Evaluating Performance | Common Core
Home > Policies > Measuring student growth: At a glance > Archived Chat: Growth Models
| print Print

Archived Chat: Growth Models

On December 6, 2007, the Center for Public Education hosted a live online chat on growth models called Doing Growth Right. The chat featured guest experts Jim Hull, Center policy analyst, and Marcus Egan, NSBA director of federal relations. Hull is the author of the Center's work on growth models called, Growth Models: A Guide for Informed Decision Making. Following is the transcript of the discussion.

A school board member from Pennsylvania writes:

We don't have a data system that can track individual student growth. Are there some interim strategies that can give credit to schools and districts for "significant" progress?

Jim Hull, Policy Analyst, Center for Public Education, answers:

As a matter of fact, there are two such models. One method is already included in NCLB to determine Safe Harbor, which is known as an "Improvement Model." This model tracks changes in test scores from one year to the next for the same grade level. For example, if 80 percent of this year's 4th graders reached proficiency compared to 70 percent of last year's 4th graders, then it is considered the 4th graders made growth of 10 percentage points. However, Improvement models are limited in their usefulness since they only compare the percent of this year's students reaching proficiency to the percent of last year's students reaching proficiency, and do not account for growth among individual students.

Another option is a Performance Index Model as used by New York State and Massachusetts. Just as with the Improvement Model, the Performance Index Model compares this year's 4th graders to last year's 4th graders. However, unlike the Improvement Model, which only gives credit for increasing the percentage of 4th graders reaching proficiency, Performance Index models give credit to schools for moving students from the lowest achievement levels, such as Below Basic, to a higher achievement level such as Basic. This model is more likely to give credit to schools making significant progress with their most underachieving students.

A school board member from Arkansas writes:

If everyone agrees the current way to measure progress is unfair and that we should measure individual students' gains from year to year, why is it so difficult to simply move forward?

Marcus Egan, Director, Federal Affairs, National School Boards Association, answers:

Great question! Given the widespread realization that the current system/framework for measuring Adequate Yearly Progress (AYP) is not presenting a fair and accurate picture of student and school performance, it is unfortunate the reauthorization timeline has slid into 2008, or later. How many more of our schools and districts will be misidentified and face more and more sanctions until the law is fixed?

The good news is that there does appear to be bipartisan support for incorporating "growth models" or the concept of measuring student growth in the next incarnation of ESEA/NCLB. Few seem to disagree that such an approach presents a fairer way of measuring performance and holding schools accountable, compared to current law.

As far as the hold up on reauthorization? Some will say it simply takes time for Congress to fully study the issue and move forward with a bill as massive as this one. Politics are certainly a factor as well and likely to become a greater obstacle as the 2008 elections approach. NSBA believes it will be a disservice to students and schools if the current law continues for another two or three school years, which is why we have urged Congress to take action now.

An educator from Virginia writes:

How likely is it that NCLB will be changed to give credit for student growth?

Marcus Egan, Director, Federal Affairs, National School Boards Association, answers:

Whenever Congress gets around to reauthorizing ESEA/NCLB, and we hope sooner rather than later, it is highly probable that schools and districts will be able to count student growth for AYP purposes. The details of such a plan may be in question, but the concept has broad support among members of Congress and the education community.

Are there benefits of growth models and alternate assessments, especially as they apply to students with special needs?

Jim Hull, Policy Analyst, Center for Public Education, answers:

Absolutely!! Actually, many educators call for growth models particularly for special education students and English language learners because they typically enter school the furthest behind. But to effectively measure growth for these students, assessments must accurately measure their true abilities. Many state assessments are currently designed to determine if a student is proficient or not and are not designed to measure how far students are from proficiency. Tests must be developed to measure student performance at the extremes of the of the achievement spectrum or true growth may not be captured. For example, if special ed students were given an AP calculus test, it's unlikely that the test will show the students' actual growth in basic arithmetic skills.

So if special education students are given adequate tests, measuring their growth will be a much more accurate way to determine how well schools are serving these students.

A parent from California writes:

If students are evaluated on the gains they make instead of whether they hit the mark, how will parents know if their kids are on grade level?

Jim Hull, Policy Analyst, Center for Public Education, answers:

This is a very important question. Because a growth measure by itself does not give any useful answers, especially when it comes to accountability. That is why no matter the growth model used, policymakers, with input from parents, educators, and the community, must answer the question "how much growth is enough growth?" One such model, Growth to Proficiency Model, answers that question, "enough growth to be proficient at a set time in the future." Most states in the NCLB Growth Model Pilot Program are using some type of Growth to Proficiency Model. In these models, students are expected to be on grade level within three or four years.

But even though growth may be a more accurate measure of school effectiveness, it is still important for parents to be informed on whether their child is on grade level or not.

A school board member from Kansas writes:

Should the federal government dictate what a state's growth model should look like?

Marcus Egan, Director, Federal Affairs, National School Boards Association, answers:

We don't believe it should. NSBA has advocated for states to have the flexibility to design and tailor a growth model that they believe is best suited to their needs. Keep in mind, growth models can be used for instructional and diagnostic purposes, beyond an accountability mechanism. Congress should have a light touch on this and not be overly prescriptive. States will have to submit their plans to the U.S. Department of Education anyway.

An association staff member from Washington, DC writes:

Can the same growth measures be used for accountability and for school improvement?

Jim Hull, Policy Analyst, Center for Public Education, answers:

Preferably not. Most experts strongly discourage using the same growth model for both accountability and school improvement; reason being, what you measure and how you measure it depends on what the growth measure will be used for.

For example, if you want to design a growth model to determine if a particular reading program is effective, then you want to use a growth model such as a Value-Added Model that can determine if students who took the reading program made greater gains than students who did not.

However, it will not tell you if those students have reached or are on pace to reach a certain goal like proficiency, which is the basis of most accountability systems. If you want to hold schools accountable for ensuring all students make a certain amount of progress each year, such as a year's worth of gains, then a simple growth model would be a much more effective tool.

A school board member from Iowa writes:

This is all very technical. Are there things school districts can do to communicate to their community the importance of looking at student gains so they understand?

Jim Hull, Policy Analyst, Center for Public Education, answers:

There are very technical aspects to incorporating growth models, however what they measure and how that information will be used needs to be clearly communicated to the community and to educators who will use, and in some cases be judged by, the outcomes of a growth model. Many experts recommend that states and districts provide training to both educators and the community at large to inform them how the growth measure works and how it will be used. This information should not be tucked away somewhere in an office or on a website but should be actively provided to the public. Just like in other education reforms, community engagement can be the key to successful implementation of a growth model.

A school board member from North Carolina writes:

Can you describe how the state pilot growth models are working and how they might have impacted AYP results?

Marcus Egan, Director, Federal Affairs, National School Boards Association, answers:

There are nine states participating in the U.S. Department of Education pilot program for growth models. So far, the impact on AYP results has not been overwhelming statistically. Of course, for the schools that did make AYP because of taking growth into account, it made all the difference in the world.

But overall, the percentage of schools making AYP because of growth that otherwise would have missed AYP has been less than fifteen percent in the participating states.

An educator from New Jersey writes:

If NCLB is changed so states can use growth models, will that open the door to using test scores to evaluate and pay teachers?

Marcus Egan, Director, Federal Affairs, National School Boards Association, answers:

Possibly, though it seems very unlikely that any legislation that relied only on test scores, even student gains or growth, will go very far in Congress: The House Committee "discussion draft" released earlier this year included a premium pay proposal that would allow districts to provide salary bonuses to exemplary teachers that took into account student growth on tests, though the draft did not specify how much test scores would have to factor in.

At this point, the data systems are not yet in place to do this. According to Data Quality Campaign, only eighteen states currently have a teacher identifier system that even allows for linking which teachers are instructing which students.

Also, as the CPE report on growth models suggests, statisticians have some concerns about using value-added models in connection with teacher evaluation. Certainly not as the only tool for conducting evaluations. There are too many outside variables in play. Another factor is the rise in team teaching. Since most models look to present the effect of single teacher on student achievement, how do you account for the impact of multiple teachers?

So the best answer to your question is that it probably is too early to tell. But the use of growth models does at least open the possibilities of improving on teacher evaluations by considering the academic gains made by their students.

A school board member from South Carolina writes:

What if a particular school is facing NCLB sanctions? Can growth models or alternate assessments be used at that point?

Marcus Egan, Director, Federal Affairs, National School Boards Association, answers:

In the U.S. Department of Education pilot program, schools in most of the participating states can make AYP using the general status model (e.g., cut scores), growth, or safe harbor. The idea behind incorporating growth models in the next version of ESEA/NCLB is to present a more accurate picture of how much schools are raising student achievement so that they are not misidentified as being "in need of improvement" in the first place, which then results in a progression of sanctions.

A parent from Alexandria, VA writes:

Even though the NCLB Growth Model Pilot has concluded, do you feel that states thinking about using growth models can benefit from peer review? If so, do you believe a peer review requirement should be included in the reauthorization of NCLB?

Marcus Egan, Director, Federal Affairs, National School Boards Association, answers:

You've hit on an issue that goes beyond growth models. NSBA believes Congress must ensure, in the reauthorized ESEA/NCLB, greater transparency of U.S. Department of Education decisions on approving state plans and that would include growth model proposals. The same goes for the peer review process. The more transparent the process and the decisions—whether approval or rejection—by the Department, the better.

With an issue as complicated and technical as the various models for accurately measuring student growth, especially if used for accountability purposes, there must be maximum transparency built into the process. States ought to be able to easily learn from one another what models work best and what pitfalls to avoid.

An educator from New York, NY writes:

Here in New York City, the mayor has rated schools using a growth measure and was still met with resistance from many in the city. Do you feel including growth in NCLB will quell the public's dismay for NCLB?

Marcus Egan, Director, Federal Affairs, National School Boards Association, answers:

Well, probably not in isolation. The list of what's wrong with NCLB isn't short. You could start with the unwillingness of Congress and the administration to adequately fund the law—going back several years now. You also have many educators, school board members, and other stakeholders who think NCLB is far too much of a federal intrusion into local and state education matters.

Incorporating growth into NCLB would be an improvement, and an important one. It would certainly be a fairer and more accurate way to measure school performance. NSBA has put forward to Congress more than four dozen recommendations for improving NCLB. In fact, H.R. 648, a bipartisan bill, incorporates these suggestions that we formulated based on input from school boards across the country who shared their implementation experiences with us. You can get more information at www.nsba.org/advocacy.

A school board member from Utah writes:

Does the kind of test you use make a difference? A lot of people are concerned that the quality of tests are driving standards down.

Jim Hull, Policy Analyst, Center for Public Education, answers:

Absolutely!!! Without quality measures of students' performance, no model, no matter how well designed, can accurately measure student growth. States must develop tests that assess a wide variety of student skills and are aligned with state standards. Unfortunately, many states' tests currently do not do this. No matter if tests are used for growth or not, they should be developed to measure higher-level skills, not just low-level memorization skills.

An educator from Attica, NY writes:

Would you say that the statistical procedures used in "most"  or "some" growth models will make it challenging for us to explain how these models work to our public? Are they based on complex modeling equations?

Marcus Egan, Director, Federal Affairs, National School Boards Association, answers:

This is a very legitimate concern. Our advocacy efforts have revolved around ensuring that the revised law provides states and districts with maximum flexibility and defers to them on key decisions. We've already seen confusion in the public and even among "school people" when high-performing schools based on the state accountability system don't make AYP under the federal law. So when it comes to growth models, we do not want Congress to be overly prescriptive. And states and districts will have to be diligent in explaining their system in layman's terms. 

Jim Hull, Policy Analyst, Center for Public Education, comments:

I absolutely agree with Marc that Congress needs to avoid being overly prescriptive when including growth in NCLB. States need the flexibility to design growth models that fit their data systems. Congress would be better off setting the goals and letting states develop models that meet those goals.

An educator from Maine writes:

Would changing to a system to measure growth be expensive? Money is always an issue.

Jim Hull, Policy Analyst, Center for Public Education, answers:

It can be, but it depends on what systems are currently in place. If your state or district already has a longitudinal data system with a unique student identifier (a database that can track individual students from year to year and from school to school), that cost will likely be substantially reduced. States like Ohio that have a longitudinal data system in place were able to incorporate a Value-Added Model into their state accountability system for only a of couple dollars per student by contracting the data analysis and professional development to outside experts.

However, the actual cost of implementing a growth model is complex. Estimating the staff time to develop and implement a growth model is hard to quantify, as is the time it takes for districts to collect the data. Both can add significant costs.

Marcus Egan, Director, Federal Affairs, National School Boards Association, comments:

Jim is right. Costs will be influenced by how far along states already are with their data systems. But Congress is looking at requiring states to develop extensive data systems that go beyond student identifiers and would also include issues like graduation and dropout data, teacher identifiers, and end of course completion rates, so it is quite likely that costs will fluctuate from one state to the next. Congress needs to be mindful of this and provide adequate funds.

A school board member from Georgia writes:

Who will pay for new tests and data systems? Doesn't Congress have a role?

Marcus Egan, Director, Federal Affairs, National School Boards Association, answers:

Yes!! Congress is going to have to be a much more reliable partner in funding the law's requirements if the law's goals are ever to stand a chance of being fully realized. The new or expanded data systems necessary to measure student growth and a host of other concepts cannot be done on the cheap. NSBA also wants to make sure that financial assistance is not limited to the state level. The capacity of local districts to keep pace with changes to data systems cannot be ignored. There are professional development costs to consider as well. Legislation from Representatives Rush Holt and Carolyn McCarthy would authorize $150 million a year for four fiscal years to fund these data systems. We believe no less than that amount should be authorized AND appropriated!

An educator from Ohio writes:

So far this discussion has focused on NCLB. But are there ways that teachers like me can use these methods in the classroom?

Jim Hull, Policy Analyst, Center for Public Education, answers:

Of course!! This may be the most overlooked benefit to growth models, especially in the NCLB debate. Value-Added models are particularly useful at the classroom level. Teachers in Tennessee, Ohio, and Pennsylvania have been using Value-Added data to make classroom decisions. I have heard from many of them that say it has really helped them improve their instruction. They are able to use the data to focus instruction to their students' weaknesses and determine which students they are reaching and which ones are struggling. Principals have also been successful in matching teachers' strengths to students' needs based on data. But remember that the data is just one tool to inform their decision-making along with grades, observations, and other tools. Value-Added data is useful, but it is not a silver bullet.

A school board member from Michigan writes:

What about high-achieving kids? I haven't heard anything about them, but my community is very concerned about challenging the top students too. We don't hear much about them in all the talk of NCLB.

Jim Hull, Policy Analyst, Center for Public Education, answers:

Your community is far from unique in this concern. Growth models can definitely help in getting schools to focus on high-achieving students, but this will not be a cure-all since the focus of NCLB is on equity and closing achievement gaps. However, growth models such as the one used in Tennessee for the NCLB Pilot Program is a step in that direction. In this model, only students on track to be proficient within three years are counted as making AYP. That includes students who are already above proficiency. So if a high-achieving student's scores start to fall, that student will not be counted as making AYP for NCLB.

Patte Barth, Center for Public Education, comments:

Thanks to everyone for a very lively discussion. This chat will be archived on the Center for Public Education website within the next 24 hours.

Home > Policies > Measuring student growth: At a glance > Archived Chat: Growth Models