Study evaluates effect of NCLB on national assessments of reading and math

Overview

Title: Tracking Achievement Gaps and Assessing the Impact of NCLB on the Gaps: An In-depth Look into National and State Reading and Math Outcome Trends

The question: The No Child Left Behind (NCLB) Act of 2001 was enacted to raise student achievement in math and reading especially for poor and minority students. To determine the impact NCLB has had on student achievement the study looked to answer three main questions:

(1) Did NAEP scores increase at a greater rate after NCLB was enacted than before?

(2) What is the difference in NAEP growth rates between states that had strong accountability polices and states that had weak accountability polices prior to NCLB?

(3) What are the differences between the proficiency rates of NAEP and state assessments and what contributes to the differences between states?

The study: The Civil Rights Project (CRP) study found mixed results on the impact NCLB has had on student achievement by analyzing 4th and 8th grade National Assessment of Educational Progress (NAEP) math and reading overall results as well as results for students in racial and socioeconomic subgroups.

The study compared the growth of NAEP scale scores and proficiency rates pre-NCLB (1990-2001) to the growth rates post-NCLB (2002-2005) at both the national and state levels. The study also compared the growth in NAEP scores of states that had weak pre-NCLB accountability policies to the scores of states that had strong pre-NCLB accountability policies.

The strength of state accountability policies was also used to determine the impact they had on the discrepancy between NAEP and state assessment proficiency rates.

Author: Jaekyung Lee, Association Professor, State University of New York at Buffalo Graduate School of Education

Foreword: Gary Orfield, Director, Civil Rights Project at Harvard University

Publisher: The Civil Rights Project at Harvard University

Funded by: The Bill and Melinda Gates Foundation and the Charles Stewart Mott Foundation

Released: June 2006

Peer reviewed: No

Research Methods: Quantitative

Link to full report: Tracking Achievement Gaps and Assessing the Impact of NCLB on the Gaps: An In-depth Look into National and State Reading and Math Outcome Trends [PDF]

Has the federal No Child Left Behind Act (NCLB)--enacted in 2001 and now up for reauthorization in 2006--improved student achievement in math and reading as intended, especially for poor and minority students? Yes and no finds a new study from the Civil Rights Project at Harvard University.

The study looked at scores on the National Assessment of Educational Progress (NAEP) before and after NCLB. Generally speaking, reading scores have remained relatively flat from 1990 to 2005 while math scores have improved significantly. So far though it does not appear NCLB has had much impact escalating the achievement rates for either subject. Furthermore, the study did not find achievement gaps between student groups narrowing in most cases.

The study found that states with strong accountability policies prior to NCLB made greater gains in NAEP scores than states with weak accountability systems; however, the growth in achievement rates for both groups of states did not accelerate after NCLB. The study also found large discrepancies in proficiency rates between NAEP and state assessments with most state assessments showing twice as many students attaining proficiency than NAEP. These findings should be interpreted cautiously though since the impact of NCLB is only judged on three years of data (2002-2005), and many states have yet to fully implement their NCLB accountability systems.

The Findings

(1) Did NAEP scores increase a greater rate after NCLB (2002-2005) than before NCLB (1990-2001)?

NAEP scores

Reading scores relatively unchanged for both 4th and 8th grades (1990-2005).
Math scores improved significantly for both 4th and 8th grades (1990-2005).
For both subjects and grades the growth rates were unchanged post-NCLB.
Poor and minority subgroups only showed small growth (1990-2005).
There has not been any significant narrowing of achievement gaps post-NCLB. Only 8th grade Hispanic math students achieved greater gains post-NCLB than pre-NCLB and had significantly narrowed the White-Hispanic achievement gap.

State assessments

A large majority of states showed only small improvement in reading (1990- 2005).
Many states made significant gains in math (1990-2005).
Most states did not show any change in their post-NCLB NAEP growth rates for either grade and in either subject.
Most states were not successfully closing their large achievement gaps.

(2) What is the difference in NAEP growth rates between states that had strong accountability polices and states that had weak accountability polices prior to NCLB?

States with strong pre-NCLB accountability policies made greater pre-NCLB gains in both math and reading but growth did not accelerate post-NCLB. These states generally started out with low NAEP scores in the 1990s.
Reading pre- and post-NCLB scores were not affected by state accountability policies.
State accountability policies had little impact on the achievement gaps in most states.

(3) What are the differences between the proficiency rates of NAEP and state assessments and what contributes to the differences between states?

States that have strong state accountability policies have larger gaps between their state proficiency rates and NAEP proficiency rates.
The percent of students meeting proficiency on state assessments is twice that of students meeting NAEP proficiency, on average.
Poor and minority students have a harder time meeting NAEP standards than their state standards.
Since NCLB there have been greater gains in state assessments than on NAEP in 25 states.

Caveats regarding the study

Be cautious when interpreting the results. The study states:

An analysis over a longer period of time could produce different results
Since there are only a few years of NAEP and state assessment data post-NCLB it may be premature to make definitive conclusions. The study is comparing the average 3-year growth rate over an 11 year time period (1990-2001) to the average growth of a three year time period (2002-2005)
NAEP scores were not adjusted to reflected changes in the demographics of test takers over time.

Conclusion

Even though the effect of NCLB on state and national achievement scores appears to be mixed, school and district leaders should analyze their own students’ achievement scores pre- and post-NCLB to determine what effect NCLB has had on their schools and why. This should include looking at the achievement of each racial and socioeconomic subgroup of students to determine if they are making adequate progress towards proficiency. School and district leaders should also determine if the state’s level of student proficiency is aligned with the skills and knowledge they established for their students at the given subject and grade level. Finally, school and district leaders should evaluate the programs they enacted in response to NCLB and determine if those programs are effective in meeting the goals of NCLB.

Posted: July 20, 2006