Report finds student achievement up but impact of NCLB unclear

The No Child Left Behind Act (NCLB) ignited widespread and passionate debate with its passage in 2002, and a new report on its impact is unlikely to douse any rhetorical flames.

The report, prepared by the Center for Education Policy (CEP), finds that student achievement in reading and math has improved since NCLB was enacted—a finding some observers call into question—but cannot say whether those improvements were realized because of NCLB.

CEP, an independent, not-for-profit organization that advocates on behalf of public education, released the report in June. Answering the Question that Matters Most: Has Student Achievement Increased Since No Child Left Behind? is CEP's fifth annual report on the impact of NCLB.

Not surprisingly, both fans and critics of NCLB find fodder in the report to support their views. Supporters zero in on gains made in student achievement as evidence the law is working as intended. Opponents emphasize that gains can't be attributed to the law.

So, what insights does the report shed on NCLB? The report makes a strong case for concluding that since 2002, student achievement has, in fact, gone up in most states (although by how much isn't clear), and that gaps between groups of students have narrowed. The biggest gains in math and reading were found at the elementary school level

But the report says gains cannot be attributed directly to NCLB as considerable federal, state, and local reforms improvement efforts have all been underway before and since 2002.

As for what did produced the gains, CEP cautions against jumping to any conclusions—increased learning, "teaching to the test," changes in population, and myriad other factors could be part of the puzzle. At any rate, NCLB does not appear to have harmed student achievement.

The report cited five key findings:

In most states, student achievement in reading and math has increased since 2002.
Achievement gaps between groups of students have narrowed since 2002
In 9 of the 13 states with sufficient data to analyze achievement trends before and after 2002, average yearly gains in test scores were greater after NCLB took effect than before.
It is difficult, if not impossible, to determine the extent to which these trends in test results have occurred because of NCLB.
Data required to perform rigorous analyses of achievement trends were not available in many states, and more attention must be given to issues of quality and transparency of state test data.

In a press release following release of the report, U.S. Secretary of Education Margaret Spellings said, “This report confirms that No Child Left Behind has struck a chord of success with our nation’s schools and students.”

Others disagree. John T. Yun, assistant professor at the Gevirtz School of Education at the University of California, Santa Barbara concurs with the report's finding that we can't definitively attribute upward trends in student achievement to NCLB, but says flaws in the evaluation methodology call into question the finding that student achievement has improved since NCLB took effect.

In a review written for the Great Lakes Center for Education Research and Practice—an organization funded by the National Education Association—Yun notes:

The same states are not included in each year's analysis since states vary in the number of years for which they have achievement data.
Comparing achievement trends and gaps between student groups only for states that have at least two years of data instead of all states, likely led to conclusions that inflated actual gains in achievement.
Findings on narrowing gaps between groups are unreliable since they are based on small number of students within subgroups and small sample sizes tend be less reliable measures of true achievement.
Results of the pre- and post-NCLB comparison are unreliable since only 13 states had enough data to be included.

CEP itself notes in the report the following factors that complicate analysis:

Each state has its own assessments aligned to its own standards and these vary from state to state.
Some states have separate accountability systems that run parallel with NCLB.
There is no single point in time when all states implemented NCLB. Although NCLB was passed in 2002, some states have just recently fully complied with with NCLB’s assessment and accountability provisions. Annual assessments for grades 3 through 8 were not even required until the 2005-06 school year.
States periodically make major changes to their assessments, making comparisons to past performances unreliable.
States and districts continually implement their own reforms and policies that are impossible to isolate from the impact of NCLB. Without the ability to isolate the impact of NCLB from other reforms and policies, it is difficult to determine whether NCLB is the real source for the change in student achievement.

Nonetheless, school board members and their districts should take away from the CEP report the importance of consistently collecting and analyzing data to track student achievement as part of continual improvement efforts—not only in terms of NCLB's requirements but also in terms of their own community's goals and priorities.

Finally, school board members are in a unique position to evaluate how their own policy decisions have been influenced by NCLB and what impact those decisions have had on student achievement to evaluate for themselves the effect that NCLB has had on their schools.