Learn About: Evaluating Performance | Common Core
Home > Evaluating performance > A guide to international assessments > Criticisms of international assessments: Fact or fiction?
| print Print


Criticisms of international assessments: Fact or fiction?

Do international comparisons of student performance provide any meaningful insights? Or are international rankings inherently flawed by an apples-to-oranges premise? This short paper sorts out the facts from the fiction about the uses of international comparisons of student achievement.

Criticism: International comparisons are unfair since most other countries educate (and, therefore, test) only the best students while the United States educates and tests all its students.
 
This is fiction.

This charge may have been true in the infancy of international assessments, but since the 1990s, this is no longer the case, especially in developed countries. For example, of the G8 countries (Canada, England, France, Germany, Italy, Japan, Russia, and the United States), only Russia has fewer than 97 percent of its 5- to 14-year-olds in formal schooling. Even at the high school level enrollment rates do not vary dramatically. The enrollment rate for 15- to 18-year-olds ranges from a low of 71 percent in Russia to a high of 89 percent in Germany, with the United States around the middle at 75 percent. Furthermore, the sampling procedures for each of the assessments have improved greatly to ensure that the samples are representative of the national population being measured.

Criticism: The rankings of overall average scores for each country do not show the full picture of each country’s education achievement.

This is fact.

For example, when PISA (Program for International Student Assessment) 2000 reported that the United States ranked 15th out of 27 countries in reading, it did not tell the complete story. It would be more honest to report how many countries scored statistically about the same, significantly higher, or lower than the U.S.. This shows that the differences are not trivial and did not happen by chance. Additionally, one should look at the scores from all the assessments that we participate in rather than defining the quality of American education on one test score, in one subject, at one grade level.

Regarding the PISA example above, only three countries scored significantly higher than the United States, four scored significantly lower, and 19 scored about the same. This means that if the assessment were given again to different samples of students, the same three countries would score higher than the U.S. and the same four countries would score below us. 

Criticism:  International assessments are not accurate measures of school effectiveness.

This is partly fact.

Not all international assessments are designed to measure school effectiveness. PISA and the Adult Literacy and Lifeskills Survey (ALL) are designed to measure the knowledge and skills students will use to succeed in the world and include knowledge gained both inside and outside of the classroom. In contrast, the Trends in International Mathematics and Science Study (TIMSS) is aligned to curriculum, meaning it is designed to measure what students have been taught in school.  Even so, a study by the American Institute of Research found a strong correlation between PISA and TIMSS, so both tests provide some indication about the educational systems of participating countries.

Criticism: International assessments provide little to no information about how to improve the quality of education in the United States.

This is fiction.

International assessments provide a wealth of information in addition to test scores. For each of the assessments, questionnaires are also distributed to students taking the assessment, their teachers, and principals of participating schools. The data collected from the questionnaires includes such things as courses students have taken and instructional techniques their teachers have used, along with other information about students’ learning environment including the number of years in school, the hours and days in the school year, class size, and others. This information helps point to policies and practices in higher performing countries that relate to higher achievement levels. The results can be analyzed by educators and policymakers to determine if similar policies could help U.S. students achieve at higher levels as well.

 


This summary is based on a document written by Jim Hull, policy analyst, Center for Public Education.

Posted: September 12, 2006

© 2006, Center for Public Education

Home > Evaluating performance > A guide to international assessments > Criticisms of international assessments: Fact or fiction?