You Just Can’t Say That

Posted by Neal McCluskey

Let’s get one thing straight: As I’ve noted on numerous occasions, you can’t look just at National Assessment of Educational Progress (NAEP) results – especially only between two years – and attribute gains or losses to specific laws or programs. There are simply too many variables at play in education – federal laws, state laws, school choice, child nutrition, teacher quality, parents’ attitudes, the weather – to confidently assert that any one is responsible for changing scores. Indeed, it is possible that nothing government has done has had any effect, and every trend just reflects changing attitudes toward education among students themselves.

And yet, some reporters identify something akin to a god variable anyway, as the Associated Press did in its coverage of the new NAEP long-term-trends report:

The biggest gains came from low-achieving students. That is probably not an accident — the federal No Child Left Behind law and similar state laws have focused on improving the performance of minority and poor children, who struggle the most.

Now, there are a lot of problems with this statement, including that several of the lowest-achieving percentiles by age and subject saw no statistically significant changes in scores between 2004 and 2008; many groups had periods of faster gains before NCLB (though we don’t even have clear before and after-NCLB data points); and NAEP offers no income-based score breakdowns, only the proxy of parents’ education – and that just for 13 and 17-year-olds in mathematics. But the biggest problem is that, all of these factual problems aside, there is no way to ascribe score changes to specific laws or government policies. The data just aren’t there.

Fortunately, most of the coverage of the NAEP report has been pretty reasonable, including from the Washington Post and New York Times. But the AP reaches a lot of people, and that means many Americans are going to get “news” about the latest NAEP findings that is little more than unsupportable conjecture.

Educational Productivity Has Collapsed — NAEP

Posted by Andrew J. Coulson

The latest Long Term Trends results of the National Assessment of Educational Progress are out. They reveal a productivity collapse unparalleled in any other sector of the economy.

At the end of high school, students perform no better today than they did nearly 40 years ago, and yet we spend more than twice as much per pupil in real, inflation-adjusted terms. I can’t think of any other service that has gotten worse during my lifetime. Our school system has failed alone.

While the stagnation in overall achievement masks a 3 to 5 percent gain in the achievement of African American 17-year-olds since 1970, the scores for whites at the end of high school are virtually unchanged.

Anyone who points to the slightly higher scores in the early grades as cause for celebration is missing the point. What parents care about is that their children are well prepared for higher education and future careers at the end of their secondary education. The fact that scores have risen somewhat in the early grades means little since those gains evaporate for the vast majority of students by the time they graduate.

Update: The Associated Press story is now out on the Long Term Trends NAEP results… and it doesn’t mention the long term trends. The story only reports changes in achievement over the most recent 4 year interval of a test whose raison d’être is to reach back to the early 1970s. I wonder why…. Fortunately, the Detroit Free Press does a better job.