Bruno: Achievement Gaps Have Closed More Than You Think
Interpreting the results usefully is inevitably difficult given the variety of factors that impact achievement levels, and often the desire to construct a desirable narrative prevents commentators from making a rigorous effort to understand the raw NAEP data.
Last week's release of new math and reading scores was, sadly, no exception. Errors common in the past - like cherry-picking state results to "prove" policy effectiveness or failing to disaggregate scores by race - continue to mislead many observers.
One of the subtlest pitfalls, however, concerns the apparent persistence of achievement gaps between different groups of students.
This mistake is so common that it's probably unfair to pick out individual examples. Just for illustrative purposes it will suffice to say that the vast majority of NAEP coverage adopts the same narrative as The Christian Science Monitor or Education Week when it comes to racial achievement gaps: It's nice that scores are going up, but we don't seem to be making progress on narrowing gaps between students of different racial backgrounds.
There is a sense in which this is approximately true: the number points separating, say, the average score for black students from the average score for white students has for years remained largely unchanged in most subjects at most grade levels.
There is another sense, however, in which this narrative is somewhat misleading, because given the across-the-board increases in scores over the last several decades the differences in performance between students of different backgrounds have been steadily diminishing in significance.
Read on to see what I mean.
To see why rising achievement matters, we can consider 8th grade reading scores. According to last week's report, the difference between the average score for black students and the average score for white students has remained exactly the same since 1998 at 26 points.
This is the very definition of a "persistent achievement gap". (The NAEP tweaked its methodology in 1998, so I'm omitting prior years' scores for simplicity.)
At the same time, though, the average reading score white 8th graders has increased from 270 points to 276 points. As a result, that 26 point gap represents a (slightly) smaller fraction of white students' overall achievement. Specifically, it means that black 8th graders have gone from scoring 90.4% as high as their white peers (on average) to scoring 90.6% as high.
In other words, the "stagnant" 26-point gap between black and white students is obscuring the fact the gap - expressed as a fraction of white student achievement - has narrowed.
Granted, closing 2% of the gap - 0.2 percentage points of a 9.6 percentage point gap - is not terribly impressive, especially over a period of 15 years. But the point is that the magnitude of the gap in points will understate progress in the context of rising scores.
The black-white gap in 8th grade reading also represents the worst-case for achievement gaps. Since 1998, for example, Hispanic students have gone from scoring 90% as well as white students to scoring 92.8% as high. That represents, in essence, a closing of 28% of the 1998-era white-Hispanic gap (expressed as a fraction of white students' average achievement).
There has also been substantial gap-narrowing in 8th grade math. Since 1996, black 8th graders have gone from scoring 85.4% as high as white students to scoring 89.5% as high, closing more than 27% of the original gap. Since 2009 alone Hispanic students have closed one-fifth of the 8th grade math gap (again, measured as a fraction of white student achievement).
None of which is to say we should be sanguine about achievement - or achievement gaps - in American education. Some gaps are more persistent than others, progress has in some cases slowed in recent years, and more rapid progress would always be better.
It is probably nevertheless fair to say that, by some measures, achievement gaps have not been quite as stubborn as we are often lead to believe. - PB (@MrPABruno)