Bruno: Add NAEP Increases To Debate Over FL Rating System

Matt Di Carlo and Matthew Ladner are having a back-and-forth on the virtues of Florida's state-level accountability system that's fascinating, if wonky, reading.

In a comment here Di Carlo goes so far as to say Florida's system is "the worst I've seen", and Ladner counters that "[g]iven the large improvements in NAEP scores for disadvantaged Florida students [since 1999], if Florida has 'the worst' system, I’m eager to see the best."

They both make some good points, but because Ladner references rising NAEP scores for the Sunshine State's disadvantaged students, this is a good opportunity for me to make one of my favorite points. Namely, any strident criticism about status quo education policy needs to be reconciled with the fact that the last decade or so has actually been really good for the NAEP scores of traditionally disadvantaged groups of students nationwide.

Even if, like Di Carlo, I'm somewhat skeptical about the causality in Florida, Ladner definitely points to some good news there for low-income students. However, I want to emphasize that the tide has been rising all across the country.  Here are the average scale score gains on NAEP tests for a variety of demographics since the 2003 administration:

Subject/Grade	Reading/4	Reading/8	Math/4	Math/8
Free Lunch	5	6	8	12
R-P Lunch	7	3	9	8
Black	7	5	8	10
Hispanic	6	7	7	11
Amerindian	0	6	2	2
EL	2	2	5	2
LD	1	6	4	8

Given that those numbers are, generally, pretty positive, I think we should be cautious about making arguments to the effect that this or that NCLB-era policy has been a "catastrophe" or "disaster".  Yes, maybe things would be somewhat better had we continued with the status quo of the 1990s, but we're not exactly slipping into an educational dystopia, either.

Now, I understand that those score increases began well before the NCLB era, and may even have slowed slightly. So the flip side of this is that claims about the indispensability of NCLB-type reforms should be taken with a grain of salt as well.

In fact, one possibility we might want to consider more carefully is that even major-seeming education policy changes like NCLB aren't, on balance, quite as big of a deal as we are inclined to believe. (And I'm including myself, here.) - PB (@MrPABruno) (Image source)

Comments

[Matt]

Thanks Paul, but just to be clear, the discussion you mention is exclusively focused on the state's grading system - i.e., the simple formula by which districts are given A-F grades. It is not about Florida's accountability system, which is a *far* wider and more important issue.

[john thompson]

Paul,

NAEP scores in Reading, especially in the more important 8th grade assessment, have slowed since 2003. (As Duncan says, 4th grade test scores won't get you a future and when they don't persist, that calls testing into question.) And few places have seen a greater slowing than Florida. Florida's improvements largely were in Bush I, before he tilted to this testing craze.

[Paul Bruno]

@Matt - Thanks for the clarification. Truthfully, your discussion about FL was mostly just an excuse for me to go off on a tangent about NAEP scores more generally.

@John - I'm reluctant to pick out a single test for use, here, especially since reading ability seems to be more heavily influenced by out-of-school factors than math. As I said in the post, I understand that score growth has slowed in some respects 2003-2011 vs. 1994-2003 (8 pts vs. 5 for black students on the reading8 test), but again I'm reluctant to pick out single data points (Hispanic students actually saw *faster* growth 2003 onward, 2 pts vs. 7) and in any case I'm skeptical about claims of causality here.

In any event, though, even if we assume that NCLB caused this very modest slowing of growth, my point was that a lot of the criticism seems hyperbolic given the apparent size of the (assumed) effect.

[john thompson]

Because reading is more influenced by out of school factors - factors that NCLB distracted from - that is more evidence that it should be the focus. Improved math scores may or may not improve a person's future prospects. Reading comprehension is essential. A minor decline at the cost of billions, and the damage done to teachers, many students, and educational principles, is not minor.

[Paul Bruno]

@John - It seems to me you're making a lot of questionable assumptions, here: that we should use scores less-related to schooling to evaluate NCLB, that the 8th grade reading test should be singled out for that purpose as the most important anyway, that NCLB caused a very slight decrease in scores for that test, and that (I think this is what you're saying) NCLB crowded out ("distracted from") other sorts of productive non-education domestic policy that the (Republican?) government would otherwise have made, and that there have been additional-but-unspecified damages to teachers, students, and principles. I'd be interested to see evidence for any/all of those assumptions, but consider me skeptical for the time being in part because, again, all of those assumptions have to be reconciled with generally rising scores for disadvantaged groups of students since 2003.