Bruno: StudentsFirst Grades Don't Add Much Value
If test scores in highly-rated states - really just LA and FL - go up faster than in other places, the thinking is that this would validate the StudentsFirst approach to education policy; otherwise they'll be discredited.
I think that's overly optimistic. The biggest problem with the theory is that it's just not credible that in the near future the data are going to be sufficiently clear to vindicate SF's letter grade assignments. Outcomes in education are subject to a wide range of variables and that's always going to muddy the water enough to let people on all sides of the issue claim vindication.
As far as StudentsFirst is concerned that ambiguity is probably a feature, not a bug. Making concrete predictions that are clearly testable over short time horizons is a high-risk, low-reward strategy for pundits and policy organizations, and I see no indication that SF has any interest in subjecting themselves to that level of accountability.
After all, StudentsFirst could have been much clearer about what they think are the merits of their positions by stating explicitly which states should expect to see test scores rise relative to the rest of the country. Instead, they framed their ratings in entirely procedural terms making it that much harder to identify what their testable - and falsifiable - claims actually are.
Alternatively, StudentsFirst could have forced a wider grade distribution to illustrate more forcefully which states SF thinks are moving in the right direction and which we should look to to see educational progress. Instead, SF clustered grades tightly at the bottom of the distribution; fully three-quarters of jurisdictions received Ds or Fs, and no state got higher than a B-. As a result it's very clear that SF is dissatisfied with state-level policy in general, but it's much less clear which states they expect to see do relatively well in the future.
It's possible that StudentsFirst has perfectly good reasons for not making clearer or more strident predictions. Predictions are hard! Crucially, though, there's no evidence that that's what SF was trying to do here anyway. Assigning letter grades to things to express your approval of them is a tried-and-true way of generating buzz for your preferences, and I see no reason to think SF was aiming for anything more than that. - PB (@MrPABruno) (image source)