About this blog Subscribe to this blog

Bruno: Standardized Tests *Do* Measure Some Important Cognitive Abilities

Failstemp ccommon flickrOne of the more interesting bits of news that you may have missed over the holidays was the announcement of findings from researchers at MIT indicating that even when schools effectively boost students' scores on standardized tests, they don't seem to do much to improve students' "fluid intelligence" -- those cognitive abilities, like working memory capacity, that can be helpfully applied across contexts.

This research seemed to strike a chord with critics of standardized testing, who leapt on the opportunity to emphasize the limits of such exams.

Unfortunately, in their eagerness to strike a blow against tests some commentators have badly over-interpreted - or plainly misinterpreted - the results.

First and foremost, and contrary to many headlines, the researchers did not find that standardized tests "don't measure cognitive ability."

In fact, the authors carefully distinguish different sorts of cognitive ability and emphasize that while improvements on standardized tests don't seem to be accompanied by gains in fluid intelligence, they nevertheless represent meaningful gains in "crystallized intelligence" (e.g., knowledge).

For most intents and purposes  "knowing things" counts as a cognitive ability. We should obviously prefer improving all cognitive abilities to improving some, but nothing in this new study calls into question the notion that standardized tests measure important cognitive abilities.

More seriously, critics of standardized testing have tried too hard to draw sweeping policy prescriptions from these limited findings.

It's natural enough to assume that if our schools aren't organized to substantially improve students' fluid intelligence, we should reorganize them.

Unfortunately, the reality is that we don't know very much about how to use schools to increase students' fluid intelligence.

The authors of this new study are able to identify a handful of studies that seem to suggest that some interventions may, in fact, be able to improve students' fluid reasoning abilities.

These studies tend to show only modest effects, however, and often only after extremely intensive interventions. And many of these studies are notably anomalous in finding significant intervention effects on fluid intelligence at all.

When looking at the research literature as a whole, it is fair to say that we know very little about which programs or interventions, if any, reliably produce meaningful gains in fluid intelligence at reasonable costs. 

Meanwhile, we know a great deal how to use schools to develop students crystallized intelligence and we know that those cognitive abilities matter both in their own right and because they help circumvent limitations in fluid abilities. It is doubtful, then, that we should be substantially reorganizing schools to focus more on developing fluid intelligence.

These wrinkles are all reasonably clear both in the article itself and from the authors' own words. In the hyper-polarized and overly-ideological world of education, however, caution and nuance often do not prevail. - PB (@MrPABruno) Imiage via Flickr.


Feed You can follow this conversation by subscribing to the comment feed for this post.

The comments to this entry are closed.

Disclaimer: The opinions expressed in This Week In Education are strictly those of the author and do not reflect the opinions or endorsement of Scholastic, Inc.