Quantifying the Unquantifiable

Diane Ravitch

In Joseph Conrad’s 1903 classic Heart of Darkness, there’s a scene where a physician, “an unshaven little man in a threadbare coat,” measures the skull of the main character, who is embarking on a journey to Africa. The doctor explains that he always does this, out of scientific interest, when examining someone bound for the Dark Continent. Asked if he repeats the head measuring after travelers return, the physician smiles cryptically. “Oh, I never see them,” he says, and adds, “Moreover, the changes take place inside, you know.”

Here was a novelist’s way of exposing the Western values system and its fixation on measurement and quantification. The scene came to mind recently as yet another education reform group issued yet another report lauding the ever-expanding use of standardized tests in American schools. Specifically, the Washington-based National Council on Teacher Quality highlighted the trend toward evaluating teachers on the basis of “objective evidence.” By which the Council meant student scores collected from the one-size-fits-all tests.

The Obama administration is using the federal purse to reward school systems that tow this line. The administration’s Race to the Top program is a tribute to this notion that understanding a teacher’s gifts or a student’s insights is pretty much a matter of doing the numbers. But is such thinking akin to the antics of Conrad’s head-measuring physician, who made a habit of quantifying the unquantifiable?

Diane Ravitch would say so. She served as an assistant secretary of education in the first Bush administration, and her most recent book is The Death and Life of the Great American School System. The title evokes Jane Jacobs’s The Death and Life of Great American Cities (1961), which ripped apart a political presumption of her time—“urban renewal.”

Ravitch is not opposed to testing as such. The education historian is against high-stakes testing, allowing scores to determine, for example, whether to fire a teacher or even shutter a school. She’s an unlikely skeptic in that regard, having once trumpeted, from her federal post and then from the conservative Hoover Institution, the very reform measures (including charter schools) she now deplores. Ravitch explains that evidence attesting to the benefits of these measures never materialized, and there’s mounting evidence that the reforms are distorting educational priorities.

As she tells it, education policy makers took a wrong turn when they started listening to, of all people, economists. Practitioners of the dismal science are behind the latest research fad, “value-added assessment,” which holds that teacher quality can be measured by test-score gains. The operative assumption is that a teacher’s worth “can be quantified, and those who do the quantification need never enter a classroom or think about how children learn,” Ravitch comments.

A Belief System Unto Itself

To be fair, complaining about economists who find answers in hard data is a bit like complaining about preachers who seek guidance in the Bible. That’s what they do. For most economists today, ultimate value is discerned largely through statistical models. As with all science, truth is reducible to fact. In the hands of many, this becomes a belief system unto itself.

There are other ways of knowing, though. Usually contrasted with the scientific method is the religious path to understanding, which draws on the symbols and imagery of faith as well as on the experience of the sacred. But truth can be gleaned also from poetry, from moral discourse, from all the disciplines of the liberal arts. In education there are rafts of research, storehouses of insight into such matters as how children learn and how teachers get through to them. Little of this lends to simple quantification.

The problem is not with economists and their econometrics. It’s with the incredible notion that all we’ve learned about masterly teaching, how to inspire and illuminate—and much that is yet to be learned—are of little consequence, now that there’s “value-added assessment.” Large swaths of the education policy establishment today appear to be in thrall to this belief.

In The Death and Life of the Great American School System, the author tells of the best teacher she ever had, Mrs. Ruby Ratliff, who taught English to seniors at Ravitch’s high school in Houston a half-century ago. Mrs. Ratliff exposed students to the great writers of the English language. Through the stories and verses, she also imparted lessons about character and discipline that spoke to young minds and hearts. The teacher had her students write long essays, and used her red pen freely.

I believe Mrs. Ratliff was a great teacher, but I don’t think she would have been considered “great” if she had been judged by the kind of hard data that is used now. How would the experts have measured what we learned? We never took a multiple-choice test. We wrote essays and took written tests in which we had to explain our answers, not check a box or fill in a bubble.

Mrs. Ratcliff would perhaps earn little credit from evaluators today, because her students would learn lessons that are hard to quantify—and because, as Conrad styled it, “the change takes place inside, you know.” …read more