Quantifying the Unquantifiable

Diane Ravitch

In Joseph Conrad’s 1903 classic Heart of Darkness, there’s a scene where a physician, “an unshaven little man in a threadbare coat,” measures the skull of the main character, who is embarking on a journey to Africa. The doctor explains that he always does this, out of scientific interest, when examining someone bound for the Dark Continent. Asked if he repeats the head measuring after travelers return, the physician smiles cryptically. “Oh, I never see them,” he says, and adds, “Moreover, the changes take place inside, you know.”

Here was a novelist’s way of exposing the Western values system and its fixation on measurement and quantification. The scene came to mind recently as yet another education reform group issued yet another report lauding the ever-expanding use of standardized tests in American schools. Specifically, the Washington-based National Council on Teacher Quality highlighted the trend toward evaluating teachers on the basis of “objective evidence.” By which the Council meant student scores collected from the one-size-fits-all tests.

The Obama administration is using the federal purse to reward school systems that tow this line. The administration’s Race to the Top program is a tribute to this notion that understanding a teacher’s gifts or a student’s insights is pretty much a matter of doing the numbers. But is such thinking akin to the antics of Conrad’s head-measuring physician, who made a habit of quantifying the unquantifiable?

Diane Ravitch would say so. She served as an assistant secretary of education in the first Bush administration, and her most recent book is The Death and Life of the Great American School System. The title evokes Jane Jacobs’s The Death and Life of Great American Cities (1961), which ripped apart a political presumption of her time—“urban renewal.”

Ravitch is not opposed to testing as such. The education historian is against high-stakes testing, allowing scores to determine, for example, whether to fire a teacher or even shutter a school. She’s an unlikely skeptic in that regard, having once trumpeted, from her federal post and then from the conservative Hoover Institution, the very reform measures (including charter schools) she now deplores. Ravitch explains that evidence attesting to the benefits of these measures never materialized, and there’s mounting evidence that the reforms are distorting educational priorities.

As she tells it, education policy makers took a wrong turn when they started listening to, of all people, economists. Practitioners of the dismal science are behind the latest research fad, “value-added assessment,” which holds that teacher quality can be measured by test-score gains. The operative assumption is that a teacher’s worth “can be quantified, and those who do the quantification need never enter a classroom or think about how children learn,” Ravitch comments.

A Belief System Unto Itself

To be fair, complaining about economists who find answers in hard data is a bit like complaining about preachers who seek guidance in the Bible. That’s what they do. For most economists today, ultimate value is discerned largely through statistical models. As with all science, truth is reducible to fact. In the hands of many, this becomes a belief system unto itself.

There are other ways of knowing, though. Usually contrasted with the scientific method is the religious path to understanding, which draws on the symbols and imagery of faith as well as on the experience of the sacred. But truth can be gleaned also from poetry, from moral discourse, from all the disciplines of the liberal arts. In education there are rafts of research, storehouses of insight into such matters as how children learn and how teachers get through to them. Little of this lends to simple quantification.

The problem is not with economists and their econometrics. It’s with the incredible notion that all we’ve learned about masterly teaching, how to inspire and illuminate—and much that is yet to be learned—are of little consequence, now that there’s “value-added assessment.” Large swaths of the education policy establishment today appear to be in thrall to this belief.

In The Death and Life of the Great American School System, the author tells of the best teacher she ever had, Mrs. Ruby Ratliff, who taught English to seniors at Ravitch’s high school in Houston a half-century ago. Mrs. Ratliff exposed students to the great writers of the English language. Through the stories and verses, she also imparted lessons about character and discipline that spoke to young minds and hearts. The teacher had her students write long essays, and used her red pen freely.

I believe Mrs. Ratliff was a great teacher, but I don’t think she would have been considered “great” if she had been judged by the kind of hard data that is used now. How would the experts have measured what we learned? We never took a multiple-choice test. We wrote essays and took written tests in which we had to explain our answers, not check a box or fill in a bubble.

Mrs. Ratcliff would perhaps earn little credit from evaluators today, because her students would learn lessons that are hard to quantify—and because, as Conrad styled it, “the change takes place inside, you know.”


  1. Vincent G Sbano says

    I think a great deal of this reliance comes from a lack of faith. It is much easier to say here are the numbers than to make a qualitative judgement about what someone has learned. It seems to me that education although given to everyone (in greatly varying degrees of quality regardless of the measure) was never meant to devlop what most people would think is an educated person i.e. but one who could reason and think clearly and critically. It’s purpose seems to be to get each individual developed so that they can take their place in American society. When that society was agrian, you didn’t need much education. As it industrialized basic reading and writing and arithmatic skills were needed. As science was wed to industry in the first half of the 20th century a high school education was required and thats what most people got. The statistical model it seems to me is a great success story in that it does what it is intended to do. Measure student’s ability to adapt and fit in. The problem seems to be not everyone is destined to be a philopher king but schools seem to that everyone is going to college.

    • William Bole says

      Yes, but I think what we’re seeing now is new. Among other things, I’m concerned about the unintended consequences–e.g., the best teachers avoiding the schools and students that need them the most, because they’re being judged strictly by test scores.

  2. Ravitch was and remains such a shyster. She was on (hell, driving) the bandwagon and now that she sees it wobble she jumps off.


Leave a Comment