The priest-gods

In the process of interviewing health care economists for the article I'm writing, David Cutler of Harvard told me that education and health care are the two segments of the economy where measurement of outcomes is hardest. It's unsurprising, then, he told me, that payment schemes in those two systems are so haphazard.

But even when there is good evidence, the systems don't use it. One of the most extreme examples is back surgery. The evidence seems to be completely overwhelming that back surgery is a terrible idea for back pain in all but a small minority of cases. Yet we do a phenomenal amount of back surgery. I presume that the surgeons performing this surgery are not venally crippling their patients in order to charge them exorbitant amounts of money. So why do they do something that shows so little evidence of improving the lives of their patients? And why do insurance companies and government programs pay for it?

The uncharitable explanation I have received is that doctors are incredibly good at detail and memorization and pattern recognition, and generally very bad at abstract analysis; they are also, because med school is so hard to get into, usually very arrogant.

The charitable (well, more charitable, anyways) explanation is that doctors are like the rest of us: they have an unreasonable belief that they are above average. (80% of people think that they are better-than-average drivers). And so even though they recognize that back surgery doesn't work on average, they think that they can beat the odds. They also naturally resist anything, like evidence-based medicine, that routinizes their jobs and cuts down on discretion. And we let them because medicine is important--so important that we can't stand thinking about the catastrophic possible errors, and so we prefer to turn our care over to priest-gods to whom we impute improbable, nay inhuman, wisdom.

Alex Tabarrok points out another example of this operating in education:

What we need to save inner-city schools, and poor schools everywhere, is a method that works when the teachers aren't heroes. Even better if the method works when teachers are ordinary people, poorly paid and ill-motivated - i.e. the system we have today.

In Super Crunchers, Ian Ayres argues that just such a method exists. Overall, Super Crunchers is a light but entertaining account of how large amounts of data and cheap computing power are improving forecasting and decision making in social science, government and business. I enjoyed the book. Chapter 7, however, was a real highlight.

Ayres argues that large experimental studies have shown that the teaching method which works best is Direct Instruction (here and here are two non-academic discussions which summarizes much of the same academic evidence discussed in Ayres). In Direct Instruction the teacher follows a script, a carefully designed and evaluated script. As Ayres notes this is key:

DI is scalable. Its success isn't contingent on the personality of some uber-teacher....You don't need to be a genius to be an effective DI teacher. DI can be implemented in dozens upon dozens of classrooms with just ordinary teachers. You just need to be able to follow the script.

Contrary to what you might think, the data also show that DI does not impede creativity or self-esteem. The education establishment, however, hates DI because it is a threat to the power and prestige of teaching, they prefer the model of teacher as hero. As Ayres says "The education establishment is wedded to its pet theories regardless of what the evidence says." As a result they have fought it tooth and nail so that "Direct Instruction, the oldest and most validated program, has captured only a little more than 1 percent of the grade-school market."

It's borderline criminal that even where evidence is available, we don't hold our teachers and doctors to the same standards of performance as would operate in a normal business.