Reihan: Always Bet On Bueno de Mesquita

This is slightly terrifying:

To verify the accuracy of his model, the CIA set up a kind of forecasting face-off that pit predictions from his model against those of Langley’s more traditional in-house intelligence analysts and area specialists. “We tested Bueno de Mesquita’s model on scores of issues that were conducted in real time—that is, the forecasts were made before the events actually happened,” says Stanley Feder, a former high-level CIA analyst. “We found the model to be accurate 90 percent of the time,” he wrote. Another study evaluating Bueno de Mesquita’s real-time forecasts of 21 policy decisions in the European community concluded that “the probability that the predicted outcome was what indeed occurred was an astounding 97 percent.” What’s more, Bueno de Mesquita’s forecasts were much more detailed than those of the more traditional analysts. “The real issue is the specificity of the accuracy,” says Feder. “We found that DI (Directorate of National Intelligence) analyses, even when they were right, were vague compared to the model’s forecasts. To use an archery metaphor, if you hit the target, that’s great. But if you hit the bull’s eye—that’s amazing.”



The next paragraph begins,

How does Bueno de Mesquita do this? With mathematics.



I was hoping it would begin,

How does Bueno de Mesquita do this? By crushing monkey bones and garlic into a mystical powder using his grandmother's mortar-and-pestle.



My question: how sure are we really that Bueno de Mesquita is not in fact using a mystical powder? I see no convincing evidence to the contrary.