Why Do Science and Business Experiment More Than Government?

A new book by Jim Manzi argues that more randomized control trials of government programs could measurably improve public policy outcomes.

beakers full Flickr user Amy Loves Yah.png

To understand the argument that entrepreneur Jim Manzi makes in Uncontrolled: The Surprising Payoff of Trial-and-Error For Business, Politics, and Society, it's useful to forget everything you believe about why freedom is important. Forget the Declaration of Independence, the tyrannical king to whom it was sent, and the notion that we possess natural rights to life, liberty and the pursuit of happiness. All of that is completely irrelevant to a less familiar, more utilitarian argument about why this country's citizens ought to be afforded freedom.

This argument holds that ignorance is inevitable in a world as complicated as ours -- that it's impossible for anyone to be certain of the best way forward at a given moment. But a free society permits individuals to try different ways of living. Successes are imitated. Customs develop around what works. The things that don't are abandoned or made taboo. And so long as there's always an ability to depart from custom or challenge taboos, important innovations keep happening. In this telling, the freedom to engage in trial-and-error is indispensable to progress.

These aren't new insights.

They're core to the classical liberal tradition and its skepticism about the ability of policymakers to innovate via central planning. If Uncontrolled were merely a restatement of the need for epistemic humility among wonks and legislators, interest in it might be confined to the right. The book is of broader interest, and may turn out to be important, because its author makes a compelling argument for an ideologically neutral method for improving policy, one that left and right might both plausibly embrace, even as it challenges both sides to rethink some of their reflexes.

That method is a specific kind of experimentation.

The first part of Uncontrolled is an enjoyable summary of how philosophers, scientists, medical researchers and corporations all discovered -- at different times and places -- that conducting lots of rigorous experiments is the most reliable way to figure out what works and what doesn't work. As Manzi takes pains to demonstrate, the gold standard in experimentation is the double-blind, randomized field trial with a control group. Fortunately, technology has made conducting these sorts of experiments more cost effective than ever.

But they're rare in public policy.

That's partly due to their limits. "The reason we have increasing trouble building compact and comprehensive predictive theories as we go from physics to biology to social science is the increasing complexity of the phenomena under investigation," Manzi writes, which also makes it "far harder to generalize the results of experiments."

For example:

We can run a clinical trial in Norfolk, Virginia, and conclude with tolerable reliability that "Vaccine X prevents disease Y." We can't conclude that if literacy program X works in Norfolk, then it will work everywhere. The real predictive rule is usually closer to something like "Literacy program X is effective for children in urban areas, and who have the following range of incomes and prior test scores, when the following alternatives are not available in the school district, and the teachers have the following qualifications, and overall economic conditions in the district are within the following range." And by the way, even this predictive rule stops working ten years from now, when different background conditions obtain in the society.

But that's no reason to give up. Government is just going to be a supplier of social welfare, education, policing, defense, medical research, and more. Thus the book's suggestion: conduct lots of rigorous experiments to test the effectiveness of every new policy that's implemented.

Presented by

Conor Friedersdorf is a staff writer at The Atlantic, where he focuses on politics and national affairs. He lives in Venice, California, and is the founding editor of The Best of Journalism, a newsletter devoted to exceptional nonfiction.

How to Cook Spaghetti Squash (and Why)

Cooking for yourself is one of the surest ways to eat well. Bestselling author Mark Bittman teaches James Hamblin the recipe that everyone is Googling.

Join the Discussion

After you comment, click Post. If you’re not already logged in you will be asked to log in or register.

blog comments powered by Disqus


How to Cook Spaghetti Squash (and Why)

Cooking for yourself is one of the surest ways to eat well.


Before Tinder, a Tree

Looking for your soulmate? Write a letter to the "Bridegroom's Oak" in Germany.


The Health Benefits of Going Outside

People spend too much time indoors. One solution: ecotherapy.


Where High Tech Meets the 1950s

Why did Green Bank, West Virginia, ban wireless signals? For science.


Yes, Quidditch Is Real

How J.K. Rowling's magical sport spread from Hogwarts to college campuses


Would You Live in a Treehouse?

A treehouse can be an ideal office space, vacation rental, and way of reconnecting with your youth.

More in Politics

Just In