Popper is my homeboy: a manifesto

Will Wilkinson has an amusing series of posts demonstrating increasing frustration with the macroeconomic arguments about the stimulus proposal. In one, he sums up the source of his frustration:

When I see Delong more or less indiscriminately trashing everyone at Chicago, or Krugman trashing Barro, etc., what doesn't arise in my mind is a sense that some of these guys really know what they're talking about while some of them are idiots. What arises in my mind is the strong suspicion that economic theory, as it is practiced and taught at the world's leading institutions, is so far from consensus on certain fundamental questions that it is basically useless for adjudicating many profoundly important debates about economic policy. One implication of this is that it is wrong to extend to economists who advise policymakers, or become policymakes themselves, the respect we rightly extend to the practitioners of mature sciences. There is a reason extremely smart economists are out there playing reputation games instead of trying to settle the matter by doing better science. The reason is that, on the questions that are provoking intramural trashtalk, there is no science.  

This is just about perfectly stated.

I would state Will's implicit working definition of science for the purpose of this discussion as "an intellectual discipline that produces useful, non-obvious and reliable prediction rules".  Or at least, that's mine, and it's consistent with Will's statement.  Note that this doesn't let economists, political scientists or others off the hook by saying they want to "avoid physics envy" or whatever.  To say that they are practicing non-science by this definition is to say that their theorizing produces decision rules are at least one of: useless, obvious or unreliable.

In fact, you can see debates in mature sciences that sound a lot like the one that Will describes; they just tend to be around frontier issues.  Consider the physics of wings for airplanes.  There is a reasonably stable body of findings that can be (and has been) translated into engineering practice that works.  Airplanes stay up.  Giant tubes of metal with comparatively tiny lift surfaces go up in the sky, travel thousands of miles at about the speed of sound and land safely every day (sometimes, with sufficient pilot expertise, on the Hudson River).  That's about as useful, non-obvious and reliable as anything I see around me.  This is the ten tons that sits on one side of the scale whenever somebody wants to get into an argument about whether we "really know" this physics.  This is what we lack in most parts of economics, and certainly in the kind of economics that is being shouted about in the stimulus debate.

What is the key methodological feature that distinguishes science from the kinds of economic debates that frustrate Will: experiments.  Properly controlled experiments end debates (in addition, of course, to starting new ones).

I spent about the first ten years of my career executing increasingly sophisticated quantitative analyses that used data to try to evaluate and predict the success of business initiatives in order to develop corporate strategies.  Eventually, I saw that these analyses led to the same kind of scholastic debates as we see among macroeconomists.  The root issue was that it was impossible to find a methodology that could reliably distinguish correlation from causality.  Only through exhaustion of all possible alternatives did I come to find that experiments that randomly assign units of analysis (customers, stores, sales territories, etc.) to test and control groups are the only reliable method for determining causality. 

Once I figured this out, I became so fixated on it that I started what has now become a pretty good-sized software company, named Applied Predictive Technologies (APT).  APT's tools automate the design and interpretation of experiments for a good chunk of the Global 2000.  We have generally tried to stay below the radar, but that's impossible now, as the Harvard Business Review did an article in the current issue ("How to Design Smart Business Experiments") that is mostly about what we have done at APT to make experimental learning a reliable business function.  So I can come out of the closet on it a little bit. 

Presented by

Jim Manzi

Jim Manzi is Founder and Chairman of Applied Predictive Technologies (APT), an applied artificial intelligence software company. He is In also a Senior Fellow at the Manhattan Institute and a Contributing Editor of National Review, where he writes frequently for both the print and online editions on topics related to science, technology, business and economics.

The Blacksmith: A Short Film About Art Forged From Metal

"I'm exploiting the maximum of what you can ask a piece of metal to do."

Join the Discussion

After you comment, click Post. If you’re not already logged in you will be asked to log in or register.

blog comments powered by Disqus


Riding Unicycles in a Cave

"If you fall down and break your leg, there's no way out."


Carrot: A Pitch-Perfect Satire of Tech

"It's not just a vegetable. It's what a vegetable should be."


An Ingenious 360-Degree Time-Lapse

Watch the world become a cartoonishly small playground


The Benefits of Living Alone on a Mountain

"You really have to love solitary time by yourself."


The Rise of the Cat Tattoo

How a Brooklyn tattoo artist popularized the "cattoo"

More in Business

Just In