Andrew Sullivan's Inadvertent Cherry-Picking

Andrew Sullivan's feature about Barack Obama's critics has stirred up a lot of controversy over the last few days.  On one charge, I'll defend him:  it is a little known fact that writers do not choose their headlines (or even necessarily see them).  So the next time you are set to fire off an angry letter to a writer, pause for a moment and ask yourself: "am I reacting to the piece, or to the headline?"  Of course, the publication is responsible for what headlines it slaps on things, but no one writes the publication; they send their nastygrams to the writer, who is completely unable to respond because what are they supposed to say?  "You're right, what my editors did is unforgiveable?"

However, one charge does stick: he's puffing up Obama's record by comparing gross to net. You just can't do that; any resulting numbers are useless:

Sullivan begins -- as Team Obama almost certainly will -- with Obama inheriting a terrible economy, writing that "[n]o fair person can blame Obama for the wreckage of the [first] 12 months, as the financial crisis cut a swath through employment." Yet shortly thereafter, he writes:

Since [the beginning of 2010], the U.S. has added 2.4 million jobs. That's not enough, but it's far better than what Romney would have you believe, and more than the net jobs created under the entire Bush administration.

Sullivan is comparing Obama's gross job creation to Bush's net job creation, ignoring that Bush also inherited a recession resulting from the collapse of the tech bubble. By Sullivan's own standard, this is unfair.

By the standard of net jobs created, Obama remains underwater and will be lucky to get to zero net jobs created by the end of his term. Conversely, if we simply judge Obama by therecovery, the results are terrible when compared to past recoveries. Nearly a million people have dropped out of the labor force, dropping the participation rate to an historic low, implying an unemployment rate close to 11%, instead of the official 8.5%.

Today, Andrew mounts a defense:

The official date of the recession under Bush was March 2001 - November 2001, caused by the tech crash and 9/11. So it started under Bush's tenure. The current recession - far, far deeper - began in 2007, and was already a year-old by the time Obama came into office - with a very steep swoon in the last quarter of 2008.

I really don't think this will do.  First of all, Bush took office on January 21st, 2001.  What exactly does Andrew Sullivan think that he did in one short month to hurl the economy into recession?  It generally takes 12-18 months for fiscal and monetary policy to work their way through the economy, so to the extent that you think the 2001 recession had policy foundations, it was definitionally Clinton's policies that created it.

And second of all, unemployment is a lagging indicator, so time-wise, Bush actually had it more difficult in one way--there was less time for the economy to recover from a recession that ended in November 2001, than from one that ended in June.

Now, of course you can argue that Obama presided over a much deeper recession, and that it's unfair to compare his job creation to George Bush's.  And you'd be right!  I'd go even further, and point out that politicians really have astonishingly little control over the business cycle, making all of these sorts of comparisons complete nonsense.

But Andrew did not say that it's unfair to compare the two; Andrew made the comparison.  Since he thinks it's unfair, he wants to compare Obama's gross job creation since the trough of the recession, to George Bush's net job creation from close to the peak of the internet bubble.  

But there are very good reasons that we do not allow social scientists to pick their start and end dates so as to cast their subjects in the most flattering--or "fair"--light.  And the main one is simply that it's all to easy, even without consciously meaning to, to start out with the argument you want to make, and work back to the metrics that will support that argument.  Scientists have a technical term for this sort of data analysis: nonsense.

An economist who arbitrarily decided to start a job count comparison at the beginning of one president's term, and in the middle of another's, would be laughed out of peer review; a freshman who turned in such work would flunk the assignment.  Whether or not Andrew intended to do so, he is cherry-picking, and as a result, the comparison is worthless.