Is Ohio a 'Toss-Up'?

More
Columbus.JPG

The Columbus Dispatch, the only newspaper in Ohio's biggest city, has declared Ohio a "toss-up" between President Obama and Mitt Romney. This will please the Romney camp, which has been fighting hard against the "Ohio is Obama's rock-solid firewall" narrative.

For reasons I'll explain, this headline is misleading. And it's tempting to think it's intentionally misleading. After all, the Dispatch endorsed Romney (and in fact hasn't endorsed a Democrat for president since 1916). And the lead paragraph of the story under the "toss-up" headline does have a certain Mitt Romney-pep-rally quality to it: "The 'Ohio firewall' precariously stands for President Barack Obama, but a strong Republican turnout could enable Mitt Romney to tear it down on Election Day."

Still, I think we can give the Dispatch the benefit of the doubt and assume that its own firewall -- the church-state separation that is supposed to keep a newspaper's editorial stance from coloring its reporting -- is intact. Because there's a simpler explanation for the "toss-up" headline: It rests on the same slightly-too-simple way of thinking about polling that lots of reporters and other Americans evince every four years.

Presumably the reason the headline writer felt justified in calling the race a toss-up was this paragraph in the story: "The final Dispatch poll shows Obama leading 50 percent to 48 percent in the Buckeye State. However, that 2-point edge is within the survey's margin of sampling error, plus or minus 2.2 percentage points."

That wording suggests that Obama's two-point edge has no meaning. And that's a common way for journalists to interpret results that fall within the "margin of error." For example, in September a conservative columnist in the New York Post asserted that Obama's lead in state polls didn't matter because the "polls separating the two candidates are within the margin of error -- meaning that there is no statistical difference in support between Obama and Romney."

Explaining why it's wrong to say there's "no statistical difference" in such cases will take a couple of paragraphs, so please bear with me (unless you recently took a stats course, in which case you can skip these paragraphs if not more).

Here's what pollsters never tell you, except maybe in the fine print: When they say there's a margin of error of X, they don't mean there's no chance whatsoever that the poll is off by more than X. Typically, their margin-of-error calculations are based on a confidence level of 95 percent, which means the chances of the poll being off by more than X are 5 percent.

In the Columbus Dispatch poll, X was 2.2. So if the poll had found Obama was ahead by 2.21 points, the finding would have been "outside the margin of error" and thus treated with great respect -- but the fact is that there would still have been a 5 percent chance that, in the actual voting population, Romney was ahead. (I'm assuming the Dispatch followed convention and used the 95 percent confidence threshold in calculating its margin of error -- see postscript below.) The flip side of this coin is that the Obama lead of 2 points in the poll, though less than the margin of error of 2.2 points, is by no means devoid of significance. If you did the math -- which, many years after taking quantitative analysis in college, I lack the brain cells to do precisely -- you'd find that there's a probability of somewhere in the neighborhood of 90 percent that Obama is ahead in the voting population as a whole.

Now, we generally reserve the term "toss-up" for odds in the vicinity of 50-50 -- since the term, after all, refers to a coin toss. So when the odds are more like 90-10, it's a bit misleading for the Dispatch to use the term. I mean, you could define "toss-up" that loosely, but then you could also define it loosely enough to include 95-5 odds or 96-4 odds, in which case even an Obama lead that fell outside the "margin of error" as conventionally defined could be deemed a "toss-up."

My main point is just that the definition of "margin of error" is a bit arbitrary -- a line drawn somewhere on a continuum of probabilities. (In fact, some pollsters use a 90-percent confidence level in calculating the margin of error, though most now use the 95-percent standard.) Yet we treat the "margin of error" as some binary thing -- as if numbers that fall inside it are meaningless ("a statistical tie") and numbers that fall outside it deserve complete confidence. No polling result ever deserves complete confidence, because the only way to get the confidence level up to literally 100 percent is to sample the entire voting population. We call that poll "election day."

I should add two asterisks, one of which will make Obama voters feel more confident of victory in Ohio, and one of which will make them feel less so. First the confidence booster:

So many polls are being taken in Ohio these days that you can aggregate them and considerably reduce your margin of error, since the size of the total sample population becomes much higher than the size of the Dispatch poll. And if you do that, you'll discover that, actually, the average such poll shows Obama ahead by more than the 2 points found in the Dispatch poll. So for Obama supporters it's a twofer: The size of Obama's lead grows, and the margin of error shrinks.

Jump to comments
Presented by

Robert Wright is the author of The Evolution of God and a finalist for the Pulitzer Prize. He is a former senior editor at The Atlantic.

Get Today's Top Stories in Your Inbox (preview)

CrossFit Versus Yoga: Choose a Side

How a workout becomes a social identity


Join the Discussion

After you comment, click Post. If you’re not already logged in you will be asked to log in or register. blog comments powered by Disqus

Video

CrossFit Versus Yoga: Choose a Side

How a workout becomes a social identity

Video

Is Technology Making Us Better Storytellers?

The minds behind House of Cards and The Moth weigh in.

Video

A Short Film That Skewers Hollywood

A studio executive concocts an animated blockbuster. Who cares about the story?

Video

In Online Dating, Everyone's a Little Bit Racist

The co-founder of OKCupid shares findings from his analysis of millions of users' data.

Video

What Is a Sandwich?

We're overthinking sandwiches, so you don't have to.

Video

Let's Talk About Not Smoking

Why does smoking maintain its allure? James Hamblin seeks the wisdom of a cool person.

Writers

Up
Down

More in Politics

Just In