Is Ohio a 'Toss-Up'?

More
Columbus.JPG

The Columbus Dispatch, the only newspaper in Ohio's biggest city, has declared Ohio a "toss-up" between President Obama and Mitt Romney. This will please the Romney camp, which has been fighting hard against the "Ohio is Obama's rock-solid firewall" narrative.

For reasons I'll explain, this headline is misleading. And it's tempting to think it's intentionally misleading. After all, the Dispatch endorsed Romney (and in fact hasn't endorsed a Democrat for president since 1916). And the lead paragraph of the story under the "toss-up" headline does have a certain Mitt Romney-pep-rally quality to it: "The 'Ohio firewall' precariously stands for President Barack Obama, but a strong Republican turnout could enable Mitt Romney to tear it down on Election Day."

Still, I think we can give the Dispatch the benefit of the doubt and assume that its own firewall -- the church-state separation that is supposed to keep a newspaper's editorial stance from coloring its reporting -- is intact. Because there's a simpler explanation for the "toss-up" headline: It rests on the same slightly-too-simple way of thinking about polling that lots of reporters and other Americans evince every four years.

Presumably the reason the headline writer felt justified in calling the race a toss-up was this paragraph in the story: "The final Dispatch poll shows Obama leading 50 percent to 48 percent in the Buckeye State. However, that 2-point edge is within the survey's margin of sampling error, plus or minus 2.2 percentage points."

That wording suggests that Obama's two-point edge has no meaning. And that's a common way for journalists to interpret results that fall within the "margin of error." For example, in September a conservative columnist in the New York Post asserted that Obama's lead in state polls didn't matter because the "polls separating the two candidates are within the margin of error -- meaning that there is no statistical difference in support between Obama and Romney."

Explaining why it's wrong to say there's "no statistical difference" in such cases will take a couple of paragraphs, so please bear with me (unless you recently took a stats course, in which case you can skip these paragraphs if not more).

Here's what pollsters never tell you, except maybe in the fine print: When they say there's a margin of error of X, they don't mean there's no chance whatsoever that the poll is off by more than X. Typically, their margin-of-error calculations are based on a confidence level of 95 percent, which means the chances of the poll being off by more than X are 5 percent.

In the Columbus Dispatch poll, X was 2.2. So if the poll had found Obama was ahead by 2.21 points, the finding would have been "outside the margin of error" and thus treated with great respect -- but the fact is that there would still have been a 5 percent chance that, in the actual voting population, Romney was ahead. (I'm assuming the Dispatch followed convention and used the 95 percent confidence threshold in calculating its margin of error -- see postscript below.) The flip side of this coin is that the Obama lead of 2 points in the poll, though less than the margin of error of 2.2 points, is by no means devoid of significance. If you did the math -- which, many years after taking quantitative analysis in college, I lack the brain cells to do precisely -- you'd find that there's a probability of somewhere in the neighborhood of 90 percent that Obama is ahead in the voting population as a whole.

Now, we generally reserve the term "toss-up" for odds in the vicinity of 50-50 -- since the term, after all, refers to a coin toss. So when the odds are more like 90-10, it's a bit misleading for the Dispatch to use the term. I mean, you could define "toss-up" that loosely, but then you could also define it loosely enough to include 95-5 odds or 96-4 odds, in which case even an Obama lead that fell outside the "margin of error" as conventionally defined could be deemed a "toss-up."

My main point is just that the definition of "margin of error" is a bit arbitrary -- a line drawn somewhere on a continuum of probabilities. (In fact, some pollsters use a 90-percent confidence level in calculating the margin of error, though most now use the 95-percent standard.) Yet we treat the "margin of error" as some binary thing -- as if numbers that fall inside it are meaningless ("a statistical tie") and numbers that fall outside it deserve complete confidence. No polling result ever deserves complete confidence, because the only way to get the confidence level up to literally 100 percent is to sample the entire voting population. We call that poll "election day."

I should add two asterisks, one of which will make Obama voters feel more confident of victory in Ohio, and one of which will make them feel less so. First the confidence booster:

So many polls are being taken in Ohio these days that you can aggregate them and considerably reduce your margin of error, since the size of the total sample population becomes much higher than the size of the Dispatch poll. And if you do that, you'll discover that, actually, the average such poll shows Obama ahead by more than the 2 points found in the Dispatch poll. So for Obama supporters it's a twofer: The size of Obama's lead grows, and the margin of error shrinks.

Jump to comments
Presented by

Robert Wright is the author of, most recently, the New York Times bestseller The Evolution of God and a finalist for the Pulitzer Prize. He is a former writer and editor at The Atlantic. More

Wright is also a fellow at the New America Foundation and editor in chief of Bloggingheads.tv. His other books include Nonzero, which was named a New York Times Book Review Notable Book in 2000 and included on Fortune magazine's list of the top 75 business books of all-time. Wright's best-selling book The Moral Animal was selected as one of the ten best books of 1994 by The New York Times Book Review.Wright has contributed to The Atlantic for more than 20 years. He has also contributed to a number of the country's other leading magazines and newspapers, including: The New Yorker, The New York Times Magazine, Foreign Policy, The New Republic, Time, and Slate, and the op-ed pages of The New York Times, The Washington Post, and The Financial Times. He is the recipient of a National Magazine Award for Essay and Criticism and his books have been translated into more than a dozen languages.

Get Today's Top Stories in Your Inbox (preview)

Juice Cleanses: The Worst Diet

A doctor tries the ever-popular Master Cleanse. Sort of.


Elsewhere on the web

Join the Discussion

After you comment, click Post. If you’re not already logged in you will be asked to log in or register. blog comments powered by Disqus

Video

Juice Cleanses: The Worst Diet

A doctor tries the ever-popular Master Cleanse. Sort of.

Video

Why Did I Study Physics?

Using hand-drawn cartoons to explain an academic passion

Video

What If Emoji Lived Among Us?

A whimsical ad imagines what life would be like if emoji were real.

Video

Living Alone on a Sailboat

"If you think I'm a dirtbag, then you don't understand the lifestyle."

Feature

The Future of Iced Coffee

Are artisan businesses like Blue Bottle doomed to fail when they go mainstream?

Writers

Up
Down

More in Politics

Just In