Washington November 2006

Do Polls Still Work?

The last two elections have left pollsters somewhat bloodied but unbowed
More

May I be the first to say ‘Mr. President’?” inquired Bob Shrum, rather too memorably, of John Kerry on Election Day 2004, as early exit polls showed his candidate cruising to victory. It turned out Shrum’s faith was misplaced—so badly that the episode ranks alongside “Dewey Defeats Truman” in the annals of Great Political Embarrassments.

After faulty reading of exit polls caused the Election Day 2000 debacle, this latest confusion suggested a deep futility in even trying to gauge public opinion, before or after an election.

With this month’s elections shaping up to be another nail-biter, the question “Do polls still work?” is increasingly practical. Pollsters no longer seem like wizards. It’s no secret that today they must contend with skyrocketing refusal rates and obstacles like “cell-phone-only households.” So partisans feel free to criticize them—especially when their side is losing.

The last election season saw oddly technical attacks: the liberal group MoveOn.org attacked the Gallup organization in The New York Times over its “likely voter methodology.” And startlingly vehement ones: Republicans picketed the Minneapolis Star Tribune, pounding on windows and hollering at employees, for insufficiently weighting conservatives in its survey samples.

Pollsters themselves freaked out. Contrary to their partisan caricature, legitimate practitioners of the calling are a docile and introverted breed consumed with little beyond their own narrow subspecialties. If you don’t know one, a good example is William Hurt’s character in Altered States—the rumpled academic so engrossed in his research that he fails to notice when his marathon stints in a sensory-deprivation tank transform him into a rampaging man-beast.

The polling community reached for its heavy artillery: the strongly worded journal article. Last year, Public Opinion Quarterly, the premier journal in the field, devoted a special issue to an autopsy of the 2004 preelection polls, and proved the naysayers wrong. The studies found that the polls had been among the most accurate in half a century, and concluded that blame for the mess lay elsewhere: reporters—always suckers for drama—had treated statistically insignificant day-to-day shifts as major stories rather than predictable aberrations, thereby heightening the up-one-day, down-the-next whiplash effect that tends to undermine confidence in any poll. No wonder the public was angry! For all the criticism, one author observed, most polls “were among the best recorded in the contemporary period.” Gallup’s last poll gave Bush a 2-point lead; he won by 2.5 percent. It was, as the issue’s coeditor, Lawrence Jacobs, put it, “a kick to the groin of conventional myths about polling.”

And what about Bob Shrum and the exit polls? As a series of postelection studies took pains to note, the self-styled experts drastically misread the raw exit-poll data that leaked on Election Day. Instead of seizing on the early numbers and declaring a winner, experts now tell us, bloggers and the press should have waited for all the data to be properly weighted and balanced.

The National Election Pool—the outfit that runs the exit polls—has made one useful concession to reality and set out to stop leaks from racing across the Web, hitting upon an ingeniously low-tech solution to a high-tech problem. This year the pollsters will be placed in a “quarantine room” and stripped of cell phones, pagers, Internet access, and other means of outside communication. They won’t be allowed back into the electronic real world until many hours later.

If all goes according to plan—and if the pollsters locked in the tank don’t turn into rampaging man-beasts—the results we hear later that evening will be as accurate as they should be.

Joshua Green is a senior editor at The Atlantic.
Jump to comments
Presented by

Joshua Green is a former senior editor at The Atlantic.

Get Today's Top Stories in Your Inbox (preview)

Adventures in Legal Weed

Colorado is now well into its first year as the first state to legalize recreational marijuana. How's it going? James Hamblin visits Aspen.


Elsewhere on the web

Join the Discussion

After you comment, click Post. If you’re not already logged in you will be asked to log in or register. blog comments powered by Disqus

Video

Adventures in Legal Weed

Colorado is now well into its first year as the first state to legalize recreational marijuana. How's it going? James Hamblin visits Aspen.

Video

What Makes a Story Great?

What makes a story great? The storytellers behind House of CardsThis American LifeThe Moth, and more reflect on the creative process.

Video

Tracing Sriracha's Origin to Thailand

Ever wonder how the wildly popular hot sauce got its name? It all started in Si Racha.

Video

Where Confiscated Wildlife Ends Up

A government facility outside of Denver houses more than a million products of the illegal wildlife trade, from tigers and bears to bald eagles.

Video

Is Wine Healthy?

James Hamblin prepares to impress his date with knowledge about the health benefits of wine.

Video

The World's Largest Balloon Festival

Nine days, more than 700 balloons, and a whole lot of hot air

Writers

Up
Down

More in Politics

More back issues, Sept 1995 to present.

Just In