Washington November 2006

Do Polls Still Work?

The last two elections have left pollsters somewhat bloodied but unbowed
More

May I be the first to say ‘Mr. President’?” inquired Bob Shrum, rather too memorably, of John Kerry on Election Day 2004, as early exit polls showed his candidate cruising to victory. It turned out Shrum’s faith was misplaced—so badly that the episode ranks alongside “Dewey Defeats Truman” in the annals of Great Political Embarrassments.

After faulty reading of exit polls caused the Election Day 2000 debacle, this latest confusion suggested a deep futility in even trying to gauge public opinion, before or after an election.

With this month’s elections shaping up to be another nail-biter, the question “Do polls still work?” is increasingly practical. Pollsters no longer seem like wizards. It’s no secret that today they must contend with skyrocketing refusal rates and obstacles like “cell-phone-only households.” So partisans feel free to criticize them—especially when their side is losing.

The last election season saw oddly technical attacks: the liberal group MoveOn.org attacked the Gallup organization in The New York Times over its “likely voter methodology.” And startlingly vehement ones: Republicans picketed the Minneapolis Star Tribune, pounding on windows and hollering at employees, for insufficiently weighting conservatives in its survey samples.

Pollsters themselves freaked out. Contrary to their partisan caricature, legitimate practitioners of the calling are a docile and introverted breed consumed with little beyond their own narrow subspecialties. If you don’t know one, a good example is William Hurt’s character in Altered States—the rumpled academic so engrossed in his research that he fails to notice when his marathon stints in a sensory-deprivation tank transform him into a rampaging man-beast.

The polling community reached for its heavy artillery: the strongly worded journal article. Last year, Public Opinion Quarterly, the premier journal in the field, devoted a special issue to an autopsy of the 2004 preelection polls, and proved the naysayers wrong. The studies found that the polls had been among the most accurate in half a century, and concluded that blame for the mess lay elsewhere: reporters—always suckers for drama—had treated statistically insignificant day-to-day shifts as major stories rather than predictable aberrations, thereby heightening the up-one-day, down-the-next whiplash effect that tends to undermine confidence in any poll. No wonder the public was angry! For all the criticism, one author observed, most polls “were among the best recorded in the contemporary period.” Gallup’s last poll gave Bush a 2-point lead; he won by 2.5 percent. It was, as the issue’s coeditor, Lawrence Jacobs, put it, “a kick to the groin of conventional myths about polling.”

And what about Bob Shrum and the exit polls? As a series of postelection studies took pains to note, the self-styled experts drastically misread the raw exit-poll data that leaked on Election Day. Instead of seizing on the early numbers and declaring a winner, experts now tell us, bloggers and the press should have waited for all the data to be properly weighted and balanced.

The National Election Pool—the outfit that runs the exit polls—has made one useful concession to reality and set out to stop leaks from racing across the Web, hitting upon an ingeniously low-tech solution to a high-tech problem. This year the pollsters will be placed in a “quarantine room” and stripped of cell phones, pagers, Internet access, and other means of outside communication. They won’t be allowed back into the electronic real world until many hours later.

If all goes according to plan—and if the pollsters locked in the tank don’t turn into rampaging man-beasts—the results we hear later that evening will be as accurate as they should be.

Joshua Green is a senior editor at The Atlantic.
Jump to comments

Joshua Green is a former writer and editor at The Atlantic.

Get Today's Top Stories in Your Inbox (preview)


Elsewhere on the web

Join the Discussion

After you comment, click Post. If you’re not already logged in you will be asked to log in or register. blog comments powered by Disqus

Video

Miami: The Next Big Start-Up City?

How the city became a center for innovation

Video

Video

A Brief History of Romantic Comedies

From The Atlantic's Chris Orr

Video

Life in 'the New Arctic'

A moving portrait of a fading landscape

Video

Video

The Rise of New York City

A fascinating look at Manhattan in the 1940s

Video

What Is Methane Hydrate?

"Flaming ice" is a vast natural energy source

Video

NASA's Time-Lapse of the Sun

Now with epic dubstep music

Video

Shaken Not Tuned: Cocktail Experiments

Can a tuning fork improve a cocktail?

Video

Video

Is He Cheating? A 1950s Guide

'That little blonde secretary from the office?’

Video

New Yorkers: Vintage Vacuum-Tube Amps

Risking electric shock to restore old amplifiers

Video

The DIY Piano-Bicycle

Everybody needs a hobby

Writers

Up
Down

More in Politics

More back issues, Sept 1995 to present.

In Focus

Photos of Tornado Damage in Moore, Oklahoma

Just In