What Was the Problem With the Wisconsin Exit Polls?

Leaked early exit data briefly gave Democrats hope they'd won the recall Tuesday night. How could a survey of the real electorate be so far off?

For political junkies, election-night exit polls are like crack: a short-lived high that often proves ultimately destructive.

Such was the case in Wisconsin on Tuesday, when those watching the results of the gubernatorial recall election seized on early exit poll results that purported to show a much closer race than pre-election surveys forecast. The early exits put the race between Scott Walker and Tom Barrett tied, 50-50, sending reporters, operatives, and, most of all, Democrats into a tizzy.

One labor operative, whose identity I will protect here for compassionate purposes, sent a triumphal email to the media five minutes after the polls closed, with the subject line "what 50/50 exit polls already show": "This already shows that any idea of a Walker blow out, a stinging rebuke to labor, etc are all already moot. Field, GOTV and hard work have already taken a race we were 'supposed' to lose, and were vastly outspent in, and have it too close to call."

Oops. Within less than an hour, the networks would start to call the race for Walker in what quickly became apparent as a blowout win. It turned out that the only thing the early exit polls showed was that the early exit polls were wrong -- as they have been so many times before. Remember 2004, when early, leaked exits had the left prematurely inaugurating a President Kerry?

It's no wonder exit polls get so much attention -- they come at a time when a total information vacuum coincides with maximum appetite for information. People are voting, the election is about to be a foregone conclusion, yet nothing is happening, and those of us who have spent months or years anticipating this moment can do nothing but wait in a state of tortured anticipation.

But in seizing on exit polls to sate our desperate curiosity about who has won the election, we're using them for exactly what they're not supposed to tell us. "The exit poll is ultimately designed to provide answers about who voted and why, not to call the election," notes Mark Blumenthal, a former Democratic campaign pollster who's now the senior polling editor at the Huffington Post, and who spent nearly a year puzzling over and blogging about the mystery of the flawed Kerry exits post-2004.

In seizing on exit polls to sate our desperate curiosity about who has won the election, we're using them for exactly what they're not supposed to tell us.

Taken as such, exit polls are an incredibly rich trove of data for political analysts. In the case of Wisconsin, for example, they tell us that men preferred Walker by nearly 20 percentage points, while women gave Barrett a 5-point edge; that despite unions' institutional antagonism toward Walker he still carried nearly 40 percent of voters in union households; and that the same electorate that voted to retain Walker would have given the state's electoral votes to President Obama. Over the course of the GOP primary season, exit polls have shed light on which Republican voters were holding out on Mitt Romney -- the lower-income, less educated, tea party-supporting voters who preferred Newt Gingrich and Rick Santorum. Those are fascinating demographic insights, and their significance is enhanced by the fact that they come from an actual electorate that's gone out and voted, not a telephone-based guesstimate of who's going to vote. (In less developed countries, exit polls can also serve as a valuable check on election fraud.)

It's worth a refresher in how the exit poll -- there's actually only one, shared among all the news organizations -- works. A consortium of the five news networks plus the Associated Press commissions it from a firm called Edison Research, then sells a subset of the data to other news organizations. Edison carefully, scientifically chooses a set of precincts in geographically representative locations, then stations workers there to ask people who they voted for; the workers also note the roughly observable demographic characteristics (gender, age and race) of voters who don't answer the survey.

The networks get the raw data throughout the day, and though there's supposed to be an embargo until the polls close, the exits invariably leak. At one time, this was no big deal -- a rumor that would get around among political types but wouldn't reach the broader public. Blumenthal remembers getting calls every Election Day from reporters eager to gossip about this juicy inside information. But with the advent of the Internet, the early data can get out much more widely, causing a sensation. In 2004, some Democrats believed the early exits that showed them winning discouraged some Kerry voters from going to the polls because they thought it wasn't necessary.

Why are the early exits wrong? There are many possible reasons. First, like any survey, an exit poll is a random sample that carries a margin for error. Second, different people may be voting in the morning than later in the day, and some people may be more inclined to talk to an exit pollster than others. (For whatever reason, Blumenthal says, both these groups seem to lean Democratic, giving early exits a consistent leftward tilt.) Finally, the raw data from the exit survey gets crunched with the actual electorate that shows up in the official returns, and the sample gets adjusted to reflect the result.

Another complicating factor for exit pollsters is the increasing prevalence of early and mail voting in many states. In Wisconsin, about 15 percent of the vote is thought to have been early and absentee, something the exit poll wouldn't have captured; in some states, the exit poll is now being supplemented with telephone surveys to capture people who vote before Election Day. In Wisconsin, the absentee vote appears to have gone heavily for Walker.

In the end, the early Wisconsin exits were only a few points off in either direction -- the final margin was Walker 53, Barrett 46. But because they were off in a direction that predicted a different outcome, and because demoralized Democrats were desperate for a glimmer of hope, the flawed exit information took on a life of its own. For a brief time, Democrats' hopes soared, only to crash all the more devastatingly when the truth was revealed.