When Polls Collide

Recent polls have shown President Bush with a big lead, or a small lead, or no lead in the presidential race. What's going on with the polls this year?

Nothing strange, pollsters say. "It's not at all unusual that there would be some churn and turmoil out there among voters," said Frank Newport, editor-in-chief of the Gallup Poll. "After all, they're supposed to be rethinking the race and changing their minds." With more polls than ever measuring that turmoil, expect to see a lot of variation from one poll to the next.

But some things about 2004 do appear to be different. "There seems to be higher than normal interest in this election," said Mark DiCamillo, director of California's Field Institute. "That may indicate actual turnout will increase by maybe 5 or 6 points. You might be getting people who are first-time voters, people who don't usually vote."

When polls estimate who is likely to vote, they take two things into account. One is past voting behavior, which has always been a good predictor. Pollsters take care not to penalize young people who were ineligible to vote in previous elections. The bigger problem is finding those young people, who are less likely to be at home and are, therefore, harder to reach by telephone. Since most polls use telephone surveys, pollsters have been figuring out ways to deal with that problem for decades.

The second predictor of a likely voter is interest in the election. Newly energized voters should make it into the likely-voter sample even if they have never voted, provided they show a high level of interest. A recent study by The New York Times revealed record numbers of new voter registrations this year in Democratic areas of Florida and Ohio, key battleground states. Some Democrats think that the likely-voter models used in most polls might not be picking up those new registrants.

It is true that voters with no history of participation have a harder time passing the likely-voter threshold. But that is not a flaw in polling. It reflects reality. Newly registered voters are typically the least likely to show up at the polls. So we don't yet know how much of a payoff the Democrats will get from their registration efforts.

Some observers think that the growing number of people who won't talk to poll takers is making polls unreliable. But pollsters claim that it's not a problem—precisely because non-cooperation is so widespread. As DiCamillo explained, "The nonresponders are pretty much randomly distributed across the population. It's not any one population group that we're missing."

What about the growing use of cellphones? Pollsters cannot place calls to cellphones because most users don't want to pay for a call from a stranger. But the overwhelming majority of American households still have land lines. Estimates are that only about 3 percent of U.S. households rely totally on cellphones, and these are concentrated mostly in the younger age segment that is least likely to vote. "My sense is that those with only cellphones are different from those in the same age group with land lines," Cliff Zukin, professor of public policy at Rutgers University, told The Wall Street Journal. "They're probably more mobile, more urban." Gallup's Newport said, "We're monitoring [the phenomenon]. If it came to a point where we couldn't interview on cellphones, and cellphone-only households were very widespread, we would quit our telephone survey methodology."

Close observers of the polls this year have noticed a lot of variation in party identification from one poll to the next. Can the number of self-described Democrats and Republicans fluctuate so quickly? Pollsters say yes. According to Andrew Kohut, director of the Pew Research Center for the People and the Press, "Party identification is not a particularly stable attitude. Many people go back and forth on how they regard themselves." He found that nearly one in five voters shifted partisanship between September and November 2000, mostly moving back and forth between identifying with a party or calling themselves independent.

Newport argues that party identification is not a demographic characteristic like age or gender, whose basic distribution is known. He calls partisanship a "trailing" variable, like candidate preference, that changes in response to forces in the political environment. "We think it moves in line with the ballot," Newport said.

Most pollsters reject the idea of weighting poll results to reflect a fixed distribution of partisanship. There is no national registration by party, so the actual numbers of Democrats and Republicans are unknown. Weighting by party would certainly reduce the variability of polls. But it would make polls less responsive to the campaign. In Kohut's view, "Any efforts to 'fix' the partisan balance would be misreading the flow of public opinion."

So what can a poor befuddled voter do when confronting poll results that vary widely? "The recommendation I give to lay people when you have diverging polls is just to wait for a couple of additional polls to come out and average them over the course of a week," DiCamillo says. In other words, do a "poll of polls" to get a better idea of what's going on.

A profusion of polls means that we are more likely to see discrepancies from one poll to the next. But having many different surveys is likely to produce a more reliable average. So, in the end, the problem is also the solution.