Political Pollsters Still Don't Know What They're Doing
Many of the problems behind inaccurate surveys in last year's presidential election are surfacing again in the midterm races.
Few polls have signaled Democratic peril this year quite like those in Colorado, the purple state that twice helped push Barack Obama into the White House. Just look at Quinnipiac's surveys throughout 2013. They've found Obama's public approval in Colorado ranging from the low 40s to the mid 30s; formerly high-flying Gov. John Hickenlooper in statistical dead heats with some flawed GOP opponents; and Sen. Mark Udall well short of safe in his first reelection bid. And, looking out to 2016, the statewide polls have Vice President Joe Biden badly trailing in a match against one of the most polarizing Republicans in America, Sen. Ted Cruz of Texas.
Clearly Democrats have a problem in this swing state. But a bigger problem for Democrats and Republicans alike is polling itself. It's not that pollsters are failing to accurately measure what people really think. They are still quite good at that. The recurring and increasingly disruptive problem is that the polling industry sometimes struggles to reach the right mix of people.
This could be the case in Colorado, where the surveys have a very different makeup than what exit polls suggest is the typical Colorado electorate. In Quinnipiac's Colorado surveys, whites without a college degree outnumber college-educated whites as a share of the electorate — 43 percent to 35 percent in the most recent poll. But the exit polls that measured who actually voted in recent state elections have consistently found more degree-holders than not among white voters. Those exits, which have varied, put the gap at 40 percent with degrees to 37 percent without in 2012 and as high as 56 percent with and 25 percent without in 2010.
What this means is that the polls in Colorado appear to be putting too much weight on the views of Republican-leaning voters, and thus exaggerating Democrats' struggles in the state.
The discrepancy has not gone unnoticed by Colorado Democrats. Poll "unskewing" became a popular — and widely mocked — partisan hobby during the 2012 presidential election, but many of those critiques focused on the number of self-identified Democrats or Republicans, an attitude that changes over time, included in particular polls. The number of college-degree holders is, if not a fixed point, at least a demographic measurement that should be slower to evolve. Colorado's voters probably won't look much different on that score in 2014 than they did collectively in 2012, 2010, or even 2008.
And although the difference is small, the makeup of those polled affects the results. Quinnipiac's latest survey showed Hickenlooper mired around 45 percent support for his reelection, a danger zone for incumbents. But the crosstabs — which break down the results along demographic and partisan lines — show him drawing levels of support from college and noncollege whites close to where exit polls measured him in the 2010 election. And he won that one with 51 percent of the state's votes.
Now, take Colorado's polling issues and apply them across the country, and the significance for 2014 and 2016 begins to settle in. How to measure the views of the right mix of people is a growing problem for the polling industry, especially in an era of turnout-focused elections. Research has shown that all Americans are getting harder for pollsters to reach but some more so than others, making it more difficult to determine who is likely to vote and thus what an election's final result will look like. The GOP's well-documented polling miscues leading up to the 2012 election stemmed partly from the belief that the electorate would be whiter than it proved to be.
Even the after-action political gold mines of exit polls are far from infallible. The rise of early voting and voting by mail has made it more difficult to identify voters' "official" opinions. In Colorado, more than three-quarters of voters cast their ballots by mail in 2012, and that might explain some major variance in the exit polls' college/noncollege splits. (Again, though, none has shown noncollege whites outnumbering college whites.)
One way that some pollsters try to combat these issues is by contacting their respondents using lists of known voters, instead of dialing phone numbers randomly. Quinnipiac is using a different but perfectly reasonable method to measure Colorado voters' opinions. Its pollsters randomly dial phone numbers, gather information such as race and educational attainment from respondents, and weight that sample of adults so that it reflects what census data say Colorado actually looks like. The adults who say they are registered voters also answer political questions, such as who they'll support for governor in 2014, and those registered voters simply include more blue-collar whites than college-educated ones, says Douglas Schwartz, director of the Quinnipiac University Poll.
"I don't come at it from the vantage point of, do we have too many noncollege-educated whites," Schwartz said. "I come from the vantage point that we're just weighting to what the census is telling us should be there."
For sure, even if Quinnipiac's measurements skew too blue-collar, the news is still not good for Colorado Democrats. The institute also measured opinions about Hickenlooper in 2012, before he began struggling politically, and his approval rating has plunged by double digits since then.
"There's lots of observations to be made, including that [the most recent Quinnipiac] poll may not be perfectly reflective of what 2014 may look like," says Colorado political analyst Floyd Ciruli. "But it's picking up on what's going on out there," as state Democrats have faced a backlash this year culminating in two state Senate recalls and the defeat of a Hickenlooper-backed ballot measure.
But in a political era when turnout is so important, slight changes in poll samples can dial up or down the degree of that backlash. Across the nation, and in Colorado, pollsters will struggle with how to calibrate this dial.