What Does Nate Silver Know?

The famous data journalist thinks the media are making the same mistakes this year as they did in 2016.

Daniel Savage

In November, I visited FiveThirtyEight’s offices in New York on picture day. For journalists who style themselves as nerds, the formal photo shoot was a mild form of torture. Nate Silver, the site’s founder, donned a blazer, forced a smile for his headshot, then snuck away to get back to work on the site’s 2020 primary forecast. Though FiveThirtyEight now has a staff of about 35, covering sports, pop culture, and more, the site’s essential element is still the elaborate models Silver himself builds to predict elections.

Silver, a former management consultant and professional poker player, got into the political-forecasting business in 2007, after growing frustrated by coverage of the Democratic primary on cable news. He could scarcely believe how bad the analysis was—based on little more than hunches and hoary wisdom, and either ignoring opinion polls or misusing them to create false narratives of momentum.

Exasperated by the guesswork of pundits, Silver championed the more objective science of polling. He aggregated polls, grading and weighting them to predict the outcome of the election—an egalitarian project that sought to replace the opinionating of insiders with quantitative analysis of voter sentiment. Silver’s wonky assurance seemed of a piece with the professorial cool of Barack Obama, whose victory he predicted in 2008, and again in 2012, when FiveThirtyEight correctly forecast the results in every state.

Then came 2016. Like most journalists, Silver initially underestimated Donald Trump, dismissing his chances of winning the Republican nomination. It was a rare embarrassment, one that Silver attributed to losing sight of a fundamental principle: Trust the polls. Trump had consistently led in surveys of GOP voters, but Silver had succumbed to the conventional wisdom that the interloper couldn’t possibly prevail.

By the eve of the general election, Silver had come to believe that Trump had a path to victory. FiveThirtyEight predicted that he had a 29 percent chance of winning—significantly higher than the predictions of The New York Times’ The Upshot (15 percent) or the Princeton Election Consortium’s Sam Wang (7 percent). Ryan Grim of HuffPost accused Silver of inflating Trump’s chances. Citing HuffPost’s prediction that Hillary Clinton had a 98 percent chance of winning, Grim wrote that if you have faith in the numbers, “you can relax. She’s got this.”

She did not, in fact, have it. After Trump’s victory, pollsters and prognosticators became targets of derision. Critics alleged that rapid partisan realignment, unpredictable voter turnout, and the demise of the landline had rendered poll-based predictions obsolete. Though he had been savaged days earlier for overestimating Trump’s chances, Silver, as the leader of the data revolution, now absorbed criticism for its failure to foresee Trump’s victory. “The entire 2016 campaign season was … characterized by a series of spectacular Silver blunders,” read a typical critique, in Current Affairs. It ran under the headline “Why You Should Never, Ever Listen to Nate Silver.”

As the 2020 race begins in earnest, the question of whether to listen to Nate Silver returns to the fore, which is why I was visiting FiveThirtyEight. Silver believes he got 2016 right—it’s everyone else who got it wrong, and in ways that could lead the media to get 2020 wrong as well. “I think the 2016 campaign exposed whatever your bad habits were as a newsroom,” Silver told me. “But no one actually seems to have learned very many lessons in 2016.”

Silver insists that polling is still up to the task of measuring voter sentiment in the Trump era. In 2016, national polls found Clinton leading by three points on average. In fact, she won the national popular vote by about two percentage points—making those polls more accurate than they had been in 2012. State polls fared worse—some overestimated Clinton’s support, while others underestimated it—but they weren’t bad by historical standards. (Silver arrived at Trump’s 29 percent chance of winning—roughly the same chance the campaign gave itself—by accounting in his model for possible variation in state polls.) Polling in the 2018 midterm elections proved highly accurate, correctly anticipating the wave of Democratic victories that handed the party control of the House of Representatives. “In some ways, polling is the only way in which the Trump presidency has been normal,” Silver said.

As he sees it, the problems stem not from the polls but from how the press interprets them. During the long run-up to the 2020 primary season, he saw pundits fall into familiar traps. The same sort of commentators who expected Trump to collapse four years ago have consistently predicted a Joe Biden implosion that, as of this writing, has yet to happen—perhaps in part because Biden’s core supporters, like Trump’s, are members of demographics underrepresented in the press (for Trump, non-college-educated voters and rural voters; for Biden, non-college-educated voters and black voters). Despite Biden’s durable lead, the press has been quick to crown a series of front-runners in waiting, from Kamala Harris to Elizabeth Warren to Pete Buttigieg—all while largely ignoring Biden’s most persistent rival for the top spot in the polls: Bernie Sanders.

To locate story lines where they don’t exist, commentators seize on outlier polls, like the one from Monmouth University in August that suggested a closer race than any previous survey had. (That single snapshot was covered so breathlessly that the director of the university’s polling institute took the rare step of publicly noting how much it deviated from the others.) Or pundits rely instead on what Silver described to me as “stylized facts”: A strong debate performance or fundraising quarter will kick off a round of coverage of a candidate’s supposed surge, even if polls don’t detect much movement. It’s not that these factors don’t matter—they do—but Silver’s work suggests that they don’t matter nearly as much as most journalists imagine. By Silver’s estimation, the average debate performance moves polls about as much as an average week on the trail—and, as Senator Harris can attest, even a well-received moment can take a candidate only so far.

What does this mean for coverage of the general election? Regardless of who emerges as the Democratic nominee, 2020 will have a different complexion from 2016: Trump is now an incumbent, not a curiosity, and his opponent won’t be Hillary Clinton. But a tight race with a polarized electorate offers plenty of chances to repeat common mistakes.

In Silver’s view, the media were overconfident in a Clinton victory because of long-held assumptions about the mechanics of American politics. Take the “ground game”—the business of identifying voters and getting them to the polls. Some pundits initially argued that if the election was close, Clinton’s superior campaign organization would put her over the top; then, after she lost, many flogged her for failing to get out the vote in key states. Yet decades of political science suggest that such tactics have a relatively minor effect on election results. Based on his analysis of late movement in the race, Silver argues that factors mostly beyond Clinton’s control mattered far more than the success or failure of her canvassers in the Upper Midwest. These factors included Trump’s ability to command the news cycle—Silver has found that earned media is far more valuable than the kind you can buy—and James Comey’s belated reopening of the FBI investigation into Clinton’s use of a private email server, which had a measurable impact on polling.

In part because of Trump’s already prodigious fundraising, the old temptation to overstate the importance of the campaign war chest—and the electioneering efforts it can buy—will once again be present in 2020. As will the press’s quadrennial weakness for the supposedly game-changing gaffe, another Silver bête noire. Obama’s “lipstick on a pig” flap, Clinton’s “deplorables” remark—such moments may reveal aspects of a candidate’s character, but Silver believes the media consistently exaggerate their effect on voter behavior.

On other fronts, Silver sees the press as overlearning the lessons of 2016. Journalists have obsessed over Russian interference in elections, and while foreign meddling is undoubtedly troubling, Silver sees no strong evidence that hacking or disinformation swung the 2016 result. “There’s a bias toward believing in explanations that involve secrecy or things happening that are hidden,” Silver said. The most important factors, he believes, are right there in the numbers.

Silver is now the elder statesman of a growing class of data-based journalists, including his frenemy Nate Cohn of The Upshot, with whom he likes to spar. Despite the beating forecasters took after 2016, quantitative analysis is better integrated into contemporary political coverage than it was before. FiveThirtyEight’s own primary-election model is wildly complex, evidence of Silver’s continued faith in the power of data. For all his success, however, Silver frets that his work is not well understood.

During my visit, the FiveThirtyEight staff was in the middle of a debate about how to present election forecasts so the general public can easily comprehend them. Four years ago, the site gave readers a percentage chance that each candidate would win. For the 2018 midterms, it switched to odds: Democrats had a 7-in-8 chance of taking the House; Republicans had a 4-in-5 chance of holding the Senate. Silver’s hunch is that readers find odds easier to understand than percentages, because they can imagine a series of coin flips or rolls of the dice.

Delivering forecasts in the clearest way possible is undoubtedly to the good; studies have demonstrated that the public doesn’t grasp probability well. But it’s hard to believe that FiveThirtyEight would have been spared opprobrium had it predicted Trump had a 3-in-10 chance of winning. And convincing readers to trust the model again is more than just a math problem.

Silver’s celebrity was never entirely a function of his accuracy. It helped that his predictions were congenial to Obama, and to liberals more generally. (Silver’s name became a staple of Obama-era Democratic fundraising emails, his predictions—sunny and dire alike—invoked to persuade donors to pony up.) His own political views, however, have always tended toward the libertarian and moderate, and as Democrats have moved to the left, Silver’s heresies have drawn new scrutiny. In October, he chastised “Libs” for not giving Trump credit for the death of the Islamic State’s leader, Abu Bakr al-Baghdadi. “Nate Silver has been morphing before our eyes into exactly the kind of bloviator he made his name mocking,” The New Republic announced.

But it’s Silver’s devotion to quantitative analysis as much as his politics that has put him at odds with our post-truth moment. Trump’s disregard for facts is singular, but the left, too, has grown more suspicious of technocrats and their pronouncements. Many liberals today see Obama as a president whose achievements were hobbled by bloodless calculations of what seemed possible. They now seek candidates who make grand ideological gestures, even if the math is fuzzy. Silver, to at least some extent, is an emblem of an era when it felt as if any problem could be solved with enough elite brainpower.

After Trump’s victory, the major media organizations flagellated themselves for spending too little time with the voters who elected him. To make amends, they sent reporters to Trump country, seeking to understand the sources and strength of the president’s support by lingering in Ohio diners and Pennsylvania factories. The introspection was overdue, and it tracked with Silver’s long-standing belief that journalists spend too much time talking to one another. But this reporting is by definition anecdotal and, to borrow Silver’s term, stylized—an implicit rejection of more analytic approaches to understanding the electorate.

For Silver, no number of dispatches from the heartland can deliver the insights that hard data can. “The impulse maybe isn’t bad,” he told me. “But, you know, polls are also a way of talking to voters.”

This article appears in the March 2020 print edition with the headline “Can You Still Trust Nate Silver?”