Is Political Science This Year's Election Casualty?
Lessons learned from the failures of predictive modeling
In the aftermath of Donald Trump’s unforeseen election win, there has been much anguished self-scrutiny on the part of Democratic party members, urbanites, intellectuals, upper-middle-class professionals, and others who could not imagine this political outcome. Many have rightly pointed to the fractured isolation of contemporary American life: city dwellers isolated from ruralites, whites from blacks, bourgeoisie from the working class, Republican from Democrat, and so on. No member of this democracy should remain untroubled by the erosion of dialogue and solidarity across the chasms dividing the country’s polity.
Yet one kind of isolation that played a small but very dramatic role in the elections is likely to receive comparatively less public attention. This is the practice of election forecasting—or building models of political reality by academics and statistics wonks that are said to scientifically predict and quantify the likelihood of future election outcomes. Election forecasting is part of a wider trend in higher education to present the study of politics as a “science.” But on November 8, 2016, the science of politics was almost uniformly and spectacularly wrong.
For example, on the cusp of the election, the Princeton Election Consortium (PEC) gave the astonishingly certain prediction that Hillary Clinton’s chances of winning were about 95 percent. On October 18, Samuel Wang, who helps lead the PEC, was tweeting to his followers: the election “is totally over. If Trump wins more than 240 electoral votes, I will eat a bug.”
Other influential prediction gurus—like Nate Silver at FiveThirtyEight and Nate Cohn of The New York Times’s Upshot—also predicted Clinton wins though in less absolutist terms (and without promises to eat insects). Nonetheless, watching Cohn’s forecaster on The New York Times front page for the few crucial hours of election night was the intellectual equivalent of suffering severe whiplash. At the beginning of the night, based on pre-election polls, the Upshot’s forecast called for an 85 percent chance for a Clinton victory (“about the same probability that an N.F.L. kicker misses a 37-yard field goal” the website helpfully explained). In less than a few hours, as actual results came in, the forecast flipped to a 95 percent chance of victory for Trump.
Devoted readers of The Upshot were left utterly flummoxed: How can the probability of Hillary winning go in just a couple hours from the chances of an N.F.L. kicker getting a routine field goal to less than that of drawing an ace from the top of a shuffled deck of cards? The turnout gave readers uninstructed in the technical science of statistics the impression of funny business. Combined with the emotional disappointment experienced by many, this has made for an emotional backlash against pollsters.
And already the robust questioning of the various polling and forecasting methodologies has begun. The American Association for Public Opinion Research has put together a committee to study what went wrong. Wonkish news outlets like Politico have run full articles calling out the failure of a predictive political science to materialize and offering potential reasons why. Most of the questioning has been over whether the models themselves used to forecast were the right ones. Were the right variables incorporated? Should there be less reliance on historical factors and more on demographic ones? Should economics figure into a model? Should the University of Southern California and Los Angeles Times’ use of a “panel model,” which relies on the same respondents longitudinally across time, instead be adopted?* After all, they did better than the other forecasters.
Inevitably, higher education and the mainstream in political science will follow the same line of self-criticism. And no one would be surprised if the American Political Science Association (a more-than 13,000-member association of professional, academic political scientists) invites Nate Cohn, Nate Silver, or Sam Wang to a panel discussion next summer to argue about how to make a model and tweak technical differences.
Yet when this happens, the larger philosophical questions—about whether the study of politics is indeed a science—will go unasked, and Americans will have missed a massive opportunity at self-correction in academia, the media, and society at large. To be clear, the problem is by no means mass surveys and polling (though these can always be improved). Polling, if used properly, can be an extremely helpful tool for gaining snapshots of widespread beliefs and practices within society. The problem comes with forecasting—or the attempt to report predictions as supposedly scientific or quasi-scientific findings akin to work that happens in the natural sciences.
The Pew Research Center site has already pointed out that among other potential sources of error in polling there might be factors like “shy Trumpers” or the unwillingness of some interviewed by pollsters to admit their preference for this candidate because of a sense it was socially inappropriate or embarrassing to do so. This may or may not be true—and it will be up to pollsters and other researchers to settle that important question. Yet what Pew failed to recognize is the very fact that this—humans’ tendency to form beliefs in new and unexpected ways—is even possible might point to a fundamental philosophical problem with the entire project of scientific prediction.
Humanists across the social sciences, history, literature, and legal studies have argued for decades that politics is not a science but one of the humanities. In the view of humanists like myself, political knowledge is much closer to history than to physics or biology. The reason for this, as the philosopher Charles Taylor famously put it, is because human beings are “self-interpreting animals.” That is, humans are creative agents whose beliefs are held for contingent reasons that can always change, and therefore not susceptible to the causal predictions of the natural sciences. This means demographics, economy, voting history, and the other classifications that political scientists and statistics gurus use to scientifically model predictions are never destiny for human beings.
As self-interpreting animals, humans can act on their demographic, economic, and political identity in varying ways. For example, a lifelong registered Republican might interpret his party commitment as making voting for Trump impossible (the much-talked-about “never Trump” phenomenon) or he might instead interpret his beliefs in such a way that refraining from voting for Trump becomes unfathomable. Indeed, the very same person might reinterpret his beliefs back and forth various times between these two possibilities even as he walks into the polling booth. The point is that human action, because it has this creative, self-interpretive dimension, is not susceptible to the scientific predictions of objects in the world that are incapable of such self reflection (for example, brute objects like chemical compounds or projectiles that do not form beliefs or interpretations about themselves).
If this philosophical objection to the attempt to turn the study of politics into a predictive science is valid, then statisticians ought to limit themselves to presenting polls as snapshots of political reality. But they also ought to admit they have no special power beyond anyone else to guess what will happen next on the basis of those snapshots. As the social theorist and humanist Alasdair MacIntyre rightly put it: “Our social order is in a very literal sense out of our, and indeed, anyone’s control.”
For higher education this means that the study of politics might instead be radically reconceptualized as a humanistic domain. Students of politics may still make use of mass surveys, sampling, and polling. But they should also recognize that scientific prediction and forecasting are never attainable.
And yet the attempt to turn the study of politics into a science continues to be one of the biggest and most well-funded intellectual projects of our time. Moreover those academics and intellectuals who present themselves as “scientists” are given much larger platforms than political historians, cultural experts, or legal theorists. (Tellingly, Wang is a professor of molecular biology at Princeton University and not primarily trained in the historical and humanistic forms that might have helped him to grasp the limits of human political knowledge.)
Together with many other humanists in the academy, I have written at length about how this attempt to scientifically predict the future of political reality is part of a wider century-long trend to present the studies of politics as a science akin to the natural sciences. Yet many of the political experts and voices within the academy that are more humanistic are sidelined from public discourse in favor of “science” and “rigor.”
The natural sciences are among humankind’s most impressive and important accomplishments. But the attempt to explain politics in ways modeled on the natural sciences continues to be self-defeating. The election of 2016 has been another emotional encounter with just how limited human knowledge of political life really is. The question going forward is: Will politicians, educators, and citizens reconfigure the country’s educational institutions, think tanks, and politics so that they become more humanistic? Or will they instead continue to try to build a science that the reality of human freedom thwarts?
*This article originally misattributed the "panel model" in part to the University of California, Los Angeles, instead of the Los Angeles Times. We regret the error.