What Went Right With the French Campaign Polls?

Previous experience with National Front voters helped pollsters navigate this year’s election.

French presidential election candidate Emmanuel Macron greets supporters as he leaves a polling station on May 7, 2017.
French presidential election candidate Emmanuel Macron greets supporters as he leaves a polling station on May 7, 2017. (Philippe Wojazer / Reuters)

PARIS—Pollsters have gotten a bit of bad rap lately. In both the United States and the United Kingdom, public opinion researchers were excoriated for failing to predict the results of both countries’ shock elections in 2016. As a result, in France, declarations that the National Front’s Marine Le Pen was indeed very likely to lose the presidential election to centrist Emmanuel Macron were frequently met with a reply of: “Well, the polls have been wrong before!”

So when the results came in from the first round of France’s election in late April and mirrored the numbers from final pre-election polls almost exactly, some in France declared that French pollsters, not Macron or Le Pen, were the contest’s real winners. And two weeks later, when Macron won a double-digit victory in the run-off as the polls suggested, they were again largely vindicated (even if they did underestimate Macron’s margin of victory).

This election cycle, these French statisticians have outshined their Anglophone counterparts, proving far more adept at gauging the anti-establishment political atmosphere. There’s good reason for that: They’ve spent more than 40 years dealing with the far-right National Front. As a result, pollsters are used to adjusting for so-called “shy National Front” voters in their weighting—though those voters are considerably less shy today than they were in decades past. Additionally, unlike in the U.S., internet surveys are the norm today in France, which election watchers say helps make voters more willing to share their real candidate preferences.

As other countries in Europe go to the polls later this year with populist surges of their own, French pollsters may offer a ray of hope that, unlike in 2016, the polls could give a more accurate picture of what’s going on.

“To be honest, we were under pressure,” Emmanuel Rivière, the managing director of polling firm Kantar Public in France, told me. “The French journalists and politicians … they very frequently mentioned the failure of polls for Brexit and for Trump.”

In part, that pressure came from the survey results ahead of the Brexit vote and the U.S. election, but it was also because French pollsters got the primaries wrong here. In the November 2016 primary for the Republicans, surveys completely missed the late-breaking rise of François Fillon, who ultimately won the primary. And this past January, when the Socialists held their primary, pre-primary surveys missed just how badly eventual Socialist nominee Benoît Hamon would defeat his rival, former Prime Minister Manuel Valls. Pollsters say this is a result of the relatively small number of people who voted in the primaries—just a fraction of the electorate of each party—and the fact that primaries are a relatively new concept in France. Those misses allowed Fillon, for example, to state during the presidential race that polls were simply missing his support among the French electorate, and voters to cast doubts on their veracity.

Then came the first round, a close, complex, four-way race between Macron, Le Pen, center-right candidate Fillon, and far-left candidate Jean-Luc Mélenchon, plus Socialist candidate Hamon and six fringe candidates. Further complicating matters was the fact that voter dissatisfaction with the candidates was expected to cause lower turnout (ultimately turnout was 77 percent in the first round, two points lower than in 2012). Of those four candidates, three came from outside France’s two traditional major political parties, and Macron’s En Marche! (“On the Move!”) party was brand new. Leading up to the first round, some had expressed concerns that the striking homogeneity in survey results was evidence of pollster “herding”: the idea that a firm, not wanting to be the only outlier result in a race, will adjust their results to be more in line with what others are finding, even if they’re not as objectively accurate—although there was no evidence of this.

Ultimately, the pre-election numbers were spot on. On April 21, the final day of published polls ahead of the second round, most put Macron at 23-24 percent, Le Pen at 22 or 23 percent, Fillon at 19 or 20 percent, and Mélenchon in fourth at 18 or 19 percent. When the actual results were finalized for the April 23 election, they almost exactly matched the estimates: Macron and Le Pen advanced to the second round with 24 percent and 21.3 percent, respectively, followed by Fillon with 20 percent and Mélenchon with 19.6 percent.

The unanticipated polling discrepancy in the U.S. election and the Brexit referendum may have come in part from the seemingly “shy” or late-deciding Trump and “Leave” voters who threw off the pollsters. Whereas American and British pollsters were unsure how to account for those two cycle-specific phenomena, French public opinion researchers know precisely how adjust and weight their findings to account for potentially “shy” National Front supporters who may be reluctant to admit their support. They learned this from a “Brexit” moment of their own in 2002, when Le Pen’s father, Jean-Marie Le Pen, shocked the world and advanced to the second round; since then, polling firms have worked especially hard to make sure they understand voting behaviors among National Front supporters. Now, for example, they ask participants how they voted in a past election—then cross-reference that with the actual results from that election, so if there are any discrepancies they can weight their findings accordingly. Pollsters also ask a series of questions to determine how sure voters are that they’ll vote for the candidate they name, as well as a scale of how willing they might be to vote for each candidate even if that candidate isn’t their first choice.

Bruno Jeanbart, director of the French polling firm OpinionWay, said that when the party was under Jean-Marie Le Pen only about one in two National Front supporters admitted their preference in surveys; now, with Marine Le Pen, it’s more like 75 or 80 percent of them. “Before 2007 it was very difficult to find them in the polls—they didn’t want to say it,” Jeanbart said of National Front supporters. “After 2007 and the fact that Jean-Marie Le Pen retired and let Marine Le Pen be the candidate, it was easier. It was not perfect, but easier.”

Part of that is, one can assume, because of Le Pen’s efforts to “de-demonize” her party and make it an acceptable mainstream option. But it’s also because, unlike in America, pollsters in France widely use online surveys these days. Jeanbart said his OpinionWay was the first to start using the technique for political surveys back in 2006; others have since followed. Online surveys help reduce the social pressure to answer a certain way, and have cut down on the need to adjust for shy National Front voters. “With internet polling, the expectation is that the social desirability bias should drop off,” said Jocelyn Evans, an expert on French voting behavior and the far-right at the University of Leeds. “There are simply a lot more people happy to click ‘Le Pen’ rather than say it.” Typically, citizens sign up for online panels either through a polling company’s websites or via advertisements across the internet (on sites including everything from news to travel to shopping websites, said Rivière).

Internet polling comes with its own downsides—namely, it’s difficult to reach certain parts of the populace, like rural voters and older voters who may not be regular internet users. But French firms have a fix for that: In some cases, they’ve even called possible respondents directly, in an effort to make their samples more representative from the start. “They have gone out of their way, really trying hard to recruit for those panels and using techniques to supplement them,” Evans explained.

In the second round, researchers got the overall result right. Macron, as expected, won the race by more than 20 points. But while the topline results were correct, polls significantly underestimated his margin. Most conducted in the final days showed the centrist winning by about 25 points, considerably lower than his eventual 32-point margin of victory.

Experts said this wasn’t necessarily because of misjudging support for the National Front, but because of the aftereffects of what was seen as a disastrous debate performance by Le Pen (and a consequence of France’s pre-election silence laws, keeping pollsters from publishing new numbers in the final 48 hours of the race). “I think there’s been a few green-eyed-with-envy British pollsters looking at the second round and saying, ‘You didn’t get that right,’” Evans said. “The evidence is they probably did get it right three days before, but there was that late swing.”

Macron and Le Pen debated on Wednesday, May 3, just four days before Election Day, and Le Pen’s performance was widely panned by observers and commentators: She repeatedly cited incorrect or false information, resorted to name-calling, and had trouble explaining and defending her economic plans. Under French law, no one is allowed to publish information that could influence the result of the election after Friday night at midnight, a provision which includes polls. As a result, the last polls of the race were conducted either before the debate itself or before its aftermath had a chance to fully sink in among voters.

OpinionWay, for example, published a rolling daily tracker using data from the previous three days. Its Friday poll, Jeanbart said, which gave Macron a 62-38 advantage over Le Pen, was made up of one-third data from Thursday (the day after the debate) and two-thirds from Tuesday and Wednesday (before or during the debate). And when Kantar-Sofres canvassed voters on Thursday and Friday, it found a four-point jump for Macron from its pre-debate poll, according to Rivière: from 59 percent to 63 percent. However, because the results of the poll weren’t ready before midnight on Friday, that data was never released to the public. Macron’s support “was increasingly growing and it was a clear consequence of the debate,” Rivière said.

With the coming legislative elections in June, French pollsters face yet another challenge: Macron’s En Marche will field its first-ever slate of candidates across 577 legislative districts, many of them newcomers to politics, and the traditional left-right divide in parliament is likely to splinter this year, while the National Front works to rebuild and rebrand after its presidential loss. Whether those polls will be right is still an open question. “We have a long track record with the demolition of the political system, which is quite different from the U.S., I think,” Rivière said. “These kinds of shifts, restructurings—we are used to them.”