Before any article makes it into The Atlantic magazine, it goes through Yvonne Rolzhausen and her fact-checking team. For nearly thirty years, long before the term “alternative facts” entered our lexicon, Yvonne has been making sure every word The Atlantic prints is true. Today, a special treat for Masthead members: Yvonne will walk us through her fact-checking routine, a process that continues—sometimes for months—until she and her team have confirmed every last line.
Word By Word, Line By Line
By Yvonne Rolzhausen, senior editor at The Atlantic
In a world where “fake news” thrives and basic editorial standards are often jettisoned as unnecessary expenses, fact-checkers can sometimes feel like an endangered species. But The Atlantic is dedicated to accuracy and truth—and therefore to rigorous fact-checking. Our pieces seek to be thought-provoking and interesting—but to be truly insightful, they must be right.
Checkers verify every fact published in our magazine, from specific details and quotes to larger generalities. We think about a piece on a variety of levels: Are the basic facts correct? Are the facts underlying various opinions correct? And, finally, do they all fit together into a comprehensive and solid argument? We go word by word, line by line. For an intensively-reported piece, I might have dozens of sources to contact and hundreds of questions for an author. The process can take anywhere from a few hours (for a very short article) to weeks or even months (for a complex, legally-fraught one).
Let me walk you through my process for checking this short section of “What ISIS Really Wants,” Graeme Wood’s March 2015 feature for The Atlantic, the story with the highest engagement time in the world on the internet that year. In the piece, Graeme explores the ideology of the Islamic State, arguing that the group is rooted in carefully-considered religious beliefs.
How do I go about fact-checking a piece like that? Here are the basic steps.
-
Get familiar with the material. I read the piece a few times and educate myself on the topic. Then the author either annotates the piece with sources in footnotes or simply walks me through it. This gives me a sense of how the piece was put together: What or who are the sources? Who might be difficult or sensitive to deal with? What did the author read? Then I ask the most important question: What is the author most worried about? Often it is the understandable fear that a highly sensitive source might not want to cooperate with the checking process. More on that soon.
-
Break down the piece with a red pencil. Every checker has a different system but I’m old-fashioned and still work on paper. I format the piece with wide margins so I can clearly keep track of which source is responsible for which fact. Months later, I need to be able to see the backup for everything. I then underline all the facts that have to be checked in red pencil. Proper names are highlighted. Legal sections are noted in red marker with lots of circled stars to indicate a need for triple-checking. Anything that I have confirmed gets a check mark through it—and, oh, the lovely satisfaction of making a check mark! The checked text disappears into the background, allowing me to focus on the lingering unchecked text. If I’m worried about a detail and want to discuss with the author, I’ll highlight it in yellow, and list possible solutions on a sticky note. After Graeme and I agreed on a change, I circled it with a red pen.
-
Plan interviews with the author’s sources. Next, I figure out who to contact and what to ask them. For a primary source, this could mean hours of conversations or pages of emailed questions. For a difficult or sensitive source, I create a script of what I need to find out and confirm, since these conversations are too important to leave anything to chance. Once a source hears the checking questions, and they realize what the author has or hasn’t chosen to include, they often sense the focus of the piece in a way that wasn’t clear in initial interviews. This is one reason why checking is not a job for the faint of heart. The next-to-last thing that a checker wants is to endanger a piece’s prospects for publication—but the last thing a checker wants is to allow publication of a piece that cannot withstand factual scrutiny. So it is utterly imperative to know about any potential issues prior to publication. It’s a controlled explosion, of sorts, when we still have enough time to sort out problems or at least prepare ourselves for the fall-out.
-
Start talking. One of Graeme’s main sources for this article was a young Australian named Musa Cerantonio, who, at the time, had been called one of the two most important “new spiritual authorities” guiding foreigners to join the Islamic State. And while Graeme had had hours of interviews during which to build remarkable trust with Cerantonio, I had to do the same in much shorter order. When Graeme and I first spoke about this, our fear was that Cerantonio might be reluctant to speak to a Western woman about anything, never mind spend hours confirming his philosophical beliefs. When we did finally speak, perhaps because of his confidence in Graeme, Cerantonio was surprisingly cordial and quite candid, especially given that he was being closely watched by the Australian government, under suspicion of promoting terrorism.
-
Review quotes with the author’s sources. Occasionally I will paraphrase quotes for sources, but generally I read the quotes to them. Invariably many explode (and not always in a controlled way). But if someone objects, that doesn’t mean we aren’t going to publish. We just need to make sure that the quote is accurate, that it is attributed to the appropriate source, and that the context is fair. In the second paragraph of his article, Graeme notes that, according to Cerantonio’s friends, ISIS leaders have an obligation to declare the caliphate. When I asked Cerantonio about this, he was coy, clarifying that it wasn’t “a friend” but someone he communicated with in chat rooms. Graeme immediately realized who he meant and we changed the line to say “a Western convert within the group’s ranks who Cerantonio had described as ‘something of a leader.’”
-
Call on a few experts. I went through all of the facts in this section with Cerantonio, but it was also crucial to make sure other experts agreed with him. To confirm whether or not ISIS supporters considered it “sinful” for their leaders to delay the establishment of an Islamic caliphate, I spoke to two others quoted elsewhere in the piece: Anjem Choudary, a U.K. preacher and ISIS supporter, and Cole Bunzel, a Princeton scholar of Islamic State ideology. In my notes, you can see where they confirmed various details. In one instance—in the second paragraph, where I circled “they would remove themselves from Islam,” referring to the consequences for ISIS leaders if they did not appoint a caliphate—Choudary and Bunzel felt the wording was too strong. Although Cerantonio had originally said this to Graeme, when asked about these comments directly he moderated his stance.
-
Talk everything over with the author. People often ask me if the process with the author is confrontational. It shouldn’t be. A checker should never assume to know all without giving an author the benefit of the doubt. I always try to be respectful and kind. The author has been swimming in these waters for a long time before I jump in. We never keep score. One of the best ways to avoid problems in going over questions with an author: Offer suggestions for alternative language that would solve each checking issue. If I can’t come up with an easy solution, it’s often because I don’t fully understand the problem. I need to keep digging.
Part detective, part therapist, part comrade-in-arms, fact-checkers should, above all, be guardian angels sitting on an author’s shoulder, making sure that their arguments are based in fact, rather than supposition. Such intensive scrutiny may make it seem like we are trying to tear down an argument—but our intention is opposite. We tease the argument apart only to build it back up with even greater strength. Our work requires diligence, tenacity, diplomacy, patience, and pretty much constant fear. But it is always interesting. And in a too often careless world, it can even feel noble. Check please!
Today’s Wrap Up
-
Question of the day: What other aspects of The Atlantic would you like to learn more about?
-
Your feedback: How did you like this issue of The Masthead? Take a few seconds, and fill out our daily survey.
-
What’s coming: I’m working on a profile of Kenyatta Cheese, founder of Know Your Meme, a website dedicated to cataloguing everything that goes viral on the internet (added plus: He’s also a Masthead member).
We want to hear what you think about this article. Submit a letter to the editor or write to letters@theatlantic.com.
