What happened here?
One theory: By cutting out comments, the site is better able to draw attention to its most deserving content—the articles themselves.
This intrigued me because I found it somewhat counterintuitive. I supported removing comments not because I thought traffic would spike but because it seemed a way to better preserve civil discourse; I assumed we’d lose some rubberneckers who gathered around the train-wreck comment section, but it seemed like a worthwhile trade. Yet the fact that traffic actually improved suggests that sites are better off without comments—or at least better off without unmoderated ones. That's a lesson that other news organizations are learning. As Nieman Lab wrote last month, if news organizations aren't moderating their comment sections, they can't really expect them to foster quality discussion.
But what about the many sites that opt for a less hands-on approach? Plenty of journalists will tell you that they not only don't reply to commenters, but that they don't even read the comments to begin with. An ignored comment section can't be all that harmful, right?
To find out, I ran a quick study using respondents from Amazon’s crowdsourcing platform Mechanical Turk. I asked 100 Americans to read a snippet of a National Journal article from late April. Half of them saw the article alone. The other half saw the article along with a representative sample of actual comments (user accounts redacted) on that article. In both groups, respondents were asked to read the article—the existence of comments was never acknowledged.
After reading, respondents took a short survey. The first two questions were designed simply to verify they’d read the article. This part was easy. All respondents needed to do was identify that the article was about Cliven Bundy, rather than Donald Sterling, Rush Limbaugh, or Barack Obama.
The remaining three questions, however, measured respondent perceptions of the article—and asked for the person's current mood on a five-point scale. The article was the same for all readers, so if comments had no impact, both respondents who saw comments and those who didn't would evaluate the article and their current mood the same way.
That was not the case.
Respondents who saw comments evaluated the article as being of lower quality—an 8 percent difference. In other words, authors are judged not just by what they write, but by how people respond. The presence of comments did not make a statistically significant difference in a person's likelihood to read more content by the same author, nor did it make an appreciable difference in respondent self-reported mood.
The comments used in the sample group are perhaps worse than many Internet comments. They also represent only a small sample of the whopping 7,725 comments—many of them negative or downright offensive—on the actual article. It's easy to see how a reader might reassess her opinion of an article after catching a glimpse of thousands of negative opinions about it.
There are good options for encouraging reader feedback: nice moderated comment sections, forums that build community, quick exchanges on Twitter, or lengthy feedback over email. But unmoderated comments appear to have a small, but real deleterious effect on readers' perception of the sites on which they appear. And that appears to have implications for the bottom-line metric on the web: traffic.