The Primary Way to Report Harassment Online Is Broken

Nearly every tech company uses the same imperfect mechanism to flag abuse.
Robert Pratta/Reuters

Death threats, violent misogyny, child pornography, copyright infringement. 

See any of those four very different things on a social media site and you pretty much have only one technical option: You can hit a button to mark the content as objectionable, ‘flagging’ it for review by the site’s moderators. It’s a feature as simple as it is widespread. Vine, YouTube, Facebook, and Twitter all implement the flag to some degree.

It’s also a procedure newly scrutinized. Last week, the actress Zelda Williams quit Twitter and Instagram after users on both harassed her over the death of her father, Robin Williams. Anonymous accounts were tweeting graphic images of dead bodies at her, falsely claiming them to show her father.

Following her public departure, Twitter announced it will revise its rules about harassment. While it hasn’t indicated what changes it plans to make, they will almost certainly involve the flag. 

And that’s going to be hard. A paper released last month in New Media and Society explored the many problems of relying on the flag to moderate content. Its two authors—Kate Crawford, a researcher at Microsoft, and Tarleton Gillespie, a professor at Cornell University—suggest that, while the flag had its moment, it might be time for social media sites to migrate to something new.

However, the flag, as Crawford and Gillespie point out, is not a singular thing. Different sites have tailored flags to their various purposes—and to the various degrees that their users have demanded. Vine only has a report button, a single method of telling the platform’s moderators: “I object to this.” Facebook, meanwhile, has a multi-level display for flagging content. It even has a “support dashboard,” where users can see the status of their flags and cancel them before they’re reviewed. 

In a useful appendix, the authors break out various sites’s flagging interfaces. The chart below (here’s the full size version) shows the huge variance in levels of objection a user can file between Vine’s, Twitter’s, and YouTube’s flagging features. (That fourth, most-specific level of objection a user can file on Twitter? Report an ad.)

New Media and Society

But regardless of how deep your can go in your flagging, the system itself as has drawbacks. The flag can be gamed, whether as a prank or as part of a sustained campaign. In 2012, a conservative group was accused of “flagging” pro-gay rights Facebook pages as objectionable content. When questioned, one of the group's leaders said he only encouraged users to flag-as-tactic after other groups flagged anti-gay rights pages.

And in some places only the abused can flag content, not bystanders or friends. In the hours before she quit Twitter, Williams asked her followers to report some of the worst abuse. But Twitter’s corporate policies permit only the targets of abuse themselves to flag content as objectionable. Other users’ flags did nothing, and the abuse continued.

Presented by

Robinson Meyer is an associate editor at The Atlantic, where he covers technology.

The Best 71-Second Animation You'll Watch Today

A rock monster tries to save a village from destruction.

Join the Discussion

After you comment, click Post. If you’re not already logged in you will be asked to log in or register.

blog comments powered by Disqus


The Best 71-Second Animation You'll Watch Today

A rock monster tries to save a village from destruction.


The Case for Napping at Work

Most Americans don't get enough sleep. More and more employers are trying to help address that.


A Four-Dimensional Tour of Boston

In this groundbreaking video, time moves at multiple speeds within a single frame.


Who Made Pop Music So Repetitive? You Did.

If pop music is too homogenous, that's because listeners want it that way.


Stunning GoPro Footage of a Wildfire

In the field with America’s elite Native American firefighting crew

More in Technology

Just In