How to Stop the Bullies

The angst and ire of teenagers is finding new, sometimes dangerous expression online—precipitating threats, fights, and a scourge of harrassment that parents and schools feel powerless to stop. The inside story of how experts at Facebook, computer scientists at MIT, and even members of the hacker collective Anonymous are hunting for solutions to an increasingly tricky problem.
Geoff McFetridge

In the annals of middle-school mischief, the Facebook page Let’s Start Drama deserves an entry. The creator of the page—no one knew her name, but everyone was sure she was a girl—had a diabolical knack for sowing conflict among students at Woodrow Wilson Middle School in Middletown, Connecticut. “Drama Queen,” as I came to think of her in the months I spent reporting at the school to write a book about bullying, knew exactly how to use the Internet to rile her audience. She hovered over them in cyberspace like a bad fairy, with the power to needle kids into ending friendships and starting feuds and fistfights.

In contrast with some other social networks, like Twitter, Facebook requires its users to sign up with their real names. Drama Queen easily got around this rule, however, by setting up Let’s Start Drama with a specially created e-mail address that didn’t reveal her identity. Wrapped in her cloak of anonymity, she was free to pass along cruel gossip without personal consequences. She started by posting a few idle rumors, and when that gained her followers, she asked them to send her private messages relaying more gossip, promising not to disclose the source. Which girl had just lost her virginity? Which boy had asked a girl to sext him a nude photo? As Drama Queen posted the tantalizing tidbits she gathered, more kids signed up to follow her exploits—a real-life version of Gossip Girl. She soon had an audience of 500, many drawn from Woodrow Wilson’s 750 students, plus a smattering from the local high school and a nearby Catholic school.

Students didn’t just message rumors to Drama Queen; they also commented in droves on her posts, from their own real Facebook accounts, or from other fake ones. As one kid wrote about Drama Queen on the Let’s Start Drama page, “She just starts mad shit and most of the time so do the ppl who comment.”


Drama Queen was particularly ingenious at pitting kids against each other in contests of her own creation. She regularly posted photographs of two girls side by side, with the caption “WHOS PRETTIERRR?!” Below the pictures, commenters would heckle and vote. One such contest drew 109 comments over three days. When it became clear which contestant was losing, that girl wrote that she didn’t care: “nt even tryinqq to b funny or smart.” The rival who beat her answered, “juss mad you losss ok ppl voted me ! If you really loooked better they wouldve said you but THEY DIDNT sooo sucks for you.” This exchange nearly led to blows outside of school, other students told me. And they said a fight did break out between two boys who were featured on Let’s Start Drama, in dueling photos, above the caption “Who would win in a fight?” They reportedly ended up pummeling each other off school grounds one day after classes.

Melissa Robinson, who was a social worker for the Middletown Youth Services Bureau, quickly got wind of Let’s Start Drama because, she says, “it was causing tons of conflict.” Robinson worked out of an office at Woodrow Wilson with Justin Carbonella, the bureau’s director, trying to fill gaps in city services to help students stay out of trouble. Their connecting suite of small rooms served as a kind of oasis at the school: the two adults didn’t work for the principal, so they could arbitrate conflict without the threat of official discipline. I often saw kids stop by just to talk, and they had a lot to say about the aggression on Let’s Start Drama and the way it was spilling over into real life. “We’d go on Facebook to look at the page, and it was pretty egregious,” Carbonella told me. Surfing around on Facebook, they found more anonymous voting pages, with names like Middletown Hos, Middletown Trash Talk, and Middletown Too Real. Let’s Start Drama had the largest audience, but it had spawned about two dozen imitators.

Carbonella figured that all of these pages had to be breaking Facebook’s rules, and he was right. The site has built its brand by holding users to a relatively high standard of decency. “You will not bully, intimidate, or harass any user,” Facebook requires people to pledge when they sign up. Users also agree not to fake their identities or to post content that is hateful or pornographic, or that contains nudity or graphic violence. In other words, Facebook does not style itself as the public square, where people can say anything they want, short of libel or slander. It’s much more like a mall, where private security guards can throw you out.

Carbonella followed Facebook’s procedure for filing a report, clicking through the screens that allow you to complain to the site about content that you think violates a rule. He clicked the bubbles to report bullying and fake identity. And then he waited. And waited. “It felt like putting a note in a bottle and throwing it into the ocean,” Carbonella said. “There was no way to know if anyone was out there on the other end. For me, this wasn’t a situation where I knew which student was involved and could easily give it to a school guidance counselor. It was completely anonymous, so we really needed Facebook to intervene.” But, to Carbonella’s frustration, Let’s Start Drama stayed up. He filed another report. Like the first one, it seemed to sink to the bottom of the ocean.

Facebook, of course, is the giant among social networks, with more than 1 billion users worldwide. In 2011, Consumer Reports published the results of a survey showing that 20 million users were American kids under the age of 18; in an update the next year, it estimated that 5.6 million were under 13, the eligible age for an account. As a 2011 report from the Pew Internet and American Life Project put it, “Facebook dominates teen social media usage.” Ninety-three percent of kids who use social-networking sites have a Facebook account. (Teens and preteens are also signing up in increasing numbers for Twitter—Pew found that 16 percent of 12-to-17-year-olds say they use the site, double the rate from two years earlier.)

Social networking has plenty of upside for kids: it allows them to pursue quirky interests and connect with people they’d have no way of finding otherwise. An online community can be a lifeline if, say, you’re a gender-bending 15-year-old in rural Idaho or, for that matter, rural New York. But as Let’s Start Drama illustrates, there’s lots of ugliness, too. The 2011 Pew report found that 15 percent of social-media users between the ages of 12 and 17 said they’d been harassed online in the previous year. In 2012, Consumer Reports estimated that 800,000 minors on Facebook had been bullied or harassed in the previous year. (Facebook questions the methodology of the magazine’s survey; however, the company declined to provide specifics.) In the early days of the Internet, the primary danger to kids seemed to be from predatory adults. But it turns out that the perils adults pose, although they can be devastating, are rare. The far more common problem kids face when they go online comes from other kids: the hum of low-grade hostility, punctuated by truly damaging explosions, that is called cyberbullying.

What can be done about this online cruelty and combat? As parents try, and sometimes fail, to keep track of their kids online, and turn to schools for help, youth advocates like Robinson and Carbonella have begun asking how much responsibility falls on social-networking sites to enforce their own rules against bullying and harassment. What does happen when you file a report with Facebook? And rather than asking the site to delete cruel posts or pages one by one, is there a better strategy, one that stops cyberbullying before it starts? Those questions led me to the Silicon Valley headquarters of Facebook, then to a lab at MIT, and finally (and improbably, I know) to the hacker group Anonymous.

The people at Facebook who decide how to wield the site’s power when users complain about content belong to its User Operations teams. The summer after my trips to Woodrow Wilson, I traveled to the company’s headquarters and found Dave Willner, the 27-year-old manager of content policy, waiting for me among a cluster of couches, ready to show me the Hate and Harassment Team in action. Its members, who favor sneakers and baseball caps, scroll through the never-ending stream of reports about bullying, harassment, and hate speech. (Other groups that handle reports include the Safety Team, which patrols for suicidal content, child exploitation, and underage users; and the Authenticity Team, which looks into complaints of fake accounts.) Willner was wearing flip-flops, and I liked his blunt, clipped way of speaking. “Bullying is hard,” he told me. “It’s slippery to define, and it’s even harder when it’s writing instead of speech. Tone of voice disappears.” He gave me an example from a recent report complaining about a status update that said “He got her pregnant.” Who was it about? What had the poster intended to communicate? Looking at the words on the screen, Willner had no way to tell.

In the early days of the Internet, the danger to kids seemed to be from predatory adults. But it turns out that those perils are rare compared with the problems that come from other kids.

In an attempt to impose order on a frustratingly subjective universe, User Operations has developed one rule of thumb: if you complain to Facebook that you are being harassed or bullied, the site takes your word for it. “If the content is about you, and you’re not famous, we don’t try to decide whether it’s actually mean,” Willner said. “We just take it down.”

All other complaints, however, are treated as “third-party reports” that the teams have to do their best to referee. These include reports from parents saying their children are being bullied, or from advocates like Justin Carbonella.

To demonstrate how the harassment team members do their jobs, Willner introduced me to an affable young guy named Nick Sullivan, who had on his desk a sword-carrying Grim Reaper figurine. Sullivan opened the program that he uses for sorting and resolving reports, which is known as the Common Review Tool (a precursor to the tool had a better name: the Wall of Shame).

Sullivan cycled through the complaints with striking speed, deciding with very little deliberation which posts and pictures came down, which stayed up, and what other action, if any, to take. I asked him whether he would ever spend, say, 10 minutes on a particularly vexing report, and Willner raised his eyebrows. “We optimize for half a second,” he said. “Your average decision time is a second or two, so 30 seconds would be a really long time.” (A Facebook spokesperson said later that the User Operations teams use a process optimized for accuracy, not speed.) That reminded me of Let’s Start Drama. Six months after Carbonella sent his reports, the page was still up. I asked why. It hadn’t been set up with the user’s real name, so wasn’t it clearly in violation of Facebook’s rules?

After a quick search by Sullivan, the blurry photos I’d seen many times at the top of the Let’s Start Drama page appeared on the screen. Sullivan scrolled through some recent “Who’s hotter?” comparisons and clicked on the behind-the-scenes history of the page, which the Common Review Tool allowed him to call up. A window opened on the right side of the screen, showing that multiple reports had been made. Sullivan checked to see whether the reports had failed to indicate that Let’s Start Drama was administered by a fake user profile. But that wasn’t the problem: the bubbles had been clicked correctly. Yet next to this history was a note indicating that future reports about the content would be ignored.

We sat and stared at the screen.

Willner broke the silence. “Someone made a mistake,” he said. “This profile should have been disabled.” He leaned in and peered at the screen. “Actually, two different reps made the same mistake, two different times.”

There was another long pause. Sullivan clicked on Let’s Start Drama to delete it.

Presented by

Join the Discussion

After you comment, click Post. If you’re not already logged in you will be asked to log in or register with Disqus.

Please note that The Atlantic's account system is separate from our commenting system. To log in or register with The Atlantic, use the Sign In button at the top of every page.

blog comments powered by Disqus


Cryotherapy's Dubious Appeal

James Hamblin tries a questionable medical treatment.


Confessions of Moms Around the World

In Europe, mothers get maternity leave, discounted daycare, and flexible working hours.


How Do Trees Know When It's Spring?

The science behind beautiful seasonal blooming

More in Technology

More back issues, Sept 1995 to present.

Just In