Read: The shooter’s manifesto was designed to troll
Defending himself and Facebook before Congress last year against myriad failures, including allowing Russian operatives to disrupt American elections and permitting illegal housing ads that discriminate by race, Mark Zuckerberg repeatedly invoked artificial intelligence as a solution for the problems his and other global internet companies have created. There’s just too much content for human moderators to process, even when pressed hard to do so under poor working conditions. The answer, Zuckerberg has argued, is to train AI to do the work for them.
But that technique has proved insufficient. For one part, that’s because AI is an aspirational solution for a future that has not arrived. It gives Zuckerberg and others rhetorical cover more than technological outcomes. But for another, detecting and scrubbing undesirable content automatically is extremely difficult. False positives enrage earnest users or foment conspiracy theories among paranoid ones, thanks to the black-box nature of computer systems. Worse, given a pool of billions of users, the clever ones will always find ways to trick any computer system, for example, by slightly modifying images or videos in order to make them appear different to the computer but identical to human eyes. 8chan, as it happens, is largely populated by computer-savvy people who have self-organized to perpetrate exactly those kinds of tricks.
The primary sources are only part of the problem, too. Long after the deed, YouTube has bolstered conspiracy theories about murders, successfully replacing truth with lies among broad populations of users who might not even know they are being deceived. Even stock-photo providers are licensing stills from the New Zealand shooter’s video; a Reuters image that shows the perpetrator wielding his rifle as he enters the mosque is simply credited, “Social media.”
The video is just the tip of the iceberg. Many smaller and less obviously inflamed messages have no hope of being found, isolated, and removed by technology services. The shooter praised Donald Trump as a “symbol of renewed white identity” and incited the conservative commentator Candace Owens, who took the bait on Twitter in a post that got retweeted thousands of times by the morning after the attack. The shooter’s forum posts and video are littered with memes and inside references that bear special meaning within certain communities on 8chan, 4chan, Reddit, and other corners of the internet, offering tempting receptors for consumption and further spread.
Read: White nationalism’s deep American roots
Perhaps worst of all, the forum posts, the manifesto, and even the shooting itself might not have been carried out with the purpose that a literal read of their contents suggests. On first blush, it seems impossible to deny that this terrorist act was motivated by white-supremacist hatred, an animosity that authorities like the FBI expert and the Facebook officials would want to snuff out before it spreads. But 8chan is notorious for an ironic, above-it-all approach to all of its perversities, a squalor amplified by the anonymity intrinsic to the service. Here, where users post “for the lulz,” or just to get a rise out of those who aren’t in the know, the ideology embraces chaos before it does zealotry. But the internet separates images from context and action from intention, and then it spreads those messages quickly among billions of people scattered all around the globe.