You are a small though extraordinarily wealthy technology company, afloat in a sea filled with other such companies. Some are larger than you, and some are wealthier. People in your industry use language that touches on noble virtues and planet-wide connection. But in practice, you profit from a society layered with different kinds of oppression and discrimination. You have to decide how to use your wealth and power in a world that consistently falls short. What is your duty?
One approach to this problem might be: Do no harm.
But that is a test that Snapchat, the makers of the extremely successful messaging app of the same name, have repeatedly failed to meet.
One of Snapchat’s more unusual features is its set of face-morphing filters. They’re essentially algorithmic funhouse mirrors: You can swipe different ways of contorting faces seen through your phone camera so that they have an enormous mouth, or a gold medal around their neck, or that make someone look like a comical pirate. In just the past six months, Snapchat has blundered (twice!) into releasing filters that it’s hard not to read as extremely racist.
This week, for instance, it debuted a filter that covered over a user’s eyes and forehead with closed-eye slants while enlarging their teeth and reddening their cheeks. The company called the feature “anime-inspired.”
But as highlighted by Katie Zhu, a product manager at Medium, that set of visual clues has few roots in anime. “Anime characters are known for their angled faces, spiky and colorful hair, large eyes, and vivid facial expressions,” she wrote. Instead, the filter adopts wholesale a different visual language of representing East Asians: yellowface. Indeed, two hallmarks of yellowface are squinted eyes and buckteeth.