A Photo Filter, Just for Your Face

New research promises to bring the photo-touching capabilities of portrait artists to your next selfie.
MIT-Adobe 5K dataset

Photo filters are blunt instruments. The Hefes and Amaros and Kelvins of the world—the algorithms that transform our snapshots into light-washed little works of EmoArt—are extremely un-selective about the modifications they make to our images. "Style transfer," the translation of plain images according to those algorithms, does its translating for an entire image. Which works fine for the typical Instagram shot (of, say, a blackberry gelato, a silkscreened sunset, a babbling brook), but much less fine for the photos that tend to be the most intimate of the images we take: portraits. Our faces, it turns out, are extremely hard to filter—even when you have the app that promises to make you look flawless.

And that's because our minds are extremely attuned to the nuances of other people's appearances. "Our eyes are so sensitive to human faces," YiChang Shih, an MIT graduate student in electrical engineering and computer science, tells PhysOrg. "We're just intolerant to any minor errors." Which means that the filters we use to stylize our pictures may be great for images of nature and baked goods... but they're not great for stylizing each other. Or ourselves. 

Shih and his colleagues, however, say they've solved that problem. And they've done so through a technique that localizes algorithmic filters—applying them, like so many topical headache medications, directly to the face. So: you know how portrait photographers like Diane Arbus and Richard Avedon had signature styles—the results of the artists' intentional adjustments of lens and light and shadow? Shih and his colleagues are claiming that they can replicate those styles, essentially, via algorithm. 

And achieving that would be more difficult than it might seem to the average Instagrammer. The typical digital photo filter works through what are known as "global parameters"—things like exposure, color shift, and global contrast, all of them working in tandem on the pixels of an image. "We started with those filters," Shih says, "but just found that they didn't work well with human faces." 

So he and his team—a group of scientists from MIT, Adobe Systems, and the University of Virginia—adopted a multi-step approach that uses the "local statistics" of an image, allowing for a granular approach to face filtration. Using off-the-shelf facial recognition software, the team first identified portraits that resembled—in facial orientation, style, and the like—that resembled the image they wanted to modify. "We then find a dense correspondence—like eyes to eyes, beard to beard, skin to skin," Shih says. And, from there, they do what they call a "local transfer"—a graft of the corresponding features from the model image onto the original. 

Shih and his team will present their work at the graphics conference Siggraph in August. And, from there, they're exploring commercial applications of their findings. Which means that, at some point not too far in the future, you could apply the "Arbus" filter to your selfie. You could be-Avedon yourself, using your phone. 


Presented by

Megan Garber is a staff writer at The Atlantic. She was formerly an assistant editor at the Nieman Journalism Lab, where she wrote about innovations in the media.

Things Not to Say to a Pregnant Woman

You don't have to tell her how big she is. You don't need to touch her belly.

Join the Discussion

After you comment, click Post. If you’re not already logged in you will be asked to log in or register.

blog comments powered by Disqus


Things Not to Say to a Pregnant Woman

You don't have to tell her how big she is. You don't need to touch her belly.


Maine's Underground Street Art

"Graffiti is the farthest thing from anarchy."


The Joy of Running in a Beautiful Place

A love letter to California's Marin Headlands


'I Didn't Even Know What I Was Going Through'

A 17-year-old describes his struggles with depression.


Google Street View, Transformed Into a Tiny Planet

A 360-degree tour of our world, made entirely from Google's panoramas


The Farmer Who Won't Quit

A filmmaker returns to his hometown to profile the patriarch of a family farm


Riding Unicycles in a Cave

"If you fall down and break your leg, there's no way out."


Carrot: A Pitch-Perfect Satire of Tech

"It's not just a vegetable. It's what a vegetable should be."

More in Technology

Just In