What It Means That Computers Can Tell These Smiles Apart, But You Can't

Computer algorithms can now discern the meaning behind humans' facial expressions. 

[optional image description]
Image via Mohammed Hoque, Daniel McDuff, and Rosalind Picard.

The four people above are taking part in a clinical experiment. In the screenshots shown here, each guy is smiling once out of delight (reacting to a picture of an adorable baby) and once out of frustration: made to complete an online form that keeps malfunctioning. Awww vs. argh: same general expression, totally different emotion. 

So which is which? Who's smiling out of joy, who out of annoyance? 

If you're not totally sure, you're not alone. We humans need context and narrative to be able to discern the meanings of our fellow humans' facial expressions. We're sensitive to subtleties. That's one thing that makes us different from machines. 

Except ... when it's not. In a paper just published in IEEE Transactions on Affective Computing, Mohammed Hoque, Daniel McDuff, and Rosalind Picard share a system that allows computers to become as sensitive as -- and, in fact, even more sensitive than -- humans. 

The team, members of MIT's Affective Computing Group, combined two insights to arrive at their algorithm. First, genuine smiles tend to build slowly and linger, while frustrated smiles tend to appear and disappear quickly. Second, the musculature of fake smiles tends to differ from that of genuine ones: hence "thin" smiles, "stiff" smiles, etc. 

Those types of smiles are often involuntary. When Hoque and his colleagues asked study participants to feign frustration, 90 percent of them did so without smiling. But when the researchers presented their subjects with a task that caused genuine frustration -- filling out an online form, only to find their information deleted after they pressed the "submit" button -- 90 percent of them ended up smiling. Frustratedly.

The algorithm Hoque and his colleagues developed accounts for that expressive difference. And it does so quite effectively. The team's computer-based system was able to figure out which smiles were fake 92 percent of the time. The success rate for humans who were asked to do the same: 50 percent, which is obviously the same as if they had randomly guessed.

The most immediate and obvious application of the team's findings would be to aid people diagnosed with Autism Spectrum Disorder. Emotion-reading computer programs could help the autistic to assess and interpret other people's facial expressions -- one of the biggest impediments to social interaction.

But what about the broader implications? First, a hope ... then, a caveat. 

On the one hand, as the paper puts it, the team's findings could be used "to develop automated systems that recognize spontaneous expressions with accuracy higher than the human counterpart." Facial recognition is now a fairly common technology, used in everything from Facebook to city streets. Emotion recognition is the next logical step in that progression -- a field that could bring a whole new meaning to "sentiment analysis."

Presented by

Megan Garber is a staff writer at The Atlantic. She was formerly an assistant editor at the Nieman Journalism Lab, where she wrote about innovations in the media.

Google Street View, Transformed Into a Tiny Planet

A 360-degree tour of our world, made entirely from Google's panoramas

Join the Discussion

After you comment, click Post. If you’re not already logged in you will be asked to log in or register.

blog comments powered by Disqus

Video

Google Street View, Transformed Into a Tiny Planet

A 360-degree tour of our world, made entirely from Google's panoramas

Video

The 86-Year-Old Farmer Who Won't Quit

A filmmaker returns to his hometown to profile the patriarch of a family farm

Video

Riding Unicycles in a Cave

"If you fall down and break your leg, there's no way out."

Video

Carrot: A Pitch-Perfect Satire of Tech

"It's not just a vegetable. It's what a vegetable should be."

Video

The Benefits of Living Alone on a Mountain

"You really have to love solitary time by yourself."

More in Technology

Just In