Computer algorithms can now discern the meaning behind humans' facial expressions.
The four people above are taking part in a clinical experiment. In the screenshots shown here, each guy is smiling once out of delight (reacting to a picture of an adorable baby) and once out of frustration: made to complete an online form that keeps malfunctioning. Awww vs. argh: same general expression, totally different emotion.
So which is which? Who's smiling out of joy, who out of annoyance?
If you're not totally sure, you're not alone. We humans need context and narrative to be able to discern the meanings of our fellow humans' facial expressions. We're sensitive to subtleties. That's one thing that makes us different from machines.
Except ... when it's not. In a paper just published in IEEE Transactions on Affective Computing, Mohammed Hoque, Daniel McDuff, and Rosalind Picard share a system that allows computers to become as sensitive as -- and, in fact, even more sensitive than -- humans.
The team, members of MIT's Affective Computing Group, combined two insights to arrive at their algorithm. First, genuine smiles tend to build slowly and linger, while frustrated smiles tend to appear and disappear quickly. Second, the musculature of fake smiles tends to differ from that of genuine ones: hence "thin" smiles, "stiff" smiles, etc.
Those types of smiles are often involuntary. When Hoque and his colleagues asked study participants to feign frustration, 90 percent of them did so without smiling. But when the researchers presented their subjects with a task that caused genuine frustration -- filling out an online form, only to find their information deleted after they pressed the "submit" button -- 90 percent of them ended up smiling. Frustratedly.
The algorithm Hoque and his colleagues developed accounts for that expressive difference. And it does so quite effectively. The team's computer-based system was able to figure out which smiles were fake 92 percent of the time. The success rate for humans who were asked to do the same: 50 percent, which is obviously the same as if they had randomly guessed.
The most immediate and obvious application of the team's findings would be to aid people diagnosed with Autism Spectrum Disorder. Emotion-reading computer programs could help the autistic to assess and interpret other people's facial expressions -- one of the biggest impediments to social interaction.
But what about the broader implications? First, a hope ... then, a caveat.
On the one hand, as the paper puts it, the team's findings could be used "to develop automated systems that recognize spontaneous expressions with accuracy higher than the human counterpart." Facial recognition is now a fairly common technology, used in everything from Facebook to city streets. Emotion recognition is the next logical step in that progression -- a field that could bring a whole new meaning to "sentiment analysis."