Computers Are Getting Better Than Humans at Facial Recognition

Technology could chip away at freedom of expression—including our "right to lie."
A U.S. Army soldier scans an eye of a suspect in Kandahar, Afghanistan, in 2010. (Shamil Zhumatov/REUTERS)

Perceiving whether someone is sad, happy, or angry by the way he turns up his nose or knits his brow comes naturally to humans. Most of us are good at reading faces. Really good, it turns out. 

So what happens when computers catch up to us? Recent advances in facial recognition technology could give anyone sporting a future iteration of Google Glass the ability to detect inconsistencies between what someone says (in words) and what that person says (with a facial expression). Technology is surpassing our ability to discern such nuances.

Scientists long believed humans could distinguish six basic emotions: happiness, sadness, fear, anger, surprise, and disgust. But earlier this year, researchers at Ohio State University found that humans are capable of reliably recognizing more than 20 facial expressions and corresponding emotional states—including a vast array of compound emotions like “happy surprise” or “angry fear.” Recognizing tone of voice and identifying facial expressions are tasks in the realm of perception where, traditionally, humans perform better than computers. Or, rather, this used to be the case. As facial recognition software improves, computers are getting the edge. The Ohio State study, when attempted by a facial recognition software program, achieved an accuracy rate on the order of 96.9 percent in the identification of the six basic emotions, and 76.9 percent in the case of the compound emotions. Computers are now adept at figuring out how we feel.

Much of this kind of computation is based on the so-called Facial Action Coding System (FACS), a method developed by Paul Elkman, a specialist in facial micro-expressions, during the 1970s and 1980s. FACS decomposes emotional expressions into their distinct facial elements. In other words, it breaks down emotions to specific sets of facial muscles and movements: the widening of the eyes, the elevation of the cheeks, the dropping of the lower lip, and so on. FACS is used in the design and construction of characters in animated films. It’s also used by cognitive scientists to identify genes, chemical compounds, and neuronal circuits that regulate the production of emotions by the brain. Such mapping could be used in the diagnosis of disorders like autism or post-traumatic stress disorder, where there is difficulty in recognizing emotions from facial expressions.

As surveillance technologies become more widespread, so do applications for sophisticated facial recognition software. These technologies appear to be ready to move from the laboratory into real life—commercialized, distributed to the masses in any number of fields, contexts, and situations. All this is happening at a time when computers are getting smarter and smarter at reading human emotion.

A team of researchers at the University of California, San Diego, founded the company Emotient, which uses machine-learning algorithms to detect emotion. The company is currently developing an app for Google Glass which will soon be on the market, according to lead scientist Marian Bartlett. The app is designed to read the emotional expressions of people who appear in the user’s field of vision in real time. Though it’s still in the testing phase, it can recognize happiness, sadness, anger, repugnance, and even scorn.

Once this kind of technology is commercially available, it’s downloadable for any Google Glass user. And beyond the recognition of specific emotions through analysis of facial movement patterns, another application of this new technology, tested by Bartlett’s team, is one that allows it to distinguish fake from true emotional expressions. In other words, it can tell if you’re lying.

The app works based on the idea that false and true expressions of emotions involve different brain mappings. While true emotional expressions are executed by the brain and the spinal cord just like a reflex, fake expressions require a conscious thought—and that involves regions of motor coordination from the cerebral cortex. As a result, the representative facial movements of true and fake emotions end up being different enough for a visual computation system to be able to detect and distinguish them. And here’s the key: Computers can make these distinctions even when humans cannot.

Upon testing, the system developed by Bartlett managed—in real time—to identify 20 of the 46 facial movements described in the FACS, according to a March report by Bartlett in Current Biology. And, even more impressive, the system not only identifies, but distinguishes authentic expressions from false expressions with an accuracy rate of 85 percent, at least in laboratory settings where the visual conditions are held constant. Humans weren’t nearly as skilled, logging an accuracy rate of about 55 percent.

Yes, Bartlett incorporated a lie detector into the facial recognition technology. This technology promises to catch in the act anyone who tries to fake a given emotion or feeling. Facial recognition is evolving into emotional recognition, but computers—not just people—are the ones deciding what's real. ( If we add voice detection to face recognition, we end up with a complete lie detection package.)

Technology and the Right to Lie

So we can begin to imagine a near future in which we’re equipped with glasses that not only recognize faces and voices, but also truths and lies—a scenario that would provoke a revolution in human interaction. It would also constitute a serious limitation on the individual’s autonomy. Let’s start with the implications of this technology at the level of social behavior and move on to analyze the implications at the individual level.

Presented by

Norberto Andrade

Norberto Andrade is a postdoctoral research fellow at the University of California, Berkeley, School of Law.

How to Cook Spaghetti Squash (and Why)

Cooking for yourself is one of the surest ways to eat well. Bestselling author Mark Bittman teaches James Hamblin the recipe that everyone is Googling.

Join the Discussion

After you comment, click Post. If you’re not already logged in you will be asked to log in or register.

blog comments powered by Disqus


How to Cook Spaghetti Squash (and Why)

Cooking for yourself is one of the surest ways to eat well.


Before Tinder, a Tree

Looking for your soulmate? Write a letter to the "Bridegroom's Oak" in Germany.


The Health Benefits of Going Outside

People spend too much time indoors. One solution: ecotherapy.


Where High Tech Meets the 1950s

Why did Green Bank, West Virginia, ban wireless signals? For science.


Yes, Quidditch Is Real

How J.K. Rowling's magical sport spread from Hogwarts to college campuses


Would You Live in a Treehouse?

A treehouse can be an ideal office space, vacation rental, and way of reconnecting with your youth.

More in Technology

Just In