Movies of the Mind: Using Brain Scans to See What Others Think

More

While it only works now with clips from actual movies, researchers have taken a major step in decoding what people see in their mind's eye

MovieReel-Shutterstock-Post.jpg

Every day, the world unfolds much like a movie does. Yet no two people experience the world in quite the same way. Visually, it's as if everyone were watching slightly different versions of the same movie.

Researchers at the University of California, Berkeley have just taken a major step in decoding and reproducing just what people are seeing, using brain scans taken while three of the researchers watched movie clips. And while the technique only works right now for clips from actual movies, down the road it may be able to show what people are seeing and thinking in their everyday mental movies -- like mindreading.

In the past, researchers were only able to decode people's visual images of stationary objects, like a chair, by using functional magnetic resonance imaging (fMRI). These brain scans record the amount of blood oxygen in specific areas of the brain, which increases as an area becomes more active. But these changes are relatively slow compared to changes in activity of the nerves themselves. To obtain imaging information that can decode and reproduce moving objects also requires information on the fast changes that are occurring within nerve cells.

The researchers were able to obtain this information by using a sophisticated computer program that could approximate the quickly occurring neural changes from the relatively slow blood oxygen changes in the fMRI signals. This two-step process gave scan data with enough information to successfully reconstruct moving objects as well as stationary ones.

Because the study involved sitting for hours inside a scanner and watching movie clips, three members of the research team served as subjects.

The subjects each watched two hours of movie clips while undergoing an fMRI scan. The scan data was fed into a computer program that learned, second by second, the correlation between the fMRI scan signals and the image that a subject was watching at the time.

The subjects then watched a second, different set of movie clips, nine minutes total, with each clip repeated ten times. Using information from the first set of scans, the computer program was able to decode what clips the subjects were watching from the brain scan data with good accuracy.

Decoding was done by having the computer program pick from 5,000 hours (one million clips) of random movie clips taken from the Internet. The program then averaged images from the 100 clips that best resembled the scan data from the subject. The result was a single blurry but recognizable reconstruction of what the subject had actually been watching -- a successful decoding.

This is only a baby step toward true mind-reading, but it's a step no one's been able to take before.

The ability to reproduce what a person is seeing has many possible applications, from communicating with people unable to express themselves to changing the entire dating experience and beyond. Reconstructing movie clips is just a taste of things to come. It brings us one step closer to the day when we read the brain and are able to tell what our loved ones and not so loved ones are seeing and thinking, for better or for worse.

An article detailing the study was published online by Current Biology on September 22, 2011, and will also appear in a future print edition of the journal.

Image: Iaroslav Neliubov/Shutterstock.


This article originally appeared on TheDoctorWillSeeYouNow.com.

Jump to comments
Presented by
Get Today's Top Stories in Your Inbox (preview)

Why Are Americans So Bad at Saving Money?

The US is particularly miserable at putting aside money for the future. Should we blame our paychecks or our psychology?


Elsewhere on the web

Join the Discussion

After you comment, click Post. If you’re not already logged in you will be asked to log in or register. blog comments powered by Disqus

Video

The Death of Film

You'll never hear the whirring sound of a projector again.

Video

How to Hunt With Poison Darts

A Borneo hunter explains one of his tribe's oldest customs: the art of the blowpipe

Video

A Delightful, Pixar-Inspired Cartoon

An action figure and his reluctant sidekick trek across a kitchen in search of treasure.

Video

I Am an Undocumented Immigrant

"I look like a typical young American."

Video

Why Did I Study Physics?

Using hand-drawn cartoons to explain an academic passion

Writers

Up
Down

More in Health

Just In