How Amazon Speeds Up Economics Experiments


Social science experiments are important. They allow us to ground sociological and economic theory in real human behavior, and helps establish causal links between events.

"But the problem is that they are a a pain in the ass to do," said David Rand, a Berkman Center for the Internet and Society fellow. "You have to get someone to get down to the lab and spend time in transit. There are fixed time costs."

The Berkman Center is a leading institution for important research into the uses and impacts of digital technologies. We'll be previewing their regular brownbag lunches here on The Atlantic Technology Channel.

That's why he and fellow online social experiment pioneers like Harvard's John Horton are turning to Amazon's Mechanical Turk and other online labor markets to find willing participants. The services take the pain out of doing social science experimentation, decreasing costs and increasing the speed of idea development.

For example, Rand submitted a paper to a journal, which asked that he carry out another experiment before accepting his work. "I got the rejection on Thursday. I designed the experiment on Friday and by Sunday I had 500 people recruited," he said.

Rand will be talking about the value of these online research tools at the Berkman Center today. You can tune in live.

Mechanical Turk allows anyone to submit a set of small tasks, which are farmed out to a bunch of anonymous workers all across the world for tiny amounts of money. It tends to draw people from India, who both speak English and can make meaningful cash from completing tasks for a few dimes a piece.

How do these experiments actually play out? Researchers design experiments that mimic the kind of work people on the sites normally do -- say, labeling photos -- but with small manipulations that allow them to test economics ideas.

So, a whole class of economics experiments ask people to split money between themselves and a stranger. People tend to act fairly generously when presented with that opportunity. But Rand said that some have questioned the results of that experiment because the money is seen as "free, manna from heaven." Using Mechanical Turk, Horton designed an experiment where the split came in the form of divvying up a bonus for real work.

Of course, for all these experiments to contribute to the existing literature, people like Rand have to prove that their work has validity in the offline world. He's confident that it does. At his talk, he'll present evidence for "why it's reasonable that the people you find on the online labor markets are representative of normal people."

And, Mechanical Turk workers are less likely to be WEIRD -- Western, Educated, Industrialized, Rich, and Democratic -- than college undergraduates, the normal subjects for economics experiments, which is probably a good thing.

Jump to comments
Presented by

Alexis C. Madrigal

Alexis Madrigal is the deputy editor of He's the author of Powering the Dream: The History and Promise of Green Technology. More

The New York Observer has called Madrigal "for all intents and purposes, the perfect modern reporter." He co-founded Longshot magazine, a high-speed media experiment that garnered attention from The New York Times, The Wall Street Journal, and the BBC. While at, he built Wired Science into one of the most popular blogs in the world. The site was nominated for best magazine blog by the MPA and best science website in the 2009 Webby Awards. He also co-founded Haiti ReWired, a groundbreaking community dedicated to the discussion of technology, infrastructure, and the future of Haiti.

He's spoken at Stanford, CalTech, Berkeley, SXSW, E3, and the National Renewable Energy Laboratory, and his writing was anthologized in Best Technology Writing 2010 (Yale University Press).

Madrigal is a visiting scholar at the University of California at Berkeley's Office for the History of Science and Technology. Born in Mexico City, he grew up in the exurbs north of Portland, Oregon, and now lives in Oakland.

Get Today's Top Stories in Your Inbox (preview)

CrossFit Versus Yoga: Choose a Side

How a workout becomes a social identity

Join the Discussion

After you comment, click Post. If you’re not already logged in you will be asked to log in or register. blog comments powered by Disqus


CrossFit Versus Yoga: Choose a Side

How a workout becomes a social identity


Is Technology Making Us Better Storytellers?

The minds behind House of Cards and The Moth weigh in.


A Short Film That Skewers Hollywood

A studio executive concocts an animated blockbuster. Who cares about the story?


In Online Dating, Everyone's a Little Bit Racist

The co-founder of OKCupid shares findings from his analysis of millions of users' data.


What Is a Sandwich?

We're overthinking sandwiches, so you don't have to.


Let's Talk About Not Smoking

Why does smoking maintain its allure? James Hamblin seeks the wisdom of a cool person.



More in Technology

Just In