How Amazon Speeds Up Economics Experiments

Social science experiments are important. They allow us to ground sociological and economic theory in real human behavior, and helps establish causal links between events.

"But the problem is that they are a a pain in the ass to do," said David Rand, a Berkman Center for the Internet and Society fellow. "You have to get someone to get down to the lab and spend time in transit. There are fixed time costs."

The Berkman Center is a leading institution for important research into the uses and impacts of digital technologies. We'll be previewing their regular brownbag lunches here on The Atlantic Technology Channel.

That's why he and fellow online social experiment pioneers like Harvard's John Horton are turning to Amazon's Mechanical Turk and other online labor markets to find willing participants. The services take the pain out of doing social science experimentation, decreasing costs and increasing the speed of idea development.

For example, Rand submitted a paper to a journal, which asked that he carry out another experiment before accepting his work. "I got the rejection on Thursday. I designed the experiment on Friday and by Sunday I had 500 people recruited," he said.

Rand will be talking about the value of these online research tools at the Berkman Center today. You can tune in live.

Mechanical Turk allows anyone to submit a set of small tasks, which are farmed out to a bunch of anonymous workers all across the world for tiny amounts of money. It tends to draw people from India, who both speak English and can make meaningful cash from completing tasks for a few dimes a piece.

How do these experiments actually play out? Researchers design experiments that mimic the kind of work people on the sites normally do -- say, labeling photos -- but with small manipulations that allow them to test economics ideas.

So, a whole class of economics experiments ask people to split money between themselves and a stranger. People tend to act fairly generously when presented with that opportunity. But Rand said that some have questioned the results of that experiment because the money is seen as "free, manna from heaven." Using Mechanical Turk, Horton designed an experiment where the split came in the form of divvying up a bonus for real work.

Of course, for all these experiments to contribute to the existing literature, people like Rand have to prove that their work has validity in the offline world. He's confident that it does. At his talk, he'll present evidence for "why it's reasonable that the people you find on the online labor markets are representative of normal people."

And, Mechanical Turk workers are less likely to be WEIRD -- Western, Educated, Industrialized, Rich, and Democratic -- than college undergraduates, the normal subjects for economics experiments, which is probably a good thing.