A leading development economist speaks on the virtues and limitations of a data-driven approach to healing the world's most intractable problems
Do free bed nets in some countries lead to more cases of malaria? Could anti-parasite pills raise school attendance in one country and have no effect in another? How cheap does preventative care have to be for low-income families to see the doctor?
There might not be a perfect way to answer these thorny questions on a country-by-country basis. But some leading scientists think the most rigorous answer comes from what they call "randomized controlled trials."
Esther Duflo is widely recognized as the world's leading advocate of randomized controlled trials in development economics. As a methodology, RCTs have been used for over a half-century in clinical medicine, where the effect of a drug or medical procedure is confirmed or denied in scientific experiments involving control and treatment groups. The use of RCTs to address global poverty is a phenomenon of the last decade, but it has caught on with the force of a paradigm shift in economics, public policy, and other disciplines.
Last year, Duflo* and her co-conspirator at J-PAL, Abhijit Banerjee, published a book called Poor Economics: A Radical Rehinking of the Way to Fight Global Poverty. The book overviews much of what they have learned through RCTs and otherwise, and it stakes a larger claim against "grand universal answers" and "sweeping conclusions" about poverty. Instead, they recommend a data-driven approach that seeks specific, targeted answers to what actually works, what works better, and what works cost-effectively.
Having been trained as a physicist and an engineer, I appreciate and support Duflo's scientific approach to fighting poverty. (Full disclosure: I'm on the board of Innovations for Poverty Action, a close J-PAL partner.) Yet, reading the book, two things repeatedly came to mind: First, the best science requires theory as much as experimentation. Data without good theory is only measurement, not knowledge, and powerful theory is often sweeping. Purely as a practical matter, theory helps to sustain us when we lack data. Second, Duflo and Banerjee seem uncomfortable with their own stance against grand answers. Most of the book's chapters conclude with sections that make general pronouncements about public health, education, microfinance, and entrepreneurship, often not entirely backed up by experiment. Their final chapter, titled "In Place of a Sweeping Conclusion," nevertheless offers five broad statements about poor people worldwide.
I asked Duflo some questions about RCTs and her book over e-mail...
KT: What do you think is the greatest contribution of RCTs so far in international development?
ED: First, we have made a lot of progress on debates that we would probably still be endlessly debating: Does giving bed nets away for free discourage usage? Do non-zero prices discourage use of preventive care? Second, we have identified some promising interventions, and those are starting to be scaled up, with deworming being a prime example. [KT: Deworming refers to anti-parasite pills which when given to children have been shown to increase school attendance.] Third, we have accumulated enough knowledge in some domains (primary education, for example) that we are beginning to have a broad systemic view of problems and solutions. Fourth, perhaps most importantly, the importance of trying things out under rigorous observation, of leaving open the possibility of failure, is now more recognized within governments and organizations. I think this shift in culture could turn out to be the biggest contribution in the long run.
KT: What is the greatest challenge in the way that RCTs are actually conducted, interpreted, or talked about in practice? Or, to ask the question a different way, what dominant practice within the community of people who run or read about RCTs would you most want to change?
ED: I don't have much to complain about actually. I think at the moment, people are being very creative, trying to push the boundaries in many directions: incorporating more theory, combining structural models and experiment, combining "lab experiment" with RCT, trying really bold things such as randomizing the wages for civil servants.