Podcast: When Field Research Goes Wrong

Dean Karlan PhD ’02, a professor of economics at Yale University, published Failing in the Field: What We Can Learn When Field Research Goes Wrong this fall, coauthored with Jacob Appel.

The book addresses some of the most common failures of field research and randomized control trials and argues for a more transparent culture in academia.

Karlan, the founder of Innovations for Poverty Action, a research and policy non-profit that discovers and promotes effective solutions to global poverty problems, has had his own share of failures, many of which he chronicles in the book. Listen to a podcast interview with Karlan about his latest book, published this fall by Princeton University Press.

(Read the episode transcript.)

“When we are out there in the field trying new ways of collecting data, if we don’t collect our failures on the research process, we’re not going to learn as well as we could,” says Karlan. “Let’s document some of these and let’s also create a little more of a culture of sharing some of these stories. We’re all going to do better in the way we do research if we’re a little more open about learning from our failures.”

After detailing the kinds of pitfalls researchers typically fall into – from conducting field research from afar to choosing inappropriate settings for randomized trials – Karlan shares stories of his own failures in the developing world conducting research on topics like micro-lending and better bed nets for malaria prevention.

Karlan also points to the failure of academia and scholarly journals to share and accept “no-impact” results, despite the enormous amount of work behind them.

“That’s a problem, because as long as it was a good theory and there’s a plausible story as to why A should cause B, if you learn that it didn’t, that’s important, and it’s just as important as publishing that it does cause B,” says Karlan.

To encourage more sharing of failures in research settings, Karlan and Appel created a space on the World Bank’s Development Impact blog for researchers to contribute some of their own valuable lessons from failure.

“Our hope is that enough people will start writing some in that we’ll have a living history of them, so when you’re doing a project, someone can say I know someone who tried that and failed. Write it up, get it out there, and let people know it happened,” he says.

Listen to the full interview above then visit the Alumni Association’s Slice of MIT Podcast page to hear the archive of past episodes.