In @Saundra_S’s absence, it’s only fitting that someone step in to compile and share the barrage of articles that have been regarding or related to RCTs in the past month or so due to the release of More Than Good Intentions: How a New Economics is Helping to Solve Global Poverty and Poor Economics: A Radical Rethinking of the Way to Fight Global Poverty. En homage to Good Intentions Are Not Enough, here’s my attempt to do so, in no particular order. Please do share additional posts and articles via the comments section.
I am also including at the bottom the content of Friday’s “tweet debate” regarding RCTs that demonstrates some of the polarities in thinking, to which I hope I’ve done justice to those who participated.
Here’s to the continued conversations…
World Bank’s Development Impact Portal: “Dean Karlan’s New Book: RCTs – this time it’s personal” and “Stuff You Cannot Randomize” and “Esther Duflo’s refreshing perspective on fighting poverty” and “Is it the program or is it participation? Randomization and placebos”
Boston Review: “Small Changes, Big Results,” a forum on applying behavioral economics to global development.
Tom Murphy in the Huffington Post: Proven Impact: Better International Development Through Rigorous Research
Nicholas Kristof in the New York Times: “Getting Smart on Aid”
Arvind Subramanian, Center for Global Development: “Nicholas Kristof and Aid”
The Wall Street Journal: Is ‘Nudging’ really enough? and Bill Easterly’s review of More Than Good Intentions
Owen Barder: “Do Economists Have Better Tools?”
Rachel Strohm: “Development Aid in Context”
Ben Goldacre in The Guardian: “How can you tell if a policy is working? Run a trial”
Madeleine Bunting in The Guardian: “Duflo and Banerjee take the guesswork out of policies that help the poor” (on the glaring omission of an understanding of power)
Guardian Editorial: “In praise of…Esther Duflo and Abhijit Banerjee”
Duncan Green of Oxfam GB reviews Poor Economics and guest post by Karl Hughes “Can we demonstrate effectiveness without bankrupting our NGO and/or becoming a randomista?”
Rick Davies on MandE News “Do we need a minimum level of failure?” and “Relative rather than absolute counterfactuals: A more useful alternative?”
Ian Thorpe: “The problem with the big picture”
Tales from the Hood on the necessity of “good vibes”
Development Policy Blog: “Why abundance evidence won’t inevitably lead to abundance”
Lawrence Haddad of IDS: “A New Harvest of RCTs?”
Good Intentions Are Not Enough: “’More Than Good Intentions’ and ‘Three Cups of Tea’”
Ben Ramalingam at ODI: “When will we learn how to learn?”
Tate Watkins at Short Sentences: “Why most aid won’t be ‘good aid’”
Interaction Institute for Social Change: “95% Failure”
Mike Klassen at Doorway to the Universe: “At Second Glance”
GiveWell blog: “Suggestions for the Social Sciences”
Marc Bellemare: “Thoughts on the RCT Debate”
Chasing Fat Tails: “Don’t Bite the Silver Bullet”
David Week: “Who guards the RCT guardians?”
The Monkey Cage: “Why aren’t there institutional review boards for the development industry?”
Centro Studi Luca d’Agliano: “Lentils for Vaccines” and “Dangerous Liaisons: On the ambiguous relationship between development and development economics”
Stuff Expat Aid Workers Like: “#60 Randomized Controlled Trials”
Dennis Whittle: “If not randomized trials, then what?”
Ian Thrope: “The development ideas hype cycle”
Rasmussen, Malchow-Møller & Andersen, University of Southern Denmark: “Walking the talk: the need for a trial registry for development interventions”
Mark Engler: “On Nicholas Kristof’s Boneheaded ‘Paean to Economists‘”
Philip Auerswald: “Why Randomized Controlled Trials Work in Public Health…and Not Much Else.”
Tim Ogden: “Revisiting Objections to Randomized Control Trials” and “Making RCTs Better” (love the call for humility)
David Roodman: “The Smartest RCT Critic”
Marc Maxson: “What Aid Experts Could Learn From Fantasy Football”
Brendan Rigby: “The decline effect: a storm on the horizon for RCTs?”
@Dochasnetwork May 19, 10:29pm @intldogooder @owenbarder I’ve a huge problem with Randomised Trials. Immoral, in my book. Overkill. Measuring Outcomes should suffice.
@intldogooder 4:18am Moral ? is how to justify control group that doesn’t get services & how they are later brought into progs @dochasnetwork @owenbarder #RCTs
@viewfromthecave 4:23am @intldogooder From my understanding of RCTs there are multiple ways of dealing with ethics of control groups. One is lack of funds which…means that only so many can get the services, so some will naturally be without. If enough resources for all, interventions…can be gradually timed to get a comparative, but not deny services. Or all can get different kinds of services, etc…Maybe @poverty_action will be kind enough to post on the ethics of RCTs and determining control groups…
@intldogooder 4:55am: The “how” discussion is important given attn RT @viewfromthecave: Maybe @poverty_action will be kind enough to post on the ethics of #RCTs
@DevEconHealth 4:31am @intldogooder @dochasnetwork @owenbarder Not sure that is a serious moral ?. Same issue with any proj trgt group and med. trial.
@viewfromthecave 5:24am via @intldogooder You read Karlan’s book, right? [Ethics] is addressed a bit in the beginning
@intldogooder 5:33am @viewfromthecave I did and you’re right, they include that & many other caveats. Important for me to keep beating the “how matters” drum…Have seen such poorly-executed evals/research in the field that I have visions of that now for wide-scale #RCTs
@poverty_action 6:32am @intldogooder @dochasnetwork how about the ethics of potentially wasting limited money on unproven projects?
@intldogooder 7:00am Also vital but deficit mentality has its drawbacks RT @poverty_action: @intldogooder @dochasnetwork ethics of potentially wasting limited $?
@poverty_action 7:04am @intldogooder which are..?
@intldogooder 7:19am @poverty_action econ tools “often used unhelpfully due to hubris & a shocking level of ignorance about being studied” http://ow.ly/4Z9UN
@poverty_action 7:19am @txtpablo @viewfromthecave @intldogooder any ethical issues with using control group for medical trials? Or should we just hope for the best
@intldogooder 7:20am @poverty_action thus why sharing more of your experience on HOW to do #RCTs well so important, many of us grounded in assets-based approach
@txtpablo 7:21am @poverty_action @viewfromthecave @intldogooder Well, let us know! For sure there are some discussions around.
@poverty_action 7:27am @intldogooder we run trainings for partner orgs & run exec ed with J-PAL http://bit.ly/mwVNkY this WB book is great http://bit.ly/iP5848…but always grateful for suggestions of how we can do more!
@intldogooder 7:33am Lets acknowledge differences btwn social change & medical science @txtpablo @poverty_action @viewfromthecave re: ethics of control groups
@poverty_action 7:37am @intldogooder @txtpablo @viewfromthecave true, but not an argument against striving for the greatest rigor in evaluation possible
@txtpablo 7:40am @poverty_action: @intldogooder @viewfromthecave agree, but rigor in #evaluation is not only RCT, it is much more
@poverty_action @txtpablo there are other rigorous methods but RCTs generally agreed to be the best way of attributing causal impact
@txtpablo @poverty_action I agree it is good for causal impacts but in simple contexts, where you can manipulate few variables, but in complex ones?
@poverty_action @txtpablo do you have a specific example you are thinking of?
@txtpablo @poverty_action what causes a revolution? what causes development in a region? RCT only tells us part of what we need to know
@poverty_action @txtpablo spot on. Lots of important questions that RCTs can’t answer. But for the ones they can, they are best method going.
@intldogooder 7:41am Does genuine learning on an individual level always come from rigor? RT @poverty_action: @intldogooder @txtpablo @viewfromthecave @itsjina
@BonnieKoenig 7:44am @intldogooder I don’t believe in ‘always’, ‘best’ etc.. See previous tweets @poverty_action @txtpablo @viewfromthecave…Yes these differences r impt 4 diff situations
@viewfromthecave 7:42am @intldogooder Both impact livelihoods of many and known solutions can lead to improvement for many in the future #DevilsAdvocate
@intldogooder 7:47am At times this is like atheism vs. faith debate. What is comfort level with not knowing? @txtpablo @poverty_action @viewfromthecave @itsjina
@sabes_a_arim 7:50am @intldogooder what?
@viewfromthecave 7:50am @intldogooder There has to be comfort in not knowing everything, but that does not mean we cant know more @txtpablo @poverty_action @itsjina
@poverty_action 7:53am @viewfromthecave @intldogooder @txtpablo @itsjina will always be unknowns, but not knowing what works to fight poverty IS uncomfortable
@intldogooder 8:12am What works is investing in people, not just ideas- must pay attn to that 2 @poverty_action @viewfromthecave @intldogooder @txtpablo @itsjina
@viewfromthecave 8:25am @intldogooder no disagreement on that.
@poverty_action 8:15am @intldogooder we should test that theory….