Are the “randomistas” answering to the methodological and ethical critiques put forward in the book? Is there at present a playing field for scientific confrontation?
Lant: I think not. At present the randomistas have too much financial support, too much power, too much prestige to be willing to engage in debate. A few months ago a debate was arranged between two Nobel Prize winners, Angus Deaton and Abhijit Banerjee, about RCTs. Several people who were there describe the occasion as that, after Agnus had made his presentation Abhijit got up and said ‘I don’t feel like debating or engaging with what Angus said so let me just tell you about our research.” I was at Harvard Kennedy School (and so around the hotbeds of the randomistas) and in the early 2000s we would do joint courses about “the latest in development economics” for development practitioners who wanted a refresh that had people like Dani Rodrik and Ricardo Hausmann together with Esther Duflo and Rohini Pande. But, as they got more and more support they basically were too busy doing what they were doing and realized they no longer needed to share a platform or even engage intellectually with those who weren’t in their camp.
In, I think, 2008 Bill Easterly organized a two day conference that assembled randomistas and non-randomistas at Brookings (the results of which are published in the 2009 book “What works in Development: Thinking Big and Thinking Small”). My personal impression is that among an audience of fellow economists (including future Nobel Prize winners like Angus and Paul Romer) they really could not justify their position and, in a pretty heated couple of days, they mostly were unpersuasive and felt attacked and frustrated. So, again my impression, is that they decided that they didn’t need the approval or support or agreement of their fellow economists and so have systematically refused to engage in open, public, debate in person or in publications. Which, I have to say, makes a ton of sense. If you have money raining on your heads and adulation from the semi-intellectual media and can do anything you want to do, why bother addressing critics?
Editors: This book was precisely intended to launch a controversy. We solicited a dozen randomistas: either they declined (Duflo, Karlan, among others) or never responded. The only ones who responded positively believe in the virtues of RCTs but recognize their limitations (Ogden, Morduch). In the introduction, we discuss how randomistas respond to criticism:
The absence of public dialogue does not prevent the randomistas from adapting their methods and practices (Ogden, Chapter 4), even though responses vary by groups of researchers. Some make their data available, thereby encouraging replications. Some acknowledge the legitimacy of methodological pluralism and combine RCTs with other methods. Some focus in detail on the impact mechanisms and processes and use specific theories (based mainly on behavioural economics). Others take the question of external validity seriously and ramp up the number of case studies in different settings (the special issue on microcredit edited by Banerjee, Karlan and Zinman (2015) is a typical example of this; Bédécarrats, Guérin and Roubaud, Chapter 7), or reanalyse ex-post a number of RCTs (Meager 2019). Still others take the question of ‘thinking small’ seriously and focus on large-scale programmes and national policies. On the question of little bearing on public policies (Pritchett, Chapter 2), some randomistas create dedicated bodies if not become decision-makers themselves.
What remains to be seen is the extent to which the implementation of this new generation of RCTs in development economics can withstand contingencies on the ground and really evaluate more complex interventions. At the risk of repeating ourselves, we must emphasize the fact that one of the cruxes of the debate is this obsession with the protocol, seen as the priority over its feasibility and its ethical issues. Yet the more complicated the programmes and policies studied, the more likely it is to find tweaking, compromises made and also risks of compromise with the initial protocol. The point is not just to adjust the technique, but to relinquish a scientistic epistemological position in the sense defined above.