Question 11

What are your perspectives on the following comment? “The controversies around RCTs are often cast in methodological terms. However, the central issue is more a matter of how comfortable we are in dealing with uncertainty in our knowledge. Many of the strong proponents of RCTs are perhaps very uncomfortable with the idea of epistemological fallibilism.“

Lant: I think one point that is relevant is that the main “randomistas” (as opposed to the use of RCTs in scaled evaluations in Mexico, for instance) were very young people with no real experience in development. The young, having not had life kick the shit out of them (at least every now and again) and having not been completely sideswiped by historical events that no one saw coming (when and how it did)—e.g. the “lost decade” in Latin American in the 1980s/1990s or the “transition from communism” in the 1990s—tend to think in black and white terms. I think they are all a lot older and wiser now and actually, as I argued in my NYU presentation “the Debate is Over” have much less dogmatic and certain views—and have completely change their rhetoric and practices—but without every admitting how wrong they were.

I think it is pretty hilarious that Dean Karlan’s first book was “More than good intentions” (2011): “let me, who have never actually done anything of substance in development, tell you all how to do it right with my toy method”—and his next book “Failing in the Field” (2016) and The Goldilocks Challenge basically walks back nearly all the rhetoric about RCTs.

Editors: Epistemological disagreements are indeed what most clearly opposes randomistas and others. This is what we argue in the introduction to the volume. Randomistas claim to provide clear and universal answers to all problems and to derive policy prescriptions from them. As if policy decisions could be devoid of ethical and value considerations.

And the more we interact with randomistas, the clearer this becomes. For example, randomistas are completely unable to understand the difference between a theoretical protocol and its application in the field, yet the gap between the two is often great, and it is particularly great for RCTs given the heaviness and complexity of their protocol. Faced with these discrepancies, the response of randomistas is systematically technical in nature. Faced with people, in the flesh, who answer questionnaires differently from what is expected, faced with people who do not want an intervention, faced with interviewers who also behave differently from what is expected, the response cannot be technical.

The same applies to the ethical question: if randomistas refuse to take the ethical question seriously, it is because science comes first. The supposed advances of science outweigh the well-being and protection of populations.