What did you find about how suitable RCTs are for emergency situations? Are there ethical and human rights considerations particular to their use in such situations?
Lant: They are not at all suitable for use in emergency situations, for two reasons. First, the feedback loops are just too slow (see this paper of mine with Sara Nadel about how quick feedback loops need to be in uncertain situations). For emergency situations one needs rapid and accurate reporting about the on-the-ground facts, not a comparison of counter-factual. When I was the task manager of a set of crisis safety net programs in Indonesia during the financial (and political) crisis period in 1998-2000, we did not have any RCT type evaluation of any of the programs as we knew the “emergency” nature of the programs meant they would be done before any lessons could be learned and applied.
Second, the essence of an RCT is a “counter-factual” of people who did not get the program, which, in emergency or humanitarian relief situations seems completely unethical. One could do variation in treatment arms perhaps across regions (of an “equipoise” nature that are variations in design about what is going to work about which there is uncertainty), but there this is more like A-B testing than what is a traditional RCT.
Editors: we fully agree with Lant, and the pandemic situation is a good illustration of this.
Comments 1
Pingback: Panel responses to questions not addressed during the RCTs in Development: A critical perspective Book Launch event | IDEAS