Another source for inspiration and international best practices has just become available on the internet. In 2014 the second international conference on evaluating climate change took place in Washington, DC. The follow-up book, building on some of the most interesting and innovative presentations and discussions at that conference, is now published on the website of Springer Verlag. Its title is “Evaluating Climate Change Action for Sustainable Development” and it has been edited by Juha Uitto, Jyotsna Puri and I, with support from Lee Cando-Noordhuizen, and it was a joy to work on. Thanks to a grant from the German government the publication is open source and thus free to download. If you want to have a paper copy, you will still have to pay, but electronic copies of the whole book or individual chapters are freely available.
I have contributed two chapters to the book, the first written together with Lee Cando on whether the increasing demand for impact evidence can be met by the biggest players in the field. That chapter paints a rather gloomy picture. Based on seven major evaluations at corporate and programme level we conclude that there is evidence that the portfolios of the main actors in global public funding of climate change are effective. However, these portfolios are outspent in a huge way by government subsidies for unsustainable use of energy resources. In other words: anything that is being achieved is undone by activity elsewhere that is a magnitude higher than the positive action. We have the evidence that shows why success in climate change at the micro level is not visible at the macro level. We used to have the naïve idea that bringing evidence to policy makers would change policies…
The second issue is that reporting on impact on climate change is only possible for mature portfolios that are consistent in their approach to measure the reduction of greenhouse gas emissions. Most of the portfolios are not mature and are inconsistent.
The third and perhaps counter-intuitive evaluation finding is that the portfolios are rated relatively low as far as efficiency is concerned, but relatively high for effectiveness. This seems odd. One would expect low efficiencies to also lead to low effectiveness, but this is not the case. This finding gains authority because they were repeated in all evaluations – whether they were done by evaluation units of the multilateral banks, the UN or independent evaluation professionals. In science we often see a call for “reproduction” of research results, and many a theory has been abandoned because initial positive testing could not be replicated. For climate change action seven major evaluations on portfolios all independently came to the same conclusion that climate action tends to be inefficient, but effective.
The explanation seems to be that climate action involves many actors and tend to be multi-dimensional in its approach; in other words, we are talking of complex actions that aim to change government regulations, investment decisions in the energy sector, use of energy in households, introduction of technological innovations and support from the banks, preferably all at the same time and in harmony. While this is a recipe for delays, because one or two of the key actors take more time to prepare what they need to do, the outcomes of interventions tend to be solid and in principle more sustainable than alternatives. Now if only they would not be outspent by action in the wrong direction…
While we seem to have arrived in a post-truth world with alternative facts, it is good to remind ourselves that evaluators will continue to provide transparent and objective evidence on what happened when, where and for whom, under what circumstances… These question are difficult to answer, and the range of demand for impact evidence is what I discuss in the third chapter of the book. While politics seem to become more populist, on the evaluation side we continue to recognize the full range of impact questions and should continue to deliver evidence that meets this range, regardless whether this is what the mighty news makers seem to think is relevant.
Similar dilemmas, trade-offs and lack of coherence between actions is discussed in various regions and countries throughout the book, and I can recommend it heartily. The good thing is that downloading and reading is not going to cost you anything except your subscription to an internet provider. Have a look and see whether any of the chapters has a particular appeal to you!
Rob D. van den Berg
President, IDEAS and Visiting Professor, King’s College London