Evaluation for Transformational Change Award

Background info

What: A biennial award competition for evaluations in the field of international development that promote or result in transformational change. The award relates to evaluative work that is linked to the Sustainable Development Goals (SDGs).

When: Every two years. The first award was presented at the IDEAS Global Assembly in Prague in October 2019. 

Budget: The award is generously supported by the IOE and IEG.

Target Audience: Evaluators or evaluation teams from all over the world, commissioned by wide range of organizations, including multilateral- and bilateral organizations, not-for-profit organizations (NGOs, foundations), academic institutions, the private sector and government departments. To avoid conflict of interest, the IDEAS Board, IEG and IOE staff will not participate.

What will be awarded: The evaluation, not the intervention, through the evaluator or evaluation team responsible for the evaluation.

Goal: Create an incentive for conducting evaluations that promote or contribute to transformational change in the context of the SDGs.

Objectives: 

  1. To enhance awareness and understanding among different stakeholder groups of the value of evaluation in generating knowledge for learning and accountability in a domain of great importance and urgency, i.e., transformational change;
  2. To make evaluators aware of each other’s contributions and of what is possible in this area of work; 
  3. To build on previous experiences and develop good practices for evaluations in support of transformational change;
  4. To garner support from wider society for transformational change in the context of the SDGs.

Incentives to organize an award competition 

The role of evaluation – It will raise the profile of the field of evaluation and highlight the value it adds to efforts towards urgent and essential transformations.

Good cause – It will raise awareness of the urgency and importance of the transformational changes that are essential for achievement of the SDGs, and provide evaluators with incentives to participate in an award competition. 

New opportunities and contacts – It will provide excellent potential for networking, establishing partnerships and generating new evaluation opportunities.

Visibility – It will give exposure to the Evaluation for Transformational Change Award during the IDEAS Global Assembly, and generate a buzz in VOPEs and in social and mainstream media.

Promotion of good evaluation work and learning – It will provide insight into international evaluative trends and promote learning on how transformational change takes shape and can be promoted.

Prestige and recognition – A win or nomination will help the evaluators involved grow and gain profile by having their efforts recognized and acknowledged by experts.

Credibility – By publicly recognizing excellence, it will contribute to good practices in evaluation for transformational change and thus increase the credibility of evaluations that follow these practices.

How to make it work

Award Steering Committee. An Award Steering Committee has been appointed by IOE/IFAD, IEG/World Bank Group and IDEAS for a first period of four years. It provides oversight, leadership and coordination, including setting up calls for nominations, liaising with judges and potential participants, conducting and supporting marketing activities, and guiding the Award Secretariat. 

IDEAS support. The Award Secretariat, hosted by IDEAS, will administer the competition. In the first year this included one-time website development and the creation of a library of transformational change evaluations accessible through the website. The Secretariat will organize and facilitate the judging process, and organize and manage the Award ceremony at the IDEAS Global Assembly under the direction of the Steering Committee. The communication offices of IEG and IOE will support the dissemination efforts of the Secretariat.

Judging Panel. A Judging Panel of three or more judges will be set up by the Steering Committee to review the submissions and select the Award recipients based on formal terms of reference. Criteria for constitution of the Panel are as follows:

  1. Balanced in terms of gender. 
  2. Balanced in terms of international representation. 
  3. Representative of the diversity of knowledge systems that shape evaluation. 
  4. Credible, well-known specialists in evaluation, in transformational change in the context of the SDGs, and/or in policy design and implementation. They need to exhibit authority, expertise and a high profile, ideally known as ‘influencers’, ‘trendsetters’ or ‘change agents’ in evaluation for transformation.

The Steering Committee will decide at what stage the Judging Panel will be announced; visibility of the Judging Panel and the assessment process can be facilitated through the website, various (social) media publications, interviews and the Award Ceremony.

There should be no conflict of interest between the judges and the participants. 

Partners and sponsors. While the partnership for the Award remains with the three organizations, sponsors for Award events are welcome.

Participation fee. A participation fee could be considered in the future, depending on the visibility and recognition of the award in the evaluation community. A decision will be made on the basis of the interim assessment (see paragraph 30).

Broad timelines. The Steering Committee should aim for a 24-month period for the whole process, divided into five main stages:

  • Assessment of the previous edition and preparation of the new edition;
  • Public awareness and Call for nominations (registration);
  • End of submissions, review of nominations, translations and selection of eligible submissions;
  • Expert assessment by Judging Panel;
  • Preparation of the Award Ceremony and winner’s announcement during the IDEAS Global Assembly.

The assessment process and criteria. These should be clear, understandable and transparent. The scoring system will be drafted in advance and further refined by the Judging Panel.

The Award. The winner of each category will receive a financial award (US$ 5,000) and will be invited to write a chapter in an IDEAS publication after the Global Assembly. First and second runners-up will receive either a trophy or certificate.

Nominated evaluations. All nominated (and eligible) evaluations will be made accessible through the electronic library, which will be promoted as a reminder that the evaluators behind these reports have contributed to the greater good; this may inspire others (Such exposure is important in the international development sector and could be deemed just as important as the idea of a (financial) award or broader recognition).

Library. IDEAS will host, on the Award website, a library of “evaluations for transformational change” to highlight the category winners.

Marketing and media coverage. The communication functions of IOE, IEG and IDEAS will develop and implement a communication strategy to make sure the call for nominations and other information about the award are announced and disseminated appropriately.

Data protection. Although nominations may be shared for marketing purposes, their data should be protected and safely stored in line with the General Data Protection Regulation (GDPR) of the European Union.

The Award Ceremony during the IDEAS Global Assembly. The Award Ceremony will take place during an appropriate plenary event, such as a gala dinner or reception, and representatives of the host country or host institution will be asked to hand over the awards. 

Evaluation. After two or three Award periods, an interim evaluation will be conducted to inform future directions as well as the institutional commitments of IOE/IFAD, IEG/World Bank Group and IDEAS.

The Award categories and criteria

Award categories. Category names and descriptions should be clear and simple to attract nominations. Nominations should specify the award category for which they should be considered, and may identify both categories, as further explained below.  

There will be two Award categories:

  1. Evaluation in support of transformation. Evaluations can assess and inform efforts aimed at initiating, supporting and/or contributing to desired transformations, whether ecological, social, political, economic, scientific or technological, or a combination – for example, moving countries or rural areas out of poverty traps, developing green cities, reversing the destruction wrought by the COVID-19 pandemic, or creating circular economies.  
  2. Evaluation as transformation. An evaluation process, if implemented in an innovative, culturally responsive and empowering way, can in itself be transformative for one or more stakeholder groups, and/or for the initiatives in which they are engaged. 

Possible assessment criteria applied to both award categories are the following (to be further developed by the Steering Committee and the Judging Panel):

  • Credibility: The evaluation is based on a clear and reliable methodological design. Findings are clearly underpinned by (multiple) sources of evidence and there is evidence of intra- and inter-method triangulation. Potential biases to the validity of findings are adequately addressed or explained as limitations of the evaluation.
  • Innovation: The evaluation is based on a novel conceptual framework and/or methodological approach. What qualifies as novel is to be determined by the Judging Panel. The use of novel sources of data or using existing data more effectively (e.g., through data science) as well as the use of systemic approaches to address complexity challenges are examples of innovative approaches.
  • Influence: The evaluation shows evidence of the use of evaluation findings in organizational learning and decision-making or even beyond (e.g., in the public debate, as a source of influence of other evaluations, as a source of influence on intervention design, etc.). Moreover, there is evidence of a (potential) contribution of the evaluation to transformational change in the context of the SDGs and/or significant systemic changes at country, regional or sectoral levels. Evaluations may provide inspiration for overcoming barriers to change and shift societal and institutional debates in new directions to promote change.

The influence criterion will receive a relatively higher weight in the assessment process for the award category of evaluation as transformation.

AWARD Partners

IDEAS

The International Development Evaluation Association (IDEAS) was inaugurated in September 2002 to meet the need for a global professional association for internationally active development evaluators. IDEAS positions itself as the only global professional evaluation association which focuses on international sustainable development. Over the years it remains the only home for evaluators working globally and on global issues, with a global perspective. It is deeply involved in strengthening and promoting the profession, fostering capacity development, and improving and advancing evaluation theories and practice, methods and use of evidence. See more information here

Independent Evaluation Group (IEG)

The Independent Evaluation Group evaluates the work of the World Bank Group to find what works, what doesn't, and why. IEG evaluations provide an objective assessment of World Bank Group results, and identify lessons learned from experience. Through independent evaluation, IEG is helping the World Bank Group achieve its twin goals of eradicating extreme poverty and boosting shared prosperity.  See more information here

International Fund for Agricultural Development (IFAD)

The Independent Office of Evaluation of IFAD (IOE) is the independent evaluation entity of the International Fund for Agricultural Development, the specialized agency of the United Nations founded in 1977 to eradicate poverty and hunger in rural areas of developing countries. IOE promotes accountability and learning through independent, credible and useful evaluations of IFAD’s work, in order to increase the impact of IFAD’s operations for sustainable and inclusive rural transformation through excellence in evaluation. In 2019, IOE sought to initiate the Evaluation for Transformational Change Award, together with the Independent Evaluation Group (IEG) of the World Bank and IDEAS, in recognition of the fact that transformational change is crucial if our societies, economies and our relationship with the environment are to become sustainable. See more information here