The lack of ex-post project evaluation at the World Bank: One has no power

by Kelsey Lopez | Apr 15, 2014 | Bilateral organizations, Evaluation, International aid, JICA, Results, Sustainability, Sustainable development, Uncategorized, World Bank

The lack of ex-post project evaluation at the World Bank: One has no power

The World Bank has a huge repository of 8,483 evaluation resources in its e-library database, so naturally Valuing Voices was very interested in investigating how many of those resources were ex-post evaluations of past World Bank projects. After searching the term “ex post evaluation” in the e-library, I ended up with 260 hits from those initial 8,483 resources that were a match. This looked like great news to have so many potential ex-post evaluations to analyze from such a powerhouse in international development as the World Bank. From the initial 260 hits, I expected about 50 of them to be what we consider to be ‘true’ ex-post, which is an evaluation conducted a few years after a project has been completed to assess for factors such as sustainability and long-term effectiveness of the program after donor resources had been withdrawn.

However, when I began to sift through the resources in more detail, the results were not exactly as we had anticipated. In order to determine how many of the 260 “ex-post evaluation” hits were true ex-post the process was simple, albeit time consuming. I looked through every hit, reading the abstracts provided by the World Bank and investigating individual resources in more detail if they seemed promising. While doing this, I categorized each hit by document type, keeping a tally of all the totals. The results were as follows:

Did anything about these results surprise you? Yes, you read that right. Out of all 260 hits that came up from the search “ex-post evaluation” in the World Bank online database, an astounding grand total of one was a true ex-post evaluation of a past project. A bilateral counterpart, Japan’s JICA, has done 236 in 2009-2011 on past ODA projects, one of our rare stars in ex-post learning.

Suffice it to say, Valuing Voices was shocked by these results. There exists a clear need, based on this research, for the World Bank to contribute to the process of informing future projects by learning from past experiences, successes, mistakes, and community feedback through the valuable ex-post evaluation method. While impact and retrospective evaluations are indeed important, the nature of compiling many evaluations into one broad analysis doesn’t allow for a detailed assessment of how individual projects performed, especially when the respondents in many other multilateral ex-posts tend to remain government counterparts rather than local respondents. This type of comprehensive analysis of the long-term sustainability of completed projects can only be done by conducting ex-post evaluations for projects on a case-by-case basis.

Jindra Cekan (head of ValuingVoices) was invited to attend and speak at the World Bank’s Civil Society Organization spring meetings last week, and, armed with my findings, asked why we don’t see ex-posts at two sessions. Astrid Marroh, a senior staffer tasked with setting new strategy for the Bank, answered that longitudinal learning is, “a nut we have not yet cracked”.  Varun Gauri, writing the major World Development Report 2015 on Mind and Culture at the Bank, said that not only do Bank staff need to, “change the incentives from managing projects as managers to focus on the project’s ultimate aim,” but also that the Bank, “needs to follow the private sector’s approach by ‘Dog-fooding’ our projects– living our own projects“ (where private sector producers try and eat the dog food themselves).           

So what is the takeaway lesson learned form all this? Organizations like the World Bank are what set the precedent in international development, yet even this influential international organization fails to conduct regular ex-post evaluations. Despite having plentiful literature and recommendations on how to conduct ex-post evaluations and why they are important to the development process, it is clear that ex-post is not happening at the World Bank. Now is time for the organization to change the status quo and start valuing the voices of their project participants by conducting rigorous ex-post evaluations of their projects including feedback from the community level, in order to finally address this deficiency and establish a cycle of feedback loops and informed decision making that will benefit all involved – and make ‘development aid’ obsolete.