Valuing Voices Better Ex-Post Evaluation White Paper: Nordics & the Netherlands

Donors often talk about #SustainableDevelopment, and we know they try. But do they return #expost to #evaluate it? Here are lessons from #Nordic (#Norway#Finland#Sweden & #Denmark) and the #Netherlands in our #ValuingVoices #WhitePapers. Let us know what you think… Thanks, Jindra & Preston Stewart..

The document(s) cover:
1) the search process,
2) how ex-post evaluations are defined and categorized,
3) what was done well by each country’s ex-posts,
4) sustainability-related findings and lessons, and
5) what M&E experts in each country can improve on ex-post evaluation practices.
6) Case studies of the Netherlands, Norway, and Finland
One big finding is that there were only 32 evaluations that seemed to be ex-post project, and only 16 of them actually were at least 2 years after project closure.

Executive Summary/ Abstract of the set of White Papers

Ex-post completion or ex-post project evaluations are surprisingly rare given widespread commitments to ‘sustainable development’. Given the well-known commitment of the Nordic countries of Norway, Finland, Sweden, and Denmark as well as the Netherlands, Valuing Voices, a consulting firm specializing in ex-post project evaluation, chose these countries as case studies. We hired an intern from Harvard College, who searched the national databases (in English only) and presented these results. The findings were surprising. There were far fewer ex-posts in total than expected. Those that were included a surprising number that was co-evaluated alongside final evaluations, although final evaluation illuminates relevance, effectiveness, efficiency (and the new OECD criteria of coherence), while ex-post project evaluation illuminates sustainability and impact two or more years after closure.  There were lessons for Ministries about the searchability as well as the quality (and in the case of Denmark and in part, Sweden), the dearth of ex-posts at all. Overall, the recommendations for the five countries researched for this paper include better and more standardized definitions of ex-post project, greater absolute numbers done, transparent sharing of those done in publicly searchable manners,  methodologically clear comparisons (baseline and midterm), and clarity in differentiating different evaluations.

 

There are caveats to these papers. This research was privately by Valuing Voices so the samples featured from the five countries were limited to public sources about ex-post and ex-post evaluations.  While we reached out to Ministry evaluative staff in all countries, only two made themselves available for consultation (the Netherlands and Finland) and provided a handful of additional ex-post evaluations.  This paper focuses on what such research yielded, not definitive findings of programs or multi-year country strategies that are funded for 20-30 years continuously, nor projects funded by country-level embassies which did not feature on the Ministry site. We focus on project bilateral project evaluations, not multilateral funding of sectors. We also presented these findings in a 2021 webinar, during which we received input that Sweden’s EBA has a (non-project) portfolio of ‘country evaluations’ which looked back over 10 or even 20-year time horizons.. Still, we found very few ex-post evaluations at EBA (see cite in Search, below).  No input was received by Norway or Denmark.

 

There are four papers in this series combined in this White Paper. First, the search across databases of all five Nordics (Norway, Finland, Sweden, and Denmark) and the Netherlands (pages 1-21). This is followed by country-specific, detailed, papers for those with ex-posts: The Netherlands (pages 22-37), Norway (pages 38-47), and Finland (pages 48-59).  We look forward to your feedback. Jindra@ValuingVoices.com for the Valuing Voices team.

Let us know what you think… Thanks, Jindra & Preston Stewart

Upcoming Jan Webinar: Lessons from Nordic / the Netherlands’ ex-post project evaluations: 14 Jan 2021

Upcoming Webinar: Lessons from Nordic / the

Netherlands’ ex-post project evaluations: 14 Jan 2021

In June-August 2020, Preston Stewart, our Valuing Voices intern, conducted through government databases of the four Nordic countries – Norway, Finland, Sweden, and Denmark – and the Netherlands to identify ex-post evaluations of government-sponsored projects. The findings and recommendations for action are detailed in a four-part white paper series, beginning with a paper called The Search.

We would like to invite you to a presentation of our findings, and a discussion of what can be learned about:

1) the search process,

2) how ex-post evaluations are defined and categorized,

3) what was done well by each country’s ex-posts,

4) sustainability-related findings and lessons, and

5) what M&E experts in each country can improve on ex-post evaluation practices.

One big finding is that there were only 32 evaluations that seemed to be ex-post project, and only 1/2 of them actually were at least 2 years after project closure.

 

We have many more lessons about conflicting definitions, that ex-post evaluation is not the norm in the evaluations processes of the five governments, that development programs could, if committed to ex-post evaluation, learn about sustainability by engaging with the findings from many more such evaluations, and to increase accountability to the public and for transparent learning, ex-post evaluations should be shared in public, easy-to-access online repositories.

Join us!

REGISTER: https://www.eventbrite.com/e/webinar-lessons-from-nordic-the-netherlands-ex-post-project-evaluations-tickets-132856426147 

Your ticket purchase entitles you to the webinar, its meeting recording, associated documents, and online Sustainability Network membership for resources and discussion. Payment on a sliding, pay-as-you-can scale.