Catalysts for Ex-post
Learning
Good news: Kudos to these self-Sustainability Catalyst organizations learning from sustained and emerging results of project activities and using mixed methods, including valuing the voices of participant feedback. They are doing this by evaluating, learning, redesigning, funding anew or differently. We have much to learn from their efforts to listen to participants and partners… OVER 90 PROJECTS ARE FEATURED BELOW
* The Adaptation Fund has invested in ex-post sustainability and resilience training materials (2021-22) and made a commitment to multiple years of sustainability evaluations that are 3-5 years ex-post. The first two pilots of evaluating sustainability and resilience to climate change evaluations are Samoa (2021), Ecuador (2021), and Argentina (2023).
* Asian Development Bank post-project synthesis (desk study, 2010) and while no post-project evaluations, they have final evaluations up to 10 years later, e.g. Nepal Hydro project
* European Commission on Transport (to be reviewed) and European Union on Rural Development (2012), Nano-Technologies (2011) and a 2018-discovered treasure trove of ex-posts
* Federation of the Red Cross/ Red Crescent/ Ethiopian Red Cross: Valuing Voices research on sustainability: ERCS Tigray’s Food Security program and blogs on self-sustainability at final evaluation and trade-offs during impact (2014)
* The Government of Finland’s Nicaraguan Livestock Development (2013) and Nepal Environmental Administration (2016)
* Germany’s KfW Bank has mostly desk studies but interesting measures on programs in 11 countries
* Germany’s GIZ via Humbolt University Mekong Delta Watershed (2014) (unpublished) and GIZ Science Education Indonesia (2013)
* Government of Australia on Stronger Families and DFAT Indonesia’s roads loan/grant project
*Integrity Action Sustainability and Citizen-Centered Accountability (2020)
* Interamerican Development Bank, Social welfare & agriculture in Brazil (2011)
* ILO via the Springfield Centre Enterprise Development in Sri Lanka (2020)
* Japan’s JICA, who we consider the world leader in ex-post evaluation, about 2,500, including 3 years ex-post project evaluation of grants and loans and further monitoring 7 years after. Sectors include agricultural irrigation, watershed management, erosion or water: Cambodia (2017), Morocco (2017), Indonesia (2008), Philippines, and Vietnam. JICA for Disaster Risk Reduction in the Philppines (2017), Infrastructure: VietNam (2017) and Tanzania (2020), and an eval overview. I continue to be impressed with JICA’s candor such as the challenges of ex-post and failures of projects (Sri Lanka 2015). JICA even evaluates climate change support leading to measurable change (VietNam 2019)
* The Government of Nepal and UK Aid (Koshi Hills)
* The Government of the Netherlands’ Nicaraguan health (2012), and many ex-posts within 10 Public Infrastructure (2015) and Chronic Crises (2019)
* Norwegian Government’s Kenyan Rural Development (2009) and Roads & Infrastructure for Transport (2015)
* OECD, 200 evaluations (!) of partner countries, albeit mostly ending research with partners, not participants
* OXFAM’s Myanmar Resilience (2016)
* Save the Children Ethiopia Child Sponsorship (2020) and a Scoping Guide which includes an evaluability rubric of support and evidence (2018)
* Sweden’s SIDA Oxone Depletion China (2005)
* UKAid, FSD Africa and SEEP via L-IFT: Long-Term Performance and Evolution of Savings Groups (2019)
*UK’s EveryChild 5-year ex-posts in Cambodia, India, Malawi, and Nepal checking on ‘responsible exits’ (2021)
* UN’s IFAD’s Project Performance Evaluations are ex-post (2/3 as desk reviews, 1/3 with fieldwork)
* UN’s WHO Practical Guidance for Conducting Post-Project Evaluations in Adolescent Sexual and Reproductive Health (2019)
* USAID Food For Peace’s food security evaluation of Bolivia, Honduras, India, and Kenya in 2015 with the Bolivia Food Security case. USAID (by Cekan) via MSI Ugandan Sustained Outcomes in Primary Education (2018) and ex-post systems synthesis for Ghana, Namibia RSA and Uganda (2018) (Uganda Peacebuilding (2014, to be reviewed. A new study using contribution analysis is Cambodian Market Development (2016). USAID Microlinks Zambia and Cambodia (both 2016). USAID Regional Citizen Engagement and Reconciliation in Thailand (2017). USAID Water/ Sanitation/ Hygiene ex-post series: Madagascar, Indonesia, Ethiopia, India, and Senegal (PEPAM) (2017-19) and Ghana (2018) and Mozambique (2020). Also Timor-Leste Community Governance (2019). USAID also recently funded Long-Term Impact Evaluation Guidance (2018). Cekan of Valuing Voices worked on Mail’s ex-post on unemployed youth (2022). Also civil society evaluation of 3 Serbian projects (2017)
- US Department of State (likely ex-post on Rule of Law) meta ex-post evaluation across 32 countries in 2019.
* US CARE, mostly Village Savings and Loan Associations (VSLA) CARE Zanzibar, Uganda (2013), Malawi (2018), Benin (2019), and CARE, OXFAM, PACT Cambodia, plus other sectoral ex-posts: Zambia Water Trust (2014), Ethiopia Adolescent Girls Economic/ SRH (2017), and Mozambique Climate Resilient Agriculture (2019).
* US Catholic Relief Services CRS/ Ethiopia (2004), CRS/Niger with video in English and French (2015) and CRS Country X (approval pending, 2016), and Uganda SILC with ITAD (2019)
* US Inter-American Foundation (part of US Government) – 30 post-project evaluations but summaries only
* US Lutheran World Relief: Drought resilience with the BMGates Foundation (Niger, 2013) Cash crop value chain Tanzania (2018) and Cash cooperatives & gender, Nicaragua (2019)
* US Mercy Corps MILK Program in Niger and Community Recovery in Central Asia
* US Millenium Challenge and Plan International Girls Education in Burkina Faso (2016) – to be reviewed
* US Partners for Democratic Change
* US Plan International: Kenyan Livelihood Project and one from the Philippines
* US PCI Bolivia and PCI Indonesia on Maternal Child Health with Notre Dame support
* US Save the Children Long-term impact evaluation in Ethiopia
* West African Development Bank – ex-post “socio-economic impact evaluation” – no proof other than final eval.
* World Bank Nigeria Fadama Livelihoods Project (2014) and two other projects in India and Kenya
* World Vision Water Sanitation and Hygiene WASH studies: Ghana, Mali, Niger and Ethiopia including the detailed WASH Ethiopia ex-post technical report (2004), and a Sustainability Learning Brief (2019) and with UNC-Chapel Hill, a WASH Ghana ex-post summary (2016)
*World Wildlife Fund Ecosystems (sites anonymous) (2020)
* Government of Zanzibar and EDC- Impact of Radio Instruction on Education in Zanzibar (2015)
TOTALS: 39 organizations are sustainability catalysts… 90+ post-project sustained outcomes and impacts evaluations in our database and in around 400 publicly available sustainability evaluations with participant in-country worldwide so far (of over 1000 reviewed).
Notably, key international development players are missing any field-based ex-post evaluation: France Development, Canada’s CIDA/ Global Affairs, African Development Bank, Sweden and Denmark’s governments, among others.
Please contact us if you want to post your ex-post/ post-project or if we can evaluate a new one for you as these are less expensive than you may think!
Inspiring Guidance and Sustainability Leaders
* Adaptation Fund Ex-post Evaluation process and training for sustainability and resilience to climate change (2021) and the first two ex-post evaluations of sustainability and resilience: Samoa and Ecuador (2022, Annexes to the Board document)
* WHO Reproductive Health Ex-post Manual (2019) “The project has ended but we can still learn from it”
* Dr. Eric Sarriot from Save the Children presents at the Quarterly Meeting of the Resource Team of the Scaling-up Community of Practice (CoP) on December 5, 2018.
* JICA’s website of 2,500 ex-posts (ex-post evaluation, and even later ex-post monitoring)
* Valuing Voices Evaluability Checklists
* Valuing Voices Exiting for Sustainability Checklists
* Valuing Voices SEIE guidance
* Cekan/ova SmartHeadTalks Interview (November 2020)
* Engel Jones Episode with Jindra Cekan, PHD (April 2019)
Related Blog Posts
USG/USAID
Reblog USAID’s Unpacking the Drivers of WASH Sustainability
Reblog Unpacking the Drivers of WASH Sustainability In Nov 2019 I highlighted several water/ sanitation/ hygiene ex-posts happening at USAID's E3 Water Bureau. They've just come up with a synthesis, which is important, both in learning and transparency for...
Sustainable Exit Strategies: USAID vs. EU
Sustainable Exit Strategies: USAID vs. EU Once malnutrition has decreased, students’ attendance has risen, or the number of small businesses has doubled, program implementers may be quick to pack their bags and leave. But the impact of their work may be...
Sustaining “Sustainable Development”
Sustaining "Sustainable Development"? As a global development industry, we have almost no evidence of how (un)sustained the outcomes or impacts of 99% of our projects because we have never returned to evaluate them. But from early indications based on...
Learning from a river of ex-post project evaluations, tools and guidance… Thanks USAID!
Learning from a river of ex-post project evaluations and tools... Thanks USAID! Dear ex-post aficionados. It's raining ex-post project evaluations. Here's hoping learning from such evaluations in water/ sanitation, maternal/child health and even capacity building/...
Setting a higher bar: Sustained Impacts are about All of us
Setting a higher bar: Sustained Impacts are about All of us Global development aid has a problem which may already affect impact investing as well. It is that we think it's really all about us (individuals, wealthy donors and INGO implementers) not all of us (you, me,...
Can’t wait to learn from post-project sustainability evaluation? If not why?
Can't wait to learn from post-project sustainability evaluation? If not why? A colleague who has been promoting ex-post sustainability evaluation in her organization questioned my claim that doing them had “benefits” for future programming. It was an...
Making up your mind. Prioritizing and making it happen
Making up your mind. Prioritizing and making it happen * As Dr. Martin Luther King Jr. said, "every man must decide whether he will walk in the light of creative altruism or in the darkness of destructive selfishness". ...
Embedding Sustainability Everywhere – All Five Slices Now
Embedding Sustainability Everywhere - All Five Slices Now It has been a tumultuous year, and next year does not look like we will have much stability as a respite. As domestic concerns grow larger in two huge economies, US and UK, the question of the...
Presenting Lessons on (post-project) Sustained and Emerging Impact Evaluations from the U.S. AEA Conference
Presenting Lessons on (post-project) Sustained and Emerging Impact Evaluations from the U.S. AEA Conference Dear readers, attached please find the Barking up a Better Tree: Lessons about SEIE Sustained and Emerging Impact Evaluation presentation we did...
Maximizing what we’ve got… Time is now!
Maximizing what we've got... Time is now! We had a stirring conversation here in D.C. with someone very knowledgeable about sustainability; this person is a strong proponent of local ownership of all development. They also said...
JICA
Sustaining “Sustainable Development”
Sustaining "Sustainable Development"? As a global development industry, we have almost no evidence of how (un)sustained the outcomes or impacts of 99% of our projects because we have never returned to evaluate them. But from early indications based on...
Wishing for ex-post evaluation Christmas Lights rather than Needles in Haystacks
Wishing for Ex-Post Evaluation Christmas Lights Rather than Needles in Haystacks This is what the life of most ex-post evaluation researchers looks like, mostly without the counting congratulator: I recently spent three days looking for ex-post...
When Funders Move On (Originally published by Stanford Social Innovation Review 03/15)
When Funders Move On: Donors and nonprofits need to learn more about how to help program participants keep progressing after the support ends. Imagine standing in Detroit or South-Central Los Angeles. A team of experts has come to help you out of grinding poverty....
Data for whose good?
Data for whose good? Many of us work in international development because we are driven to serve, to make corners of the world better by improving the lives of those that live there. Many of us are driven by compassion to help directly through working...
The lack of ex-post project evaluation at the World Bank: One has no power
The lack of ex-post project evaluation at the World Bank: One has no power The World Bank has a huge repository of 8,483 evaluation resources in its e-library database, so naturally Valuing Voices was very interested in investigating how many of those resources...
What are the prospects for self-sustained impact at the Macro level? (Part 1)
What are the prospects for self-sustained impact at the Macro level? (Part 1) After studying various impact evaluations, it became clear that different development projects also have different scales, with multilaterally-funded macro-level being focused on higher...
(I)NGOs
Hard-wiring and Soft-wiring in Sustainability via health program examples
Hard-wiring and Soft-wiring in Sustainability via health program examples: by Laurence Desvignes and Jindra Cekan/ova Overview We all want things to last. Most of us joined the ‘sustainable development’ industry hoping our foreign aid projects not only do good while...
Aid providers: More puzzle pieces, including unexpected outcomes; ours is not the whole picture
Aid providers: More puzzle pieces, including unexpected outcomes; ours is not the whole picture When we did our first ex-post evaluation/ delayed final evaluation in 2006 in Niger for Lutheran World Relief (LWR) funded by the Bill & Melinda Gates Foundation (pg75...
Who is accountable for the ‘sustained development’ of those who suffer, and for how long? US/Czech and Kenyan Views
Who is accountable for the ‘sustained development’ of those who suffer, and for how long? Some of the rarely discussed myths of ‘sustainable development is that the aid we donors and implementers bring will help everyone, and that recipient governments can take...
ReBlog Ex-post Eval Week: Evaluating Peace for Sustainability by Peter Kimeu Ngui
Reblog Ex-post Eval Week: Evaluating Peace for Sustainability by Peter Kimeu Ngui Reblogged from https://aea365.org/blog/ex-post-eval-week-evaluating-peace-for-sustainability-by-peter-kimeu-ngui/ January 20, 2021 Peter Kimeu Ngui Peace is with you. My name is Peter...
ReBlog Ex-Post Eval Week Tips for Conducting an Ex-Post Evaluation by Wendi Bevins
Repost: Reblog: Eval Week: Tips for Conducting an Ex-Post Evaluation by Wendi Bevins Published on the American Evaluation Association AEA365blog series https://aea365.org/blog/ex-post-eval-week-tips-for-conducting-an-ex-post-evaluation-by-wendi-bevins/ January 19,...
Reblog: ITAD/CRS “Lessons from an ex-post evaluation – and why we should do more of them”
Reblog: ITAD/CRS "Lessons from an ex-post evaluation – and why we should do more of them" Reposted from: https://www.itad.com/article/lessons-from-an-ex-post-evaluation-and-why-we-should-do-more-of-them/ Even as evaluation specialists, rarely do we get the chance to...
Assuming Sustainability and Impact is Dangerous to Development (+ OECD/DAC evaluation criteria)
Assuming Sustainability and Impact is Dangerous to Development (+ OECD/ DAC evaluation criteria) We all do it; well, I used to do it too. I used to assume that if I helped my field staff and partners target and design funded projects well enough, and try...
Investing in Youth for Project Effectiveness and Sustainability
Investing in Youth for Project Effectiveness and Sustainability One out of every six people on earth is between the ages of 15-24, says the UN. That is 1.2 billion youth. As one young leader says, “if the world’s problems are to be solved, it’s not going to happen...
Building the Evidence Base for Post Project Evaluation: A report to the Faster Forward Fund
Building the Evidence Base for Post Project Evaluation: A report to the Faster Forward Fund We are delighted to share Valuing Voices' report on the value added of post-project evaluation, which compares findings from eight end-of-project and subsequent...
How ‘new’ are our projects… and who is aiming at the right outcomes?
How 'new' are our projects... and who is aiming at the right outcomes? Valuing Voices exciting news is we have received research grant funding from the esteemed evaluator, Michael Scriven's Faster Forward Fund. We're looking into the value-added of...
European, Multi-laterals & Foundations
Evaluating global aid projects ex-post closure: Evaluability… and how-to via Sustainability Evaluation training materials
Evaluating global aid projects ex-post closure: Evaluability, and how-to via Sustainability Evaluation training materials: How do we evaluate projects ex-post, and are all projects evaluable after donors leave? How do we learn from project documents to ascertain...
The Contentious Power of Evaluations, Guest Blog by Hanneke de Bode
THE CONTENTIOUS POWER OF EVALUATIONS or why sustainable results are so hard to come by…. A while ago, I reacted to a discussion among development aid/cooperation evaluators about why there are so few NGO evaluations available. It transpired that many people do...
Aid providers: More puzzle pieces, including unexpected outcomes; ours is not the whole picture
Aid providers: More puzzle pieces, including unexpected outcomes; ours is not the whole picture When we did our first ex-post evaluation/ delayed final evaluation in 2006 in Niger for Lutheran World Relief (LWR) funded by the Bill & Melinda Gates Foundation (pg75...
Valuing Voices Better Ex-Post Evaluation White Paper: Nordics & the Netherlands
Donors often talk about #SustainableDevelopment, and we know they try. But do they return #expost to #evaluate it? Here are lessons from #Nordic (#Norway, #Finland, #Sweden & #Denmark) and the #Netherlands in our #ValuingVoices #WhitePapers. Let us know what you...
“What IS Sustainability?” It depends on whom you ask: OECD, the UN, or Harvard Business School
“What IS Sustainability?” It depends on whom you ask: OECD, the UN, or Harvard Business School Recently I’ve had conversations where I had to define which sustainability we were talking about. Was it: ex-post-project sustainability of outcomes and impacts,...
Upcoming Jan Webinar: Lessons from Nordic / the Netherlands’ ex-post project evaluations: 14 Jan 2021
Upcoming Webinar: Lessons from Nordic / the Netherlands' ex-post project evaluations: 14 Jan 2021 In June-August 2020, Preston Stewart, our Valuing Voices intern, conducted through government databases of the four Nordic countries – Norway, Finland, Sweden, and...
Sustainable Exit Strategies: USAID vs. EU
Sustainable Exit Strategies: USAID vs. EU Once malnutrition has decreased, students’ attendance has risen, or the number of small businesses has doubled, program implementers may be quick to pack their bags and leave. But the impact of their work may be...
Sustaining “Sustainable Development”
Sustaining "Sustainable Development"? As a global development industry, we have almost no evidence of how (un)sustained the outcomes or impacts of 99% of our projects because we have never returned to evaluate them. But from early indications based on...
Assuming Sustainability and Impact is Dangerous to Development (+ OECD/DAC evaluation criteria)
Assuming Sustainability and Impact is Dangerous to Development (+ OECD/ DAC evaluation criteria) We all do it; well, I used to do it too. I used to assume that if I helped my field staff and partners target and design funded projects well enough, and try...
Setting a higher bar: Sustained Impacts are about All of us
Setting a higher bar: Sustained Impacts are about All of us Global development aid has a problem which may already affect impact investing as well. It is that we think it's really all about us (individuals, wealthy donors and INGO implementers) not all of us (you, me,...