by Jindra Cekan | Jul 7, 2025 | Adaptive Sustainability, Bangladesh, Climate Investment Funds (CIF), emerging outcomes, foreign aid, ICF, Mozambique, Resilience, Samoa, Sustainability, Sustainable development, Valuing Voices
Adaptive Sustainability lessons the Climate Investment Funds’ PPCR evaluation, including three ex-posts
We have evaluated less than 0.002% of how well and long results last (at least USAID data before the DEC disappeared). Hence we need to do more. I am excited to share our latest Climate Investment Funds (CIF) Adaptive Sustainability report including three ex-post evaluations conducted by our Valuing Voices team for ICF in Samoa, Bangladesh, and a former colleague in Mozambique (Appendix E). They are part of the PPCR project’s evaluation of this “$1 billion Pilot Program for Climate Resilience (PPCR) [that] champions pioneering resilience strategies and approaches, in the countries and regions most vulnerable to the destructive impacts of climate change”. We evaluated sustainability, climate resilience, and transformation, building on ex-post evaluation work done at the Adaptation Fund with Margaret Spearman. I reviewed nearly 40 other country reports to identify the drivers of sustainability and did intensive ex-post evaluations 3-5 years after closure in this Final report (April 2025) just published. Sincere thanks to the excellent ICF team (Jessica Kyle, Isabel Morford-Cheibub with Laura Kuhl) and the CIF team for this learning opportunity.
More broadly, here are sustainability and resilience-relevant lessons from the final report summary findings:
“The PPCR’s stated goal is to integrate climate resilience into countries’ core development planning and provide incentives for scaled-up action and transformational change, and was also meant to pilot and demonstrate ways to build resilience that could guide future adaptation initiatives. The evaluation finds that the PPCR advanced climate resilience through pioneering programs, planning, and investments, and modernized climate information services, successfully demonstrating new, innovative, and effective ways to embed climate resilience into national development. However, while the PPCR contributed to transformational change, it sometimes missed opportunities to deepen systemic change, enhance private sector engagement, increase scale, and strengthen long-term sustainability….”
More granularly, for the three ex-post case studies, we aimed to answer three Adaptive Sustainability evaluation questions:
1. Have the project outcomes/impact(s) been sustained since project completion, including what emerged?
2. How are the sustained project outcome(s) climate-resilient?
3. What signals of transformational change were noted by interviewees?
Adaptive Sustainability
The case studies evaluated outcome(s) ex-post, including the levels of specific outcomes and impact(s) at ex-post compared to the final project evaluation. They identified which drivers of sustainability (e.g., ownership, resources, partnerships, capacities) contributed to existing and emerging outcomes (what emerged from local efforts to sustain results differently ex-post). Findings are organized based on what was positive and what was negative, as well as what unexpected results or instances of maladaptation were observed ex-post. The teams looked for Emerging vs Advanced Transformation Signals across: a) Flexibility and dynamism, b) Capacity, ownership, and resources, and c) Vulnerability reduction and redundancy.
Some of the high-level lessons across all the PPCR projects regarding adaptive sustainability were:
1) High local ownership has contributed to most PPCR-supported infrastructure and natural assets still being operational, and agricultural practices continuing
2) Project outcomes have been sustained so far, though this is unlikely to last many more years.
3) Some unintended distributional justice issues arose after project completion, with vulnerable groups not experiencing the same lasting benefits as other stakeholders
4) There is limited evidence of follow-up or ongoing monitoring after project completion, representing a lost learning opportunity.
All three ex-post detailed cases’ adaptive sustainability results were mixed. Table 2 below shows Ownership, followed by Capacity, and Partnerships being moderate, and Resources to sustain results being weaker drivers of adaptive sustainability. Therefore planning for ongoing resources to sustain operations and maintenance of assets is a clear lesson.
In the fieldwork, some elements continued to be sustained, mainly due to local ownership and other resilience sources (see the Samoan pool below). “There are multiple backup systems in place. Village governance structures, along with rehabilitated pools, provide redundancy and alternative water sources in case of government water supply disruptions due to climate events.”
Several assets faltered due to a lack of sufficient ongoing operations and maintenance while other assets disappeared or held on. One Bangladesh market stall sank months after construction, in part due to climate shocks and in part to insufficient design for resilience, while other assets, such as roads, withstood years of cyclones thanks to local maintenance:
In Mozambique, more farming irrigation assets were sustained than failed in the face of management design and climate stresses, but ownership was the driver to ensure continuation. At one site, a lack of resources, capacities, and partnerships limited success; however, the local farmers’ determination found additional resilience resources to compensate for irrigation issues, utilizing a natural spring and reverting to manual watering.
Across the cases, local ownership is the primary driver; however, resources are still needed. The CIF found the lessons to be very illuminating and is incorporating them into future programming. Let us know how you can use these lessons as well…
by Jindra Cekan | Sep 24, 2021 | Aid effectiveness, Bill and Melinda Gates Foundation, Donors, emerging outcomes, Evaluation, ex-post evaluation, Impact, Lutheran World Relief, Nepal, Niger, Sustainability, Sustainable development
Aid providers: More puzzle pieces, including unexpected outcomes; ours is not the whole picture
When we did our first ex-post evaluation/ delayed final evaluation in 2006 in Niger for Lutheran World Relief (LWR) funded by the Bill & Melinda Gates Foundation (pg75 on), we found all sorts of unexpected/ unintended outcomes and impacts that far outweighed the original aid’s expectations. The project measured success by the livelihoods rebuilding post-drought ravaged sheep herds and water points for them. Instead, while the LWR aid was beautifully based on ‘habbanaye,’ a pastoralist practice of lending or giving small-stock offspring to poorer family members (and was expanded to passing on animals to poorer community members), and results showed a majority of the poor benefitted, our respondents showed it was far more nuanced. We aid providers (and our expectations) are only a part of any ‘impact’, which needs to be defined by the communities themselves:
- While many families benefitted from the sheep, enabling some young boys to shepherd several at a time, it turns out the poorest chosen by the communities were not necessarily ones who lost small livestock during the drought but were, in fact, the ultra-poor who had never had them. Therefore, proof of successful ‘restocking’ post-drought left these poor who were helped the most, out, as they were… unexpected;
- Holding onto the donated sheep was not as important an indicator for some: one woman told us that selling her aid-received sheep to buy her daughter dowry to marry a wealthier husband was a far better investment for their financial future than the sheep would have been. Our main measure of success was not nuanced enough;
- The provision of water through well-rehabilitation and building in the five villages was a vital resource. Women reported they saved 8 hours every two days by having potable water in their villages. Before then, they spent three hours walking each way to the far-off well and waited two hours to fetch 50l water, which they head-carried back. With the well in place, they generated household income through weaving mats, cooking food for sale, etc., which amounted to as much as 20% of increased household income – a boon! Also, having both time and water access enabled them to bathe themselves and their children, make their husbands lunch, and make their mothers-in-law tea, which led to far more household harmony. No ‘impact’ measures outside of livestock and water were included or could be added;
- Finally, the resulting show-stopper: in the last village where we interviewed women participants, they said that the groups of fellow recipients were a boon for community solidarity across ethnic groups. In their meetings, thanks to the sheep, water, and collective moral support, they said the conversations turned from conflict to collaboration, and best of all, women reported, “our husbands don’t beat us anymore.” Peace among ethnic groups, within and between households was completely unexpected.
Such a highly unexpected outcome would fall under #2 and #4 of the Netherlands study below. Unfortunately, the Foundation seemed less interested in these unexpected but stellar results. Yet at the same time I have empathy for their position, as so many of us in global development want to help solve problems, and demand proof we have.…. so we can leave and help others, equally deserving. Taking complexity into account, seeing lives in a wider context where our aid can be helping differently or even harming makes garnering more aid hard.
As the brilliant Time to Listen series by CDA showed, aid intervenes in people’s lives in complex ways and we need to listen to our participants and partners who always share more complex views than our reports can honor..A few of hundreds of quotes of 6000 interviews, this from the Solomon Islands:
Some appreciate the aid as it is given:
“People in my village are very grateful for the road because now with trucks coming into our village, the women can now take their vegetables to the market. Before, the tomatoes just rotted in the gardens. Tomatoes go bad quickly and despite our attempts in the past to take them to the market to sell, we always lost.” Woman from East Malaita
But for others there are great caveats:
“Donors should send their officers to Solomon Islands to implement activities in urban and rural areas. This will help them understand the difficulties we often face with people, environment, culture, geography, etc. ‘no expectem evri ting bae stret’ [Don’t expect everything to go right].” Man, Honiara
“They have their own charters, sometimes we might want to go another way but they don’t want to touch that. So sometimes there is some conflict there; some projects are not really what we would like to address – because the donors only want to do one component, and not another, because it is sensitive, or because they want quick results and to get out.” Government official, Honiara
“What changes have I noticed since independence? Whatever development you see here is due to individual struggles. No single aid program is sustainable. NGOs are created by donors and are comfortable with who they know. NGOs eat up the bulk of help intended for the communities. NGOs become international employers. They do their own thing in our province. Most projects have no impact. I want to say stop all aid except for education and health. If international assistance concentrates on quality education and health, the educated and healthy people will take care of themselves.” Government official, Auki, Malaita
“The most important impacts of aid people do not think about – they are not listed, not planned, they are remote, but these are the longest lasting. Often they are the opposite of the stated objectives. So remote, unintended, unexpected impacts are very often more important and more lasting and more dramatic than the short term intended, measured ones.” Aid consultant, Honiara
How widespread is our myopic focus on our intended results? A recent Netherlands Foreign Aid IOB study found unintended effects were an evaluator’s blindspot as across 664 evaluations over 20 years, “The ‘text miners’ found that only 1 in 6 IOB evaluation documents pay attention to unintended effects.” This dearth of attention to all the other things happening in projects led to 10 micro, macro, meso, and multiple level effects, from negative price effects such as food aid on local food producers or nationalist backlash to Afghan projects to positive effects (they found 40% of projects had this) such as a harbor built happening to expand beach tourism as well.
But if we don’t look for such effects, we don’t know the true impact of our aid programming. We also don’t honor the breadth of people’s lives rather than just as narrow ‘aid beneficiaries’ (ugh), not even honoring them with the terms’ participants’ much less ‘partners’ in their own development).
In our ex-post work, we find a wide array of ways ownership and implementation of activities is done after donors leave and without additional or with different resources, capacities, and partnerships. Taking emerging outcomes and impacts examples from a different Niger project, and one from Nepal:
1. Partnerships & Ownership: Half of the members of the all-women Village Banks reported helping one another deal with domestic disputes and violence. (Pact/Nepal)
2. Capacities: Trained local women charged rates to sell course materials onward (PACT/Nepal)
3. Ownership: Participants valued clinic-based birthing and sustained it by introducing locally-created social punishments and incentives (CRS/Niger)
4. Resources: New Ministry funding reallocated to sustain [health] investments, and private traders generated large crop purchases and contracts (CRS/Niger)
The assets and capacities we bring to help people and their country systems help only a sliver of their lives, and often in unexpected ways that sadly we aid donors and implementers don’t seem interested in. There are other puzzle pieces to add…
Let us not forget, as a Sustainable Development Goals Evaluation colleague said in 2017 on a call:
I am sure you, my dear colleagues, have reams of similar findings from your fieldwork. Please share yours!
