by Jindra Cekan | Dec 31, 2024 | 3ie, Accountability, Adaptation Fund, blended finance, climate change, environment, ESG, European Evaluation Society (EES), European Union, Evaluation, ex post evaluation, ex-post evaluation, foreign aid, GIIN, IDEV, Impact, impact investing, international development, Localization, longtermism, Monitoring and Evaluation, OXFAM, Participants, post project evaluation, Private sector, Sustainability, Sustainable Finance
NEW YEAR, NEW SUSTAINABLE US?
Happy 2025!
(Image from: https://plumvillage.org/articles/can-the-new-year-really-be-new)
How will we be new this New Year? It may seem funny as an ex-post sustainability evaluator to write about the future rather than the past :). But ex-post learning is learning from past successes or failures for future improvements. Such learning is part of “longtermism” where “positively influencing the long-term future is a key moral priority of our time. It’s about taking the sheer scale of the future seriously, and how high the stakes might be in shaping it” (which for many of us means learning from the past to do better in the future). “It means thinking about the challenges we might face in our lifetimes that could impact civilisation’s whole trajectory, and taking action to benefit not just the present generation, but all generations to come…. Many social movements have fought for greater recognition for disempowered members of society. Advocates of longtermism want to extend this recognition to future people, too.”
Taking the longer view also means taking a global view and continuing to address the disparities that our economic system generates, and seems to be generating more destructively. As Oxfam’s excellent Inequality annual report tells us, “since 2020, the richest five men in the world have doubled their fortunes. During the same period, almost five billion people globally have become poorer.” At the same time, the UN cites a World Bank estimate that an additional 68 to 135 million people could be pushed into poverty by 2030 because of climate change”. Given the EU is 450 million and US is 330 million, imagine 20-30% of our populations falling into poverty due to changes we can make – but did not – in addressing inequality and a sustainable, liveable planet. I don’t know what your long-term vision for y/our world is, but I’m quite sure adding to global poverty is not on the list.
What can we do anew?
- Where life is stable: Focus on our collective communities and our planet and helping others: the NYTimes on making new year’s resolutions for collective happiness. I would add, we need to make choices from a planetary perspective – I got a plug-in electric hybrid Toyota CHR this year so I would pollute less and am a vegetarian (aspiring vegan) to decrease my tiny share of the 30% of emissions due to our food systems. Am reading the book The Logic of Sufficiency and preparing to read Less is More will help us consume less, as we need less. Remember, as we in the Global North consume less, the Global South is likely to suffer less, as they have caused so little of today’s climate crisis. Generation Climate Europe states: “In 2019, the top 10% of global emitters (771 million individuals) were responsible for about 48% of global CO2 emissions, while the bottom 50% (3.8 billion individuals) were responsible for almost 12% of all emissions.” This is a staggering statistic we all need to act to reverse.
- Take a longer view, doing ex-post evaluations, for sectors such as agriculture, even for disasters: 3ie admitted that only 4% of all impact evaluations include long-term learning about food systems interventions evaluating long-term outcomes (including climate change) and a collection of sustainability learning by IDEV’s Sustainability Matters asks us to take a holistic approach, covering the full spectrum of environmental, social and economic sustainability. Both sources tell us we need to do more evaluation and learning over the longer-term as climate shocks increase. Disaster Science also discusses the need for a longer-term view – this means putting out fires but thinking longer-term (see longtermism above :)) about how we can make our interventions more sustained. We can ‘bake-in’ or ‘hardwire” sustainability as we shared here in a VV blog about health projects. Also this includes evaluating climate adaptation ex-post as the Adaptation Fund has shared (and I spoke at the European Evaluation Society Conference) positive and negative lessons from our ex-post learning from past design/ implementation for future sustainability. Proud that the Fund has created the Adaptation Fund’s toolkit, and in 2025, look for more ex-post learning from the Climate Investment Funds and other clients!
EES Conference presentation – Rimini in 2024 (photo by the Vladislav Arnaoudov)
- Use our money to do more measurable good: pushing the private sector for more equitable investments given public funds to do good are so stretched: Dr G. Marcelle notes that we need transformative sustainable finance (which is environmental, social, and governance (ESG)) to address inequalities and address climate change harm being done. As an evaluator, we need to measure real impact as the Adaptation and Resilience Investors’ Cooperative does or the Global Impact Investor Network (GIIN) which has set up benchmarks in Agriculture, Forestry, Energy, and Finance. And we so need mandatory, transparent and comprehensive reporting which fosters trust, not greenwashing or wishful thinking. Also consider what banks you are funding, some profit from more fossil fuels.
- Appreciate and learn from what works and do more of it, patiently, especially at the community level! This BBC article celebrates 5 successes against biodiveristy loss. This Barbados conference “Sustainable Energy for Equity, Security and Prosperity” I will attend in 2025 (!) may also cover the exciting new debt for climate resilience deal which is the world’s first! IDEV reminds us conservation is most powreful at the community level, but we must learn from monitoring data, locally. IFAD reminds us sustained development takes time and resources and patience.
- What else would you add?
Happy Sustainable New Year, New Us!
by Jindra Cekan | Apr 9, 2021 | business, environment, ESG, ex-post evaluation, foreign aid, Impact, impact investing, OECD, Sustainable development
“What IS Sustainability?” It depends on whom you ask: OECD, the UN, or Harvard Business School
Recently I’ve had conversations where I had to define which sustainability we were talking about. Was it:
- ex-post-project sustainability of outcomes and impacts,
- environmental sustainability, or
- business sustainability?
Since I spend most of my time evaluating the ex-post sustained and emerging impacts of foreign aid projects years after projects close, or at least advocate for it, let’s start there.
- The Organisation for Economic Co-operation and Development (OECD) is a “forum and knowledge hub for data and analysis, exchange of experiences, best-practice sharing, and advice on public policies and international standard-setting.” Regarding evaluation specifically, the OECD has “established common definitions for six evaluation criteria – relevance, coherence, effectiveness, efficiency, impact, and sustainability – to support consistent, high-quality evaluation”. Focusing on long-term sustainability, their evaluation guidance is:
Source: OECD, Better Criteria for Better Evaluation, 2019
The good news is that in this recent publication on Applying Evaluation Criteria Thoughtfully (2021), OECD keeps the updated definition but inches towards recommending actual ex-post project sustainability evaluation, rather than just projected (and assumed “likely to continue” sustainability). For this, “likely” is the most significant reason evaluators for donors and implementers have assumed, rather than evaluated, sustainability for decades. Further, positive, ‘sustained’ trajectories are also assumed at close-out/ exit, but rarely tested ex-post.
The OECD criteria give not evaluating it as an option. I far prefer “net benefits of the intervention continue” as it is a marching order: Prove that results were sustained. In this evolution, this 2021 report states, “After the completion of the intervention, and evaluation of sustainability would look at whether or not the benefits did continue, this time drawing on data and evidence from the intervention’s actual achieved benefits.”
OECD even goes on to recommend implementing and monitoring for sustainability. The new piece de resistance is: “Sustainability should be considered at each point of the results chain and the project cycle of an intervention”:
- “The sustainability of inputs (financial or otherwise) after the end of the intervention and the sustainability of impacts in the broader context of the intervention…. as well as whether there was willingness and capacity to sustain financing (resources) at the end of the intervention
- For example, an evaluation could assess whether an intervention considered partner capacities
- Built ownership at the beginning of the implementation period…. And
- In general, evaluators can examine the conditions for sustainability that were or were not created in the design of the intervention and by the intervention activities and whether there was adaptation where required.”
Yes, Valuing Voices highlighted this at the American Evaluation Association presentation “Barking up a Better Tree: Lessons about SEIE Sustained and Emerging Impact Evaluation” in 2016, and we have developed this into 2020’s Exiting for Sustainability trainings and checklists. Wonderful to see implementing for sustainability in guidance by the OECD! Many of the elements his research cites overlap with our Exiting for Sustainability checklislts which includes ownership, resources, capacities, partnerships, and implementation/ M&E process.
Moreover, while the 2019 OECD report mentioned resilience in passing, related to sustainability, “encourages analysis of potential trade-offs, and of the resilience of capacities/ systems underlying the continuation of benefits”. Such resilience and continuation of benefits evaluation involve examining huge systems (the financial, economic, social, environmental, and institutional capacities) that projects and programs are implemented within, whose stability is needed to sustain net benefits over time. Yes, for ex-post sustainability questions for evaluators to consider should include: “To what extent did the intervention contribute to strengthening the resilience of particularly disadvantaged or vulnerable groups” on which the sustained impacts of so much of our “Leave No One Behind” myth of Sustainable Development rely.
However, OECD makes suggestions to evaluate even broader, overwhelming what is feasible: “…this involves analyses of resilience, risks, and potential trade-offs.” Whose? All stakeholders, from participants to local partners and national and international implementers, and international donors? How far back and how far forward? What a huge undertaking. Further, the OECD points evaluators to define resilience, but as I learned in my Famine Early Warning System research and a current ex-post evaluation process for the Adaptation Fund, that involves creating evaluable boundaries by determining resilient to what kinds of shocks? Vital questions current industry monitoring and evaluation budgets for all evaluations, much less (too-rare) ex-post project evaluations, are insufficient for as they hover around 3-5% of total costs.
- Slight progress at OECD is being made by acknowledging environmental sustainability first brought up by the Brundtland Report, “Our Common Future” back in 1987. This linchpin report highlighted that “critical global environmental problems were primarily the result of the enormous poverty of the South and the non-sustainable patterns of consumption and production in the North. It called for a strategy that united development and the environment – described by the now-common term “sustainable development”… that meets the needs of the present without compromising the ability of future generations to meet their own needs.”
While an OECD brief in 2008 considers the environmental aspects of our thinking about sustainability, it argues that sustainability primarily about “using economic development to foster a fairer society while respecting ecosystems and natural resources.” The 2021 Applying Evaluation Criteria Thoughtfully rather unhelpfully mostly ignores the environment’s role in sustainability: “Confusion can arise between sustainability in the sense of the continuation of results, and environmental sustainability or the use of resources for future generations. While environmental sustainability is a concern (and maybe examined under several criteria, including relevance, coherence, impact, and sustainability), the primary meaning of the criteria is not about environmental sustainability as such; when describing sustainability, evaluators should be clear on how they are interpreting the criterion.” Given rapid climate change, I would argue that any sustained and emerging outcomes and impacts of projects that does not include an evaluation of the environmental context will fail to foster sustained resilience. Yet donors’ fixed funding timeframes that set completion to disbursement without evaluating sustainability or resilience continue to be huge barriers.
In 2022 a new resource on Sustainability from the media perspective questions the OECD. Sustainability: Going Beyond the Buzzword groups sustainability described in various media as: 1) Economic viability, 2) Social Sustainability, 3) Institutional Sustainability and 4) technical sustainability. But we’ve learned in 2022 from consulting work that the technology piece mentioned in this Buzzwords study regarding ‘baking in’ sustainability via the infrastructure/ assets created (The Adaptation Fund’s ex-post and resilience evaluations) and incorporating INGOs’ considerations of buying-local, seeing what local spare parts and technicians will be available in the future is a great added consideration. So too is promoting locally-led development through the very participatory way projects are implemented, with locals leading. Among the best resources is CDA/ Peace Direct/ Search for Common Ground’s Stopping as Success studies for USAID.
- Finally, business sustainability brings together these impacts on communities and society along with impacts on the environment. These are called ESG (Environmental, Social, and Governance) criteria. A Harvard Business School brief defines sustainability as “doing business without negatively impacting the environment, community, or society as a whole. “Where applied well, the aspiration is that “beyond helping curb global challenges, sustainability can drive business success.” While Harvard Business Review highlights “What Works’ in Calculating the Value of Impact Investing, they are, like almost all of global development ‘while- we-are-there’-measures. There is one mention of ‘terminal value’, 5 years after close of ownership, and they estimate social return on investments. This is a good, step, but as insufficient as foreign aid – for these are projected, not actual results.
At Valuing Voices, we have found hopeful examples such as IKEA as well as where ‘impact investing’ hype does not match the claims. Nonetheless, increasingly businesses are trying to consider circular economy systemic principles of “economic development designed to benefit businesses, society, and the environment.” This is regenerative, aims to decouple growth from the consumption of finite resources, not generate excess waste that cannot be reused and actuals seem to be measured at least during investments. As Harvard notes, “this leads investors to look at factors such as a company’s carbon footprint, water usage (both Environment), community development efforts (Social), and board diversity (Governance).” We encourage them to measure long-term/ longitudinally. A current Harvard Business Review sobering article on the ineffectiveness so far of measuring environmental sustainability and ESG. “…reporting is not a proxy for progress. Measurement is often nonstandard, incomplete, imprecise, and misleading. And headlines touting new milestones in disclosure and socially responsible investment are often just fanciful ‘greenwishing’”.
Australia’s RMIT defines business sustainability as comprising 4 pillars: Human, Social, Economic, and Environmental which combines a) “Human sustainability focuses on the importance of anyone directly or indirectly involved in the making of products, or provision of services or broader stakeholders;… b) Social sustainability focuses on maintaining and improving social quality with concepts such as cohesion, reciprocity and honesty and the importance of relationships amongst people;… c) Economic sustainability aims to improve the standard of living [and] the efficient use of assets to maintain company profitability over time;… d) Environmental sustainability places emphasis on how business can achieve positive economic outcomes without doing any harm, in the short- or long-term, to the environment.” But how well measured?!
Would ESG success be sustained over the long-term rather than short-term shareholder profit cycles? Will the OECD start to recommend extensive ex-post evaluation? Will they develop guidance to incorporate environmental concerns in evaluation for our common good? I do not yet know, but I implore these silos to start talking. No time to waste!
As my colleague and collaborator Susan Legro commented, we need to:
1) Continue to seek clarity and specificity in the terminology that we use, ensuring that it is clear to all stakeholders and beneficiaries; and
2) Find ways to study projects and initiatives over the longer term, which is the only way to study the designation of “sustainable” for any initiatives seeking that label.
3) I’ll add spread the word, as I just found out Harvard’s Extension School course on Sustainability features this blog in its curriculum 🙂
What are your thoughts?
