“What IS Sustainability?” It depends on whom you ask: OECD, the UN, or Harvard Business School
Recently I’ve had conversations where I had to define which sustainability we were talking about. Was it:
ex-post-project sustainability of outcomes and impacts,
environmental sustainability, or
business sustainability?
Since I spend most of my time evaluating the ex-post sustained and emerging impacts of foreign aid projects years after projects close, or at least advocate for it, let’s start there.
The Organisation for Economic Co-operation and Development (OECD) is a “forum and knowledge hub for data and analysis, exchange of experiences, best-practice sharing, and advice on public policies and international standard-setting.” Regarding evaluation specifically, the OECD has “established common definitions for six evaluation criteria – relevance, coherence, effectiveness, efficiency, impact, and sustainability – to support consistent, high-quality evaluation”. Focusing on long-term sustainability, their evaluation guidance is:
The good news is that in this recent publication on Applying Evaluation Criteria Thoughtfully (2021), OECD keeps the updated definition but inches towards recommending actual ex-post project sustainability evaluation, rather than just projected (and assumed “likely to continue” sustainability). For this, “likely” is the most significant reason evaluators for donors and implementers have assumed, rather than evaluated, sustainability for decades. Further, positive, ‘sustained’ trajectories are also assumed at close-out/ exit, but rarely tested ex-post.
The OECD criteria give not evaluating it as an option. I far prefer “net benefits of the intervention continue” as it is a marching order: Prove that results were sustained. In this evolution, this 2021 report states, “After the completion of the intervention, and evaluation of sustainability would look at whether or not the benefits did continue, this time drawing on data and evidence from the intervention’s actual achieved benefits.”
OECD even goes on to recommend implementing and monitoring for sustainability. The new piece de resistance is: “Sustainability should be considered at each point of the results chain and the project cycle of an intervention”:
“The sustainability of inputs (financial or otherwise) after the end of the intervention and the sustainability of impacts in the broader context of the intervention…. as well as whether there was willingness and capacity to sustain financing (resources) at the end of the intervention
For example, an evaluation could assess whether an intervention considered partner capacities
Built ownership at the beginning of the implementation period…. And
In general, evaluators can examine the conditions for sustainability that were or were not created in the design of the intervention and by the intervention activities and whether there was adaptation where required.”
Moreover, while the 2019 OECD report mentioned resilience in passing, related to sustainability, “encourages analysis of potential trade-offs, and of the resilience of capacities/ systemsunderlying the continuation of benefits”. Such resilience and continuation of benefits evaluation involve examining huge systems (the financial, economic, social, environmental, and institutional capacities) that projects and programs are implemented within, whose stability is needed to sustain net benefits over time. Yes, for ex-post sustainability questions for evaluators to consider should include: “To what extent did the intervention contribute to strengthening the resilience of particularly disadvantaged or vulnerable groups” on which the sustained impacts of so much of our “Leave No One Behind” myth of Sustainable Development rely.
However, OECD makes suggestions to evaluate even broader, overwhelming what is feasible: “…this involves analyses of resilience, risks, and potential trade-offs.” Whose? All stakeholders, from participants to local partners and national and international implementers, and international donors? How far back and how far forward? What a huge undertaking. Further, the OECD points evaluators to define resilience, but as I learned in my Famine Early Warning System research and a current ex-post evaluation process for the Adaptation Fund, that involves creating evaluable boundaries by determining resilient to what kinds of shocks?Vital questions current industry monitoring and evaluation budgets for all evaluations, much less (too-rare) ex-post project evaluations, are insufficient for as they hover around 3-5% of total costs.
Slight progress at OECD is being made by acknowledging environmental sustainability first brought up by the Brundtland Report, “Our Common Future” back in 1987. This linchpin report highlighted that “critical global environmental problems were primarily the result of the enormous poverty of the South and the non-sustainable patterns of consumption and production in the North. It called for a strategy that united development and the environment – described by the now-common term “sustainable development”… that meets the needs of the present without compromising the ability of future generations to meet their own needs.”
While an OECD brief in 2008 considers the environmental aspects of our thinking about sustainability, it argues that sustainability primarily about “using economic development to foster a fairer society while respecting ecosystems and natural resources.” The 2021 Applying Evaluation Criteria Thoughtfully rather unhelpfully mostly ignores the environment’s role in sustainability: “Confusion can arise between sustainability in the sense of the continuation of results, and environmental sustainability or the use of resources for future generations. While environmental sustainability is a concern (and maybe examined under several criteria, including relevance, coherence, impact, and sustainability), the primary meaning of the criteria is not about environmental sustainability as such; when describing sustainability, evaluators should be clear on how they are interpreting the criterion.” Given rapid climate change, I would argue that any sustained and emerging outcomes and impacts of projects that does not include an evaluation of the environmental context will fail to foster sustained resilience.Yet donors’ fixed funding timeframes that set completion to disbursement without evaluating sustainability or resilience continue to be huge barriers.
In 2022 a new resource on Sustainability from the media perspective questions the OECD. Sustainability: Going Beyond the Buzzword groups sustainability described in various media as: 1) Economic viability, 2) Social Sustainability, 3) Institutional Sustainability and 4) technical sustainability. But we’ve learned in 2022 from consulting work that the technology piece mentioned in this Buzzwords study regarding ‘baking in’ sustainability via the infrastructure/ assets created (The Adaptation Fund’s ex-post and resilience evaluations) and incorporating INGOs’ considerations of buying-local, seeing what local spare parts and technicians will be available in the future is a great added consideration. So too is promoting locally-led development through the very participatory way projects are implemented, with locals leading. Among the best resources is CDA/ Peace Direct/ Search for Common Ground’s Stopping as Success studies for USAID.
Finally, business sustainability brings together these impacts on communities and society along with impacts on the environment. These are called ESG (Environmental, Social, and Governance) criteria. A Harvard Business School brief defines sustainability as “doing business without negatively impacting the environment, community, or society as a whole. “Where applied well, the aspiration is that “beyond helping curb global challenges, sustainability can drive business success.” While Harvard Business Review highlights “What Works’ in Calculating the Value of Impact Investing, they are, like almost all of global development ‘while- we-are-there’-measures. There is one mention of ‘terminal value’, 5 years after close of ownership, and they estimate social return on investments. This is a good, step, but as insufficient as foreign aid – for these are projected, not actual results.
At Valuing Voices, we have found hopeful examples such as IKEA as well as where ‘impact investing’ hype does not match the claims. Nonetheless, increasingly businesses are trying to consider circular economy systemic principles of “economic development designed to benefit businesses, society, and the environment.” This is regenerative, aims to decouple growth from the consumption of finite resources, not generate excess waste that cannot be reused and actuals seem to be measured at least during investments. As Harvard notes, “this leads investors to look at factors such as a company’s carbon footprint, water usage (both Environment), community development efforts (Social), and board diversity (Governance).” We encourage them to measure long-term/ longitudinally. A current Harvard Business Review sobering article on the ineffectiveness so far of measuring environmental sustainability and ESG. “…reporting is not a proxy for progress. Measurement is often nonstandard, incomplete, imprecise, and misleading. And headlines touting new milestones in disclosure and socially responsible investment are often just fanciful ‘greenwishing’”.
Australia’s RMIT defines business sustainability as comprising 4 pillars: Human, Social, Economic, and Environmental which combines a) “Human sustainability focuses on the importance of anyone directly or indirectly involved in the making of products, or provision of services or broader stakeholders;… b) Social sustainability focuses on maintaining and improving social quality with concepts such as cohesion, reciprocity and honesty and the importance of relationships amongst people;… c) Economic sustainability aims to improve the standard of living [and] the efficient use of assets to maintain company profitability over time;… d) Environmental sustainability places emphasis on how business can achieve positive economic outcomes without doing any harm, in the short- or long-term, to the environment.” But how well measured?!
Would ESG success be sustained over the long-term rather than short-term shareholder profit cycles? Will the OECD start to recommend extensive ex-post evaluation? Will they develop guidance to incorporate environmental concerns in evaluation for our common good? I do not yet know, but I implore these silos to start talking. No time to waste!
As my colleague and collaborator Susan Legro commented, we need to:
1) Continue to seek clarity and specificity in the terminology that we use, ensuring that it is clear to all stakeholders and beneficiaries; and
2) Find ways to study projects and initiatives over the longer term, which is the only way to study the designation of “sustainable” for any initiatives seeking that label.
3) I’ll add spread the word, as I just found out Harvard’s Extension School course on Sustainability features this blog in its curriculum 🙂
Did we make enough time and measurement to foster sustainability, as we phased-down and phased-over an array of activities, alongside those remaining (white paper forthcoming)? Did we abruptly phase-out leaving partners and participants at a loss? Sharing power over all these decisions will influence what lasts.
There is an amazing breadth of local, ongoing resources, skills & capacities, linkages, motivation (thanks to Tufts FHI360’s work for USAID’s Food For Peace) that we can explore and learn from. There are local innovations and an array of unplanned collaborations (e.g., funding for health staff (Niger), training in small enterprise from the national government (Bangladesh), or private sector markets (Ethiopia) that can be accessed when partnerships are transparent and created one or more years pre-exit to collaborate on post-exit.
Ideally, we design and implement for exit from the onset. When we jointly set the timeframe and jointly assess risks to sustainability and adaptively manage exit, rather than exit based on pre-set timeframes, all sides win, with partners and participants able to foster sustainability. As USAID/ GIIN wrote about Responsible Exits for Impact Investors (2018), “the foundations for a responsible exit are laid even before an investment is made. To increase the likelihood of continued impact after exit, investors often select investees based on whether impact is embedded in their business model or inextricably linked to financial success. They also seek to understand the likely growth trajectory of the business, which has implications for which exit paths and options will become available.” They also note that a “growth strategy’ is needed throughout and at (investment) exit is “a company’s continued access to the right resources, networks, and knowledge” for sustained impact.
The need for a thoughtful approach to sustainability is shown by Hiller, Guthrie, and Jones in “Overcoming Ex-Post Development Stagnation“ (2016). The authors cite “limited evidence of program efficacies coupled with government and agency preference for planning, approval, and implementation processes rather than sustainment of outputs, outcomes, and impacts means that ex-post performance, scaling, and sustainability is not well understood or well pursued…. [There is a] lack of willingness to commit time and resources to rigorous evaluation of post-project effectiveness“. This affects a vast number of projects. For instance, they found 63,000 projects in 2003 alone, and “relative to the number of development projects undertaken, ex-post project [evaluations] are not commonly carried out, meaning that rates of success are often unknown and the complexity of causalities and ex-post dynamics of interactions and processes are not well understood.“ This limits our learning from what has (not) worked and what to do more (or less) of, including those that could not be sustained with only local resources.
We make sustainability assumptions are participants long for them, as Valuing Voices also found. Hiller et al. state that “whilst project documentation commonly conveys an expectation that some process of spread will occur ex-post, it rarely does, despite strong ex-post case-study evidence of stakeholder requests for further development opportunities.” This cautionary feedback could mean some project activities could be so resource-intensive that they could not feasibly be sustained or spread without long-term support, and retaining results may be limited to less costly activities. Valuing Voices found that other activities could be remunerative enough (financially, in health or education outcomes, for instance), as to be locally demanded and continued to be pursued. We have found in our Valuing Voices research and the Tufts research that activities where incentives did not continue, tended to die ex-post, while those which continued to bring benefits, such as cash crops and credit, water supply, and health, were prioritized ex-post, even in the absence of external funding continuing.
Hiller et al. outline that there are multiple ways in which scaling-up environmental sustainability over time, over area and, interestingly, scaling-within projects. The Ex-post Development Stagnation authors are clear that “creating conditions to support longer-term sustainability beyond project completion represents a recurring challenge, and it is not uncommon for activities and institutions to become inactive ex-post or for stakeholders to revert to previously unsustainable practices.” They note that some watershed studies have even found that participants actively destroy project measures in some cases. Certainly, the inability of locals to sustain often expensive activities without a project or larger organization’s support is common. “If it is assumed that development needs remain outstanding, then there may be merit in ensuring that development projects do not just remain “isolated, one-time interventions, like unconnected dots on a white page” or “islands of salvation.” The authors concur, “based on project subsidiarity and participatory principles, scaling-within management should be devolved to the local level (local authorities and local communities) to allow communities and individuals to filter out irrelevant practices and encourage adaptation and evolution of activities which are of greatest perceived livelihood benefit”.
As Valuing Voices research on exit has shown, it is a process, not an event(forthcoming, with thanks to I. Davies). The Hiller article notes that sustainability is best enhanced by capacity-building during implementation and with time for handover where “organizations adopt modes of functioning that allow local communities and organizations to build conceptual, operational, and institutional capacities. While scaling-down does not mean that governments disengage from processes such as community-driven development — it does, however, require it to be more flexible and responsive to locally generated demand to ensure the terrain is fertile for community organizations to emerge, learn, and grow.“
Let’s work together to extend the sustainability of impacts. Would love to hear your thoughts…
Six years. That’s how long ago I began researching proof of sustained impact(s) through its ex-post project evaluation. Until now Valuing Voices has focused on aid donors. We are expanding to the private sector.
In my PhD I was sure it was a lack of researched and shared proof of successful prevention of famine that led to inaction. In Valuing Voices’ research on ex-post project evaluation, I again felt “if only they knew, they would act”. I pulled together a variety of researchers and consultants who (often pro-bono, or for limited fees) researched the shockingly rare field evaluations of what was sustained after projects closed, why, and what participants and partners did themselves to sustain impacts.
Sustaining the outcomes and achieving impacts, are, after all, what global development projects promise. These ‘sustainable development’ results are at the top (or far-right, below) of our ‘logical frameworks’. We promise the country-level partners, our taxpayers and donors, that we will achieve them, yet…
We have applied to many grants for support, unsuccessfully, and have applied to evaluate a handful more ex-post sustainability evaluations which other consultants have won – while we were disappointed, in equal measure we are happy others are learning to do this, as we share our resources freely to promote exactly such practices across hundreds of thousands unevaluated projects! We are currently doing an ex-post project evaluation of an agriculture value chain in Tanzania, yet there are a handful done per year. At one conference, our discussant Michael Bamberger joked we were lucky not to be found dead under a bridge for taking on such a dangerous topic. We remain undeterred, and delight in colleagues we promote such work and thanking us for ours.
At the same time, several things have become apparent:
Vital lessons for how aid can do better remain unexplored, and true accountability to our country-national participants and partners ends when fixed-time, fixed deliverable project resources are spent and proof of accountability for money and results that donors want are filed away. Sadly, while capacity building is done throughout implementation, knowledge management about results is abysmal as ‘our projects’ data almost always dies quietly in donor and implementer computer hard drives after close-out, rather than being accessible in-country for further learning. Go partner!
We hardly ever return after all our evaluations to share with communities which speaks to ‘partnerships’ not being with the participants, and we often ‘exit’ without giving ample time to handover so that things can be sustained, e.g. local partners found, local and other international funding harnessed, etc. Learn together!
There is a real need to fund systematized methods for such evaluations, mandate access to quality baseline, midterm and final evaluations, and mandate that all projects above a certain funding level (e.g. $1mil) include funding for such evaluation and learning 2-10 years later. Many so-called ex-post evaluations are in fact either delayed final evaluations, desk studies without any fieldwork, rather methodologically flawed comparisons or with fieldwork which doesn’t talk to the intended ‘beneficiaries’ for such pivotal ground-level feedback. Innovate by listening!
It is unclear to us how any organization that has done an ex-post sustainability evaluation has learned from it and changed their systems, although we have been told some are ‘looking for a successful project to evaluate’, and that after a failed one, they are discontinued. We know of some (I)NGOs who are putting ex-posts into their new strategies, and two INGOs who are researching exitsmore – good. Be brave!
Recently, we are delighted some new NGOs are dipping their feet into their first evaluations of sustainability, they do so bravely. The tension between accountability and learning is heightened at the prospect that implementers and donors have failed to create sustained impact. But why judge them when all the design and systems in place are to reward success while projects are running (and even those don’t always show much) so that they all get congratulations and more funding for very similar projects? Who knows who is focused on sustaining impacts with funding capacities, partnerships and country-led design, implementing with feedback loops and adjusting for the long-term, helping communities evaluate us rather than how well they are fulfilling our targets, etc. Sustaining impacts will win you funding!
Logically, here are many indications among ex-post sustainability evaluations that profitable, but low-risk and diversified agriculture, microenterprise/ business projects are better sustained (Niger, Ethiopia, Tanzania, Nepal, etc.). This does not mean that all projects need to be profitable, but cost-covering projects even in the health and education/ vocational training/ sectors is important as many of us know. Self-funding!
So rather than giving up on sustained impacts, we are adding another branch to the Valuing Voices tree.
My partners and I have extensively researched the need for and co-founded Impact Guild. We will work alongside NGOs and impact investors to foster:
FUNDING: The money available from development aid donors is shrinking in volume + value, while development financing is scaling up exponentially.
The SDGs and the Paris Agreement are prompting a massive scale-up of development financing from billions to trillions of dollars into ‘sustainable development’, yet with rare Scandinavian and Foundation exceptions, donors appear to be switching from longer-term development to humanitarian aid. Further, despite decades of experience, international and national nonprofit development implementers are mostly absent in the conversation around scaling-up the flow of capital to achieve and sustain development goals. Exceptions are some in the International NGO Impact Investing Network (AMPLIFY)
2. RESULTS:Funding for projects that can show great results (e.g. Social Impact Bonds/ Development Impact Bonds, which are in fact pay-for-performance instruments), even sustained impacts from partnering with local small and medium enterprises, national level ministries, and local NGOs. Far too long, implementers have been able to get funding for projects with mediocre results; impact investors are raising the bar and even donors are helping hedge risk. This includes M&E ‘impact’ value that rigorously tracks results (savvy private-sector donors require counterfactual/ control group data, isolating results from that intervention).
3. LEARNING: Impact Investors have a lot to learn from non-profits and aid donors as well.
They talk about impact but too often that is synonymous with generic results, while International and National nonprofits (NGOs) have detailed, grassroots systems in place;
Most seem to be content – for now – to invest in the 17 Sustainable Development Goal areas (e.g. vetting investable projects by screening criteria of not only getting a financial return, but also by broad sectoral investments, e.g. poverty, hunger, climate etc.). Many claim they have affected change, without data to prove it. The SDGs are slowly creating indicators to address this, and investors also need to be brought along to differentiate between corporate efficiency activities for their operations and those that affect change at the output, outcome and impact levels in communities;
There are still large leaps of logic and claims among investors and some know that data is lacking to claim good grassroots targeting and actual results that prove they are changing hunger, poverty and other sectors in Africa, Asia, Latin Americ. Good development professionals would see that the very design would make results accessible only to the elite of that country (e.g. $1 nutrition bars are inaccessible to most of a country’s population living on income of $2.00 a day)
We will bring with us all we know about great potential sustained impacts programming, such as Theory of Sustainability, looking for emerging results alongside planned early on, learning from failure for success, partnering successfully for country-led development, etc.
So keep watching these ‘spaces’: www.ValuingVoices.com and www.ImpactGuild.orgfor updates on bridging these worlds, hopefully for ever-greater sustained impacts. Let us know if you would like to partner!
Setting a higher bar: Sustained Impacts are about All of us
Global development aid has a problem which may already affect impact investing as well.
It is that we think it’s really all about us (individuals, wealthy donors and INGO implementers) not all of us (you, me, and project participants, their partners and governments). It’s also about us for a short time.
All too often, the measurable results we in global development aid and Corporate Social Responsibility (CSR) funded projects that last 1-5 years track and report data for two reasons:
1) Donors have Compliance for grantees to meet (money spent, not lost, and results met by fixed deadlines of 1-5 years – look at some of the European Commission Contracting rules) and
2) Fund recipients and the participants they serve are accountable to ‘our’ donors and implementers who take what happened through their philanthropic grants as ‘their’ results.
Both can skew how sustainably we get to create impacts. An example of such strictures on sustainability from USAID. As respected CGDev Elliot and Dunning researchers found in 2016 when assessing the ‘US Feed the Future Initiative: A New Approach to Food Security?‘ the $10.15 billion leveraged $20 billion from other funders for disbursement over three years (2013-16). “We are concerned that pressure to demonstrate results in the short termmay undermine efforts to ensure any impact is sustainable…. Unfortunately, the pressure to show immediate results can encourage pursuit of agricultural investments unlikely to be sustained. For example, a common response to low productivity is to subsidize or facilitate access to improved inputs… it can deliver a quick payoff… however, if the subsidies become too expensive and are eliminated or reduced, fertilizer use and yields often fall…..
With so much focus on reporting early and often about the progress in implementing the initiative, there is a risk that it increases the pressure to disburse quickly and in ways that may not produce sustainable results. For example, for 2014, Feed the Future reports that nearly 7 million farmers applied “improved technologies or management practices as a result of U.S. Government assistance,” but only 1,300 received “long-term agricultural sector productivity.” Are the millions of others that are using improved inputs or management practices because of subsidies likely to have these practices sustained? And how likely are they to continue using improved practices once the project ends?”
3) Impact investors stick to the same two paths-to-results and add a new objective: market-competitive financial returns. They also need to show short-term results to their investors, albeit with social, environmental and governance results like non-profits (future blog).
4) Altruists create things we want ‘beneficiaries’ (our participants) to have. For instance a plethora of apps for refugees cropped up in recent years, over 5,000 it is estimated, which can be appropriate, nor not so helpful. Much like #2 above, ‘we’re’ helping ‘them’ but again, it seems to be a ‘give a man a fish’… and my fish is cool sort of solution… but do our participants want/ need this?
How often is our work-for-change mostly about us/by us/ for us... when ideally it is mostly about ‘them’ (OK, given human self-interest, shouldn’t changes we want at least be about all of us?).
All too often we want to be the solution but really, our ‘grassroots’ clients who are our true customers need to generate their own solution. Best if we listen and we design for long-term sustainability together?
As the Brilliant Sidekick Manifesto stated in two of its ten steps: a) “I will step out of the spotlight: Sustainable solutions to poverty come from within are bottom-up, and flow from local leaders who are taking the risks of holding their politicians accountable and challenging the status quo.”
b) “I will read “To Hell with Good Intentions” again and again: Politicians, celebrities and billionaire philanthropists will tell me that I can be a hero. I cannot. The poor are not powerless or waiting to be saved. Illich will check my delusions of grandeur.”
We have examples of where we have stepped away and participants had to fend for themselves. At Valuing Voices, we’ve done post project-exit evaluations 2-15 years afterward. What did participants value so much that they sustained it themselves (all about them, literally)? These Sustained and Emerging Impacts Evaluations (SEIE)also give us indications of Sustained ROI (Sustained Return on Investment (SusROI) is a key missing metric. As respected evaluator Ricardo Wilson-Grau said in an email, “I think calculating cost-effectiveness of an intervention’s outcomes would be a wonderful challenge for a financial officer searching for new challenges — if not a Nobel prize in economics!”)
Most of these evaluations are pretty bad news mixed with some good news about what folks could sustain after we left, couldn’t and why not. (These are the ones folks expect to have great results, otherwise they wouldn’t share them!) While most clients are understandably interested in what of ‘theirs’ was still standing, and it was interesting disentangling where the results were attributable by implementation or design or partnership flaws or something else, what was mesmerizing was what came from ‘them’.
The key is looking beyond ‘unexpected’ results to look at emerging impacts that are about ‘them’ (aka what we didn’t expect that was a direct result of our project, e.g. spare parts were no longer available to fix the water well pump once we left or a drought rehabilitation water project that decreased violence against women), to what emerging results are attributable not to use but only to our participants and partners who took over after our projects closed.One example is a Nepalese project ended yet the credit groups of empowered women spawned groups of support groups for battered women. Another is a child maternal health project changed how it worked as women reverted to birthing at home after NGOs left; community leaders punished both parents with incarceration in the health clinic for a week if they didn’t given birth there (wow did that work to sustain behavior change of both parents!).
Many of us at Valuing Voices are shocked that funders don’t seem that interested in this, as this is where they not only take over (viz picture, sustaining the project themselves), but they are making it theirs, not ours. Imagine assuming the point of development is to BE SUSTAINABLE.
Source: Community Life Competence
Our participants and national stakeholder partners are our true clients, yet… Feedback Labs tell us Americans alone gave $358 billion to charities(equivalent to the 2014 GDP of 20 countries) – in 2014 but how much of this was determined by what ‘beneficiaries’ want? Josh Woodard, a development expert, suggests a vouchers approach where our true clients, our participants, who would “purchase services from those competing organizations… [such an] approach to development would enable us all to see what services people actually value and want. And when we asked ourselves what our clients want, we would really mean the individuals in the communities we are in the business of working with and serving. Otherwise we’d be out of business pretty quickly.”
This opens the door to client feedback – imagine if participants could use social media to rate the sustained impacts on them of the projects they benefited from? A customer support expert wrote in Forbes, “Today, every customer has, or feels she has, a vote in how companies do business and treat customers. This is part of a new set of expectations among customers today that will only grow ... you can’t control product ratings, product discussions or much else in the way of reviews, except by providing the best customer experience possible and by being proactive in responding to negative trends that come to the surface in your reviews and ratings stronger.”
So how well are we working with our participants for ‘development’ to be about them?
Impact Investing – International Development’s New Holy Grail?
There are so many things I love about the private sector such as Forbes 18 Dec Quote of the Day: “You’re going to be wrong a fair amount of times. So the issue is, how do you be wrong well?” asked Ray Dalio, Founder of Bridgewater Associate. This is a key issue for impact investors and international ‘developers’ alike.
International development suffers from the myth that failure must be downplayed. Too often only success is highlighted, whereas project shortcomings are framed as: “less successful” “numerous issues affected a less optimal…” Yet by downplaying the less great (Aka awful) results we miss vital learning that private sector expects, learns from and integrates toward the greater success. Why? Many in foreign aid believe (rightly?) such admissions might endanger winning more funding for more projects. Even as recently as 2014, U.S. foreign aid industry websites such as DevEx are still posting: “One can be forgiven for forming the impression that our development efforts are nearly perfect if typical annual reports, scientific conferences and event social media content are the basis for information. Successes are proudly packaged in glossy formats and heavily disseminated, whereas any objectives not achieved are relegated to the obligatory, and typically short, lessons learned section. This practice does not accurately represent an important reality: development efforts do in fact fail” [1].
Admitting failure, posting failure reports are awfully rare in international development, but how bad is it? The Asian Development Bank wrote in a large overview of the sustainability of post-project results, “Some early evidence suggests that as many as 40% of all new activities are not sustained beyond the first few years after disbursement of external funding” [2]. A 2017 Cambridge University study found that “using an original database of over 14,000 small development projects in Ghana, I estimate that one-third of projects that start are never completed, consuming nearly one-fifth of all local government investment” [3]. Even when they do start, complete, and even have salutary results at the end of the project, Valuing Voices research shows quick declines toward failures in as little as two years post exit, such as these post-project results at the AEA 2017 conference. The foreign aid industry is so focused on showing results while conditions are (relatively) conducive, that far fewer than 1% of all projects are evaluated for what was still standing in as little as two years after project closeout, and those are mostly those projects expected to be successful. Sustainable, long-term results suffer from what CGDev researchers are concerned “that pressure to demonstrate results in the short term may undermine efforts to ensure any impact is sustainable….Unfortunately, the pressure to show immediate results can encourage pursuit of agricultural investments unlikely to be sustained” [4]. Luckily there’s a place to go. DevEx reminds us that “Venture capitalists and corporate investors understand that less than 20 percent of new businesses will succeed,” hence my love of the private sector’s admitting, learning and improving that ‘aid’ needs [1].
As a former investment banker (Solomon Brothers) and management consultant (Price Waterhouse & Coopers and Lybrand), I know that the corporates care for results, and do not shy away from pulling money from where things don’t work and put it where they do. 30 years in international development showed me that rigid bureaucracies and fixed ‘project cycles’ and an industry focused on ‘getting money out the door’ lead to a focus on accounting for all funds, but not for changing lives over the long term. Virtually no one calculates return on (our) investment compared to the cost of projects, especially including the value of what projects generate and participants can sustain.
I am quite fervently hoping Impact Investors focused on financial ROI to firms and investors as well asSocial Return on Investment will step in, fund gathering and learning from the whole range of ‘returns’. Will they share both financial profits/ losses and feedback from the whole social ‘value chain’ of stakeholders of those involved on what succeeds and fails? Will investors learn from national partners and participants on what should be done better? If yes, all of us will win. I am heartened by cautiously optimistic statements such as Next Billion’s “a core characteristic and challenge of impact investing is the measurement and management of social and environmental impacts alongside financial returns. Development cooperation and impact investing communities can build on their respective experience in results measurement and learn from and with each other” [5]. We can IF we are going to the same place.
From my early look at impact investing, it is a ‘game changer’ with $250 billion in assets looking for a profitable home [6]. UBS Asset Manager Baldinger says “In the past you sold products to your client, now you empower your client to create a desired impact. As an industry, we’ve had to rethink everything we do — impact and sustainability is the Silicon Valley of finance and we want to be the Google” [6]. These are happy words to someone focused on sustained (and emerging) impacts but among impact investors, so far, ‘impact’ seems to be thrown about as specifically as ‘results’, and GIIN ‘sustainability’ metrics are so wide ranging as to illuminate less quality than quantity. So far, much of their metrics look more like outputs relevant to companies (‘clients served’, ‘new investment capital’) that results of SROI. While there is something to be said about measures of ‘organizations trained’, ‘poverty assessments’ done, at least as a start, yet does ‘gross profit’ indicate that corner of the world is better off (and does this measure the investment into the enterprise, or is this of the investment fund itself)? Does ‘communities served’ and ‘social impact objectives’ illuminate the quality of the impact on lives changed? Is anyone asking how long-lasting, and sustained these investments, measuring what I call SUStained Return on Investment (SUSROI), will be after these investors leave (which is what I suspect most investment participants and millennial investors think they’re buying)?
This is the start of a series of blogs exploring how we who care about generating and evaluating sustained impacts can learn from, inform, (gasp) shape impact investing’s gargantuan footprint in international development. Powerhouses such as the Rockefeller Foundation, Ford Foundation and Soros are looking, teaching, investing, and all public and private equity as well as a whole range of other investors now invest in this new hybrid [7][8]. Who else is? What can we learn to make the world better? What do you think: Is impact investing development’s holy grail?
Jindra Cekan, Ph.D. has used participatory methods for 30 years to connect with participants, ranging from villagers in Africa, Central/ Latin America and the Balkans to policy makers and Ministers around the world for her international clients. Their voices have informed the new Sustained and Emerging Impacts Evaluation, other M&E, stakeholder analysis, strategic planning, knowledge management and organizational learning.
If you don’t find what you are looking for via the search, categories, or posts above, you can go to the Blog page, scroll to the bottom, and click “previous posts” to go through all of the posts (newest–>oldest).
