Once malnutrition has decreased, students’ attendance has risen, or the number of small businesses has doubled, program implementers may be quick to pack their bags and leave. But the impact of their work may be undermined if not undone if they leave before implementing context-specific, comprehensive plans for post-project sustainability. It is for this reason that exit strategies – and particularly institution-wide mandates by donors – are crucial components of effective development programming. The United States Agency for International Development (USAID and the European Union (EU) Commission both have close-out protocols in place, but much to learn from one another in ensuring long-term progress in the wake of their close-outs.
Recommendations for Project Close-Out
Although every program should develop and follow context-specific exit measures, non-profits and ex-post evaluators have found that some exit strategies consistently lead to long-term, sustainable impact. The International NGO Training and Research Centre (INTRAC), for example, states that donors and implementers should actively plan for close-out throughout the project – from program design to the post-exit stage [1]. It recommends the following:
Implementers should “build exit thinking into the design of the project,” and during the program, representatives should be monitoring pre-determined exit indicators [1]
“Senior staff and management need to prioritize resources for exits in order to do them well, embed learning within organizations and across teams, and ensure mistakes are not repeated” [1]
“Practitioners should also consider going the extra mile to look after staff at headquarters and within partner organizations” [1]
“Document experiences and share them externally” [1]
According to USAID’s 2020 Program Cycle Operational Policy, the formal close-out process for specific projects are carried out by the USAID Mission team in the recipient country and Operating Units based in Washington, DC [2]. However, the Program Cycle does not reference the close-out process, let alone offer guidance [3]. Unlike country exits, which require thorough evaluation and international cooperation [4],ᴬ the sustainability measures of close-out procedures for activities – components of country-wide programs – seem to be delegated to implementing agencies.
Tellingly, the USAID formally defines “sustainability” as “the ability of a local system to produce desired outcomes” after its projects end, which the Agency may “contribute to” by strengthening capacity in their respective realms [5]. After close-out, USAID currently considers the sustainability of its projects to be a matter of “the country and/or targeted community’s commitment and capacity to achieve development solutions” [5]. Commitment and capacity, in turn, are gauged by 17 country-level “self-reliance metrics” ranging from “liberal democracy” to “export sophistication” [6,7].ᴮ Even post-project monitoring and evaluation are explicitly meant to “reveal whether implementation is on track and results are being achieved” with no mention of sustainability or long-term follow-up [3].
In effect, this framework not only puts the onus of project sustainability on recipients, but evaluates it based on large-scale and potentially tangential metrics. The stakes of this practice are raised by the fact that the resultant “Country Roadmaps” inform when USAID ends its programming in a given country, endangering a range of populations that these metrics do not account for, like seniors and prisoners [6].
USAID’s official Program Cycle [3]
During close-out, USAID representatives seem to focus on administrative tasks. When a Mission is preparing for activity close-out, for example, Mission Directors are responsible for developing plans that include the following:
“The retention of sufficient and suitable staff members and the delegation of authority and assignment of specific responsibility to each to carry to completion the required close-out action” [8]
“Estimates of the personnel, space, and funds required to complete the close-out” [8]
“The timetable for phasing out, transferring, or terminating U.S. direct-hire, participating agency, and contract personnel and the replacement of assistance from other resources” [8]
“Actions for terminating the services of cooperating country and third country national staff” [8]
In its 2003 version, but since removed:
“Missions must think in terms of leaving sustainable and useful units of assistance” [9]
“A monthly timeframe/activity projection for the entire close-out period [approximately six months] of major activities that must occur” [9]
Similarly, in the case of Interagency Agreements, USAID Agreement Officers’ (AO) personal and delegated close-out responsibilities are limited to the collection of financial reports and compliance-related documents [10]. AOs are expected to work in this same, limited capacity as they terminate Assistance and Acquisition Awards [11].
By contrast, when USAID gives grants to NGOs and has “little direct involvement,” recipients are strongly encouraged to adhere to six substantial, well-rounded “tips,” which include [12,13]:
*Staff refers to each NGO’s own staff; based on [13]
Among all of the terms and conditions reviewed, USAID’s exit strategies for foreign Missions, Agreements, and Awards have been almost exclusively administrative in nature. Although the Agency should not be solely responsible for ensuring the sustainability of its programming, its lack of regulation in this realm risks both undue pressure on implementing partners to do so and short-lived progress in the wake of its exit [14].
Fortunately, USAID has demonstrated its interest in sustainable close-out by commissioning studies and reports. In March 2020, USAID’s three-year “collaborative learning project” called Stopping as Success published a checklist for sustainable project transition [15,16]. The publication offers 16 concrete suggestions ranging from “build[ing] trust” with local communities to requiring implementers to report their progress in carrying out a pre-determined sustainability plan [16]. Similarly, the Food and Nutrition Technical Assistance III Project – a partnership between USAID and FHI 360 – studied the consequences of four food assistance programs and made thorough recommendations for ensuring the long-term impact of future projects [14]. These recommendations include accounting for “predictable external shocks and stressors” like natural disasters, and building “vertical linkages […] between community-based organizations or individuals and existing public or private sector institutions” [14]. Armed with this impressive base, USAID is well-equipped to account for new, evidence-based close-out practices.
Given the sheer scale of EU development programming – 28 countries providing 47% of the world’s development aid – no specific, all-encompassing mandates for the close-out process was found [17].ᶜ Perhaps as a result, there is notoriously little development policy coordination among Member States:
Due to inconsistencies among the eight documents outlining the EU’s development objectives and indicators, an internal audit characterized its collective approach as “mixed – at best” [18]
High-level meetings have explicitly prioritized “consensus on development” and “working better together,” producing appropriately vague guidelines that do not specifically address closure [19,20]
e.g. EU staff should “help to promote more inclusive, sustainable growth that does not compromise the ability of future generations to meet their needs” [19]
Its main development finance body, the Development Cooperation Instrument, has been critiqued for its “ambitio[us] desig[n] to address a very broad range of issues, from policy priorities in the agenda for change to EU policy concerns” [21]
For insight into the EU’s exit strategies, one must look to guidelines for specific types of development programming.
[33]
In the case of joint programming, when Member States work together to design and implement a country-wide, multi-sectorial project, donors must “reach agreement on a responsible exit strategy” for “phasing out sectors outside focal/centration areas, without creating a financing gap and with minimum disruption to the partner country” [20]. The policy offers two examples of such a strategy: “including scenarios in which other participating partners ‘plug the gap’ and/or other forms of cooperation beyond ODA [official development assistance] (e.g. the private sector)” [20]. The strategy must be drafted with input from the recipient country “as much as possible,” and may include either the intervention of other stakeholders or the private sector [20]. These discussions make take place at any of the following four “entry points” [20]:
“At the start of a new national development plan cycle, or a major change in the country” [20]
“At the start of a new strategic programming period or a review process for several important development partners” [20]
“During high-level country meetings […] where partner governments present their plans and solicit development partners support” [20]
“In situations of fragility or transition from humanitarian aid to more structured programmatic aid and more development partners” [20]
The development of context-specific exit strategies by Member States and recipients allows for practicable, fairly delegated, and concrete steps to exiting. However, this provision omits mention of enforcement or evaluation, may take place at the end of the implementation period, and does not encompass best practices as defined by INTRAC [1].
Outside of joint programming, the EU oversees projects funded by its executive implementing agencies, such as the Consumers, Health, Agriculture and Food Executive Agency and the Executive Agency for Small and Medium-sized Enterprises (EASME). In those cases, agency-wide final report guidelines could shed light on collective guidance but these are publicly inaccessible [22].ᴰ However, an early iteration of the Horizon 2020 Programme’s final report guide is still available online. The EU had required that implementers submit 20-30 lines of text addressing the following:
“Inform about activities undertaken during the project to ensure that your action
continues and reaches out further. Inform also about plans and intentions to continue
and sustain the project activities after the end of the project. Indicate the state
of advancement (e.g. agreed action, concrete proposal, idea)” [23].
Although field staff may not have been required to tangibly ensure the sustainability of their projects – and may not even make these assessments anymore – it is clear that at one time, the EU mandated assessments of expected sustainability at the end of program-specific project implementation.
With no overarching guidance for EU project exits, countries and individual NGOs have resorted to choosing or developing their own. The think tank E3G – funded in part by EASME [24] – has issued both internal and guidance for sustainable programming [25,26]. Its key points include “consultative engagement[s with] a broad range of stakeholders,” “integrat[ing programs] with long term national development plans,” and “strengthening city and local-level planning and institutions” [25]. These recommendations, however, are scattered among legal studies, commissioned reports, and briefing papers. By contrast, another EASME-funded program called Carbon Market Watch adheres to the project cycle guidelines of UN committees [27,28,29].ᴱ Although their Project Design Document and methodology forms request specific information about monitoring and evaluation, they do not ask for implementers’ plans for program close-out or long-term impact [30].
[32]
Other EU member-country NGOs end up developing sustainable exit measures mid-project. One example is a 2011-2013 agricultural project in Ethiopia implemented by People in Need, an international NGO based in the Czech Republic and funded in part by the EU’s Commission for Civil Protection and Humanitarian Aid [31]. Two months before its end, an evaluation of the program’s sustainability was conducted, receiving such recommendations as: 1) “increasing [recipients’] financial sustainability;” 2) “ensuring that FTCs’ [Farmer Training Centers] services benefit all, not only model farmers;” and 3) “providing books in Amharic language, not English” [32]. In the remaining time, 75% of these recommendations were met, leading to the internal assessment that the “sustainability of the project’s outcomes is likely to be largely positive” [32].
Learning from One Another
USAID and the EU operate at starkly different scales – one a country-specific bilateral donor, and the other an economic union of dozens of donor countries – but each of their exit strategies could be strengthened by incorporating elements from the other’s.
The EU could learn from the USAID’s multifaceted guidance for NGOs; the considerations listed in its final report guide are limited to the post-project involvement of EU field staff. It should explicitly promote local participation for sustainability during project close-out and beyond, such as transferring operations to local organizations or businesses.
In turn, USAID’s administration-centric strategy could benefit from the large-scale considerations of EU policies. Discussions with recipient governments, particularly in turbulent circumstances, could strengthen diplomatic relations with the US in addition to supporting reasonable transition plans. Formally soliciting exit plans from project implementers could also produce more sustainable project results.
NOTE: Valuing Voices thanks our excellent Harvard intern Rachel Sadoff for this research! We have a question: we found no EU/EC sites discussing exit strategies recommendations or guidance for global development projects. Do you have some? Please share in the comments below. Thank you.
Footnotes:
A) Projects and activities need to explain and demonstrate how their programming advances the objectives of USAID’s Country Development Cooperation Strategies for the given state (see ADS Chapter 201).
B) Current totals are publicly available on the Agency’s Country Roadmap website.
C) VV’s search spanned over five hours.
D) As of writing, final report guidelines are only available through an exclusive portal.
E) These are the Clean Development Mechanism and Joint Implementation Committee of the United Nations Framework Convention on Climate Change (UNFCCC).
[14] Food and Nutrition Technical Assistance. (2015, December). Effective Sustainability and Exit Strategies for USAID FFP Development Food Assistance Projects. Retrieved from https://www.fantaproject.org/research/exit-strategies-ffp
[20] European Commission. (2018, June). Joint Programming Guidance: Supporting EU Delegations to Work Better Together with Member States, Like-Minded Partners and Country Stakeholders. Retrieved from https://knowledge.effectivecooperation.org/node/77
[23] European Commission: Intelligent Energy Europe. (2012). Guidelines for Drafting the Final Technical Implementation Report. Retrieved June 2020 from https://ec.europa.eu/easme/en/technical-reporting
[31] Directorate-General for European Civil Protection and Humanitarian Aid Operations. (n.d.). Agreements for Humanitarian Aid and Emergency Support Within the EU Awarded in 2017 by DG ECHO. Retrieved June 2020 from https://ec.europa.eu/echo/files/funding/agreements/agreements_2017.pdf
Did we make enough time and measurement to foster sustainability, as we phased-down and phased-over an array of activities, alongside those remaining (white paper forthcoming)? Did we abruptly phase-out leaving partners and participants at a loss? Sharing power over all these decisions will influence what lasts.
There is an amazing breadth of local, ongoing resources, skills & capacities, linkages, motivation (thanks to Tufts FHI360’s work for USAID’s Food For Peace) that we can explore and learn from. There are local innovations and an array of unplanned collaborations (e.g., funding for health staff (Niger), training in small enterprise from the national government (Bangladesh), or private sector markets (Ethiopia) that can be accessed when partnerships are transparent and created one or more years pre-exit to collaborate on post-exit.
Ideally, we design and implement for exit from the onset. When we jointly set the timeframe and jointly assess risks to sustainability and adaptively manage exit, rather than exit based on pre-set timeframes, all sides win, with partners and participants able to foster sustainability. As USAID/ GIIN wrote about Responsible Exits for Impact Investors (2018), “the foundations for a responsible exit are laid even before an investment is made. To increase the likelihood of continued impact after exit, investors often select investees based on whether impact is embedded in their business model or inextricably linked to financial success. They also seek to understand the likely growth trajectory of the business, which has implications for which exit paths and options will become available.” They also note that a “growth strategy’ is needed throughout and at (investment) exit is “a company’s continued access to the right resources, networks, and knowledge” for sustained impact.
The need for a thoughtful approach to sustainability is shown by Hiller, Guthrie, and Jones in “Overcoming Ex-Post Development Stagnation“ (2016). The authors cite “limited evidence of program efficacies coupled with government and agency preference for planning, approval, and implementation processes rather than sustainment of outputs, outcomes, and impacts means that ex-post performance, scaling, and sustainability is not well understood or well pursued…. [There is a] lack of willingness to commit time and resources to rigorous evaluation of post-project effectiveness“. This affects a vast number of projects. For instance, they found 63,000 projects in 2003 alone, and “relative to the number of development projects undertaken, ex-post project [evaluations] are not commonly carried out, meaning that rates of success are often unknown and the complexity of causalities and ex-post dynamics of interactions and processes are not well understood.“ This limits our learning from what has (not) worked and what to do more (or less) of, including those that could not be sustained with only local resources.
We make sustainability assumptions are participants long for them, as Valuing Voices also found. Hiller et al. state that “whilst project documentation commonly conveys an expectation that some process of spread will occur ex-post, it rarely does, despite strong ex-post case-study evidence of stakeholder requests for further development opportunities.” This cautionary feedback could mean some project activities could be so resource-intensive that they could not feasibly be sustained or spread without long-term support, and retaining results may be limited to less costly activities. Valuing Voices found that other activities could be remunerative enough (financially, in health or education outcomes, for instance), as to be locally demanded and continued to be pursued. We have found in our Valuing Voices research and the Tufts research that activities where incentives did not continue, tended to die ex-post, while those which continued to bring benefits, such as cash crops and credit, water supply, and health, were prioritized ex-post, even in the absence of external funding continuing.
Hiller et al. outline that there are multiple ways in which scaling-up environmental sustainability over time, over area and, interestingly, scaling-within projects. The Ex-post Development Stagnation authors are clear that “creating conditions to support longer-term sustainability beyond project completion represents a recurring challenge, and it is not uncommon for activities and institutions to become inactive ex-post or for stakeholders to revert to previously unsustainable practices.” They note that some watershed studies have even found that participants actively destroy project measures in some cases. Certainly, the inability of locals to sustain often expensive activities without a project or larger organization’s support is common. “If it is assumed that development needs remain outstanding, then there may be merit in ensuring that development projects do not just remain “isolated, one-time interventions, like unconnected dots on a white page” or “islands of salvation.” The authors concur, “based on project subsidiarity and participatory principles, scaling-within management should be devolved to the local level (local authorities and local communities) to allow communities and individuals to filter out irrelevant practices and encourage adaptation and evolution of activities which are of greatest perceived livelihood benefit”.
As Valuing Voices research on exit has shown, it is a process, not an event(forthcoming, with thanks to I. Davies). The Hiller article notes that sustainability is best enhanced by capacity-building during implementation and with time for handover where “organizations adopt modes of functioning that allow local communities and organizations to build conceptual, operational, and institutional capacities. While scaling-down does not mean that governments disengage from processes such as community-driven development — it does, however, require it to be more flexible and responsive to locally generated demand to ensure the terrain is fertile for community organizations to emerge, learn, and grow.“
Let’s work together to extend the sustainability of impacts. Would love to hear your thoughts…
Assuming Sustainability and Impact is Dangerous to Development
(+ OECD/ DAC evaluation criteria)
We all do it; well, I used to do it too. I used to assume that if I helped my field staff and partners target and design funded projects well enough, and try to ensure a high quality of implementation and M&E, then it would result in sustainable programming. I assumed we would have moved our participants and partners toward projected long-term, top-of-logical-framework’s aspirational impact such as “vibrant agriculture leading to no hunger”, “locally sustained maternal child health and nutrition”, “self-sustained ecosystems”.
INTRAC nicely differentiates between what is typically measured (“outputs can only ever be the deliverables of a project or programme…that are largely within the control of an agency”) and what is not: “impact as the lasting or significant changes in people’s lives brought about by an intervention or interventions” [1]. They continue: “as few organisations are really judged on their impact, the OECD DAC impact definition (“positive and negative, primary and secondary long-term effects produced by a development intervention, directly or indirectly, intended or unintended“) allows for long-term changes in institutional capacity or policy change to be classed as impact” [1]. Do we do this? Virtually never. 99% of the time we only evaluate what happened while the project and its results is under the control of the aid implementer. Yet the five OECD/DAC evaluation criteria asks us to evaluate relevance, effectiveness, efficiency (fair enough, this is important to know if a project was good) and also impact and sustainability. So in addition to the prescription to evaluate ‘long-term effects’ (impact), evaluators are to measure “whether the benefits of an activity are likely to continue after donor funding has been withdrawn… [including being] environmentally as well as financially sustainable” [2].
How do we know we are getting to sustained outcomes and impacts? We ask people on the receiving end ideally after projects end. It is dangerous to assume sustainability and impact, and assume positive development trajectories (Sridharan) unless we consistently do “ex-post” project evaluations such as these from our research or catalytic organizations that have done at least one ex-post. At very minimum we should evaluate projected sustainability at end of project with those tasked to sustain it before the same project is repeated. Unfortunately we rarely do so and the assumed sustainability is so often not borne out, as I presented at the European Evaluation Society conference Sustainability panel two weeks ago along with AusAid’s DFAT, the World Bank, University College London and UNFEM.
Will we ever know if we have gotten to sustained impacts? Not unless the OECD/DAC criteria are drastically updated and organizations evaluate most projects ex-post (not just good ones :)), learn from the results and fund and implement for country-led sustainability with the country nationals. We must, as Sanjeev Sridharan tells us in a forthcoming paper embed sustainability into our Theories of Change from the onset (“Till time (and poor planning) do us part: Programs as dynamic systems — Incorporating planning of sustainability into theories of change” (Canadian Journal of Program Evaluation, 2018).*
There are remarkable assumptions routinely made. Many projects put sustainability into the proposal, yet most close out projects in the last 6 months. Rarely do projects take the time to properly phase down or phase over (unlike CRS Niger); many exit ceremonially ‘handing over’ projects to country-nationals, disposing project assets, and leaving only a final report behind. Alternatively, this USAID Uganda CDCS Country Transition Plan which looks over 20 years in the future by when it assumes to have accomplished sustained impact for exit [3]. Maybe they will measure progress towards that goal and orient programs toward handover, as in the new USAID “Journey to Self-Reliance” – we hope! Truly, we can plan to exit, but only when data bears out our sustained impact, not when the money or political will runs out.
As OXFAM’s blog today on the evaluation criteria says, “Sustainability is often treated as an assessment of whether an output is likely to be sustained after the end of the project. No one, well, hardly anyone, ever measures sustainability in terms of understanding whether we are meeting the needs of the present, without compromising the ability of future generations to meet their own need” and “too often in development we evaluate a project or programme and claim impact in a very narrow sense rather than the broader ecology beyond project or programme parameters” [4]. In fact, most ‘impact evaluations’ actually test effectiveness rather than long-term impact. Too rarely do we test impact assumptions by returning 2-10 years later and gather proof of what impacted locals’ lives sustainably, much less – importantly – what emerged from their own efforts once we left (SEIEs)! Oh, our hubris.
if you’re interested in the European Evaluation Society’s DAC criteria update discussion, see flagship discussion and Zenda Offir’s blog which stresses the need for better design that include ownership, inclusivity, empowerment [5][6]. These new evaluation criteria need to be updated, including Florence Etta’s and AGDEN‘s additional criteria participation, non-discrimination and accountability!
We can no longer afford to spend resources without listening to our true clients – those tasked with sustaining the impacts after we pack up – our partners and participants. We can no longer fund what cannot be proven to be sustained that is impactful. We talk about effectiveness and country ownership (which is paramount for sustainability and long-term impact), with an OECD report (2018) found “increases [in[ aid effectiveness by reducing transaction costs and improving recipient countries ownership” [7]. Yet donor governments who ‘tie’ aid to their own country national’s contracts benefit a staggering amount from ‘aid’ given. “Australia and the United Kingdom both reported … 93 percent and 90 percent of the value of their contracts respectively went to their own firms” [7]. It is not so different in the USA where aid is becoming bureaucratically centralized in the hands of a few for-profit contractors and centralized hundreds of millions in a handful of contracts. We must Do Development Differently. We can’t be the prime beneficiaries of our own aid; accountability must be to our participants; is it their countries, not our projects, and we cannot keep dangerously assuming sustained impact. Please let us know what you think…
[3] USAID. (2016, December 6). USAID Uganda Country Development Cooperation Strategy 2016-2021. Retrieved October, 2018, from https://www.usaid.gov/uganda/cdcs
[6] Ofir, Z. (2018, October 13). Updating the DAC Criteria, Part 11 (FINAL). From Evaluation Criteria to Design Principles. Retrieved from https://zendaofir.com/dac-criteria-part-11/
Embedding Sustainability Everywhere – All Five Slices Now
It has been a tumultuous year, and next year does not look like we will have much stability as a respite. As domestic concerns grow larger in two huge economies, US and UK, the question of the place foreign aid will play abound in conversations around the world.
However 2017’s transitions transform our work, for now I do have some good news,
1) Sustainability can be cheap. Far cheaper, in fact to design for sustainability, create feedback loops checking on sustainability through the eyes of our partners and participants, monitor for sustainability than to assume it’ll happen and far cheaper than finding out funds could have had far greater impact if we had valued their voices in the first place.
In our work to help our clients and partners fund, design, implement and monitor/ evaluate with sustainability in mind, we created what we hope is a helpful tool (guidance forthcoming).
a) By Designing for Sustainability with those who will sustain them, their financial buy-in and commitment are far higher (see CRS/ Niger [1]), as is advocacy and community buy-in (see new post-project OXFAM/ DRC [2]) and there are indicators the costs of start-up later are more cost-effective.
b) By clarifying Sustainability Indicators we check assumptions about who will do so, how much of a priority our activities are before we scale them up (Federation/ Ethiopian Red Cross). Retrospective post-project sustainability evaluations also enable us to learning from past successes and do better.
c) Sustainability Monitoring and Adaptation involve those pesky but pivotal feedback loops which are vital to understanding if we have gone off the rails or not, especially in terms of unexpected shocks derailing logical frameworks of designed projects. USAID’s nice recent CLA (Collaborating, Learning, Adapting) process includes donor funding for adaptation mid-stream which fosters effectiveness and sustainability [3]. Even lovelier are the Doing Development Differently examples that are often very low-cost and high-effect.
d) Informed Exit, Stakeholder Sustainability Consultation should be done throughout the cycle, at least a year earlier than most projects can begin this (note; not the last few months please). Transitioning for success leverages sunk costs for ongoing results. It includes heavy knowledge management on how the implementers managed information and resources, tracked data, sustained outputs and outcomes which now local partners will need to do, etc (FFP Exit Strategies study, a Czech study, and a new UK Results study shows range of items to consider during and post-transition) [4] [5].
e) Post-Project Sustainability: Our Valuing Voices/ Better Evaluation/ Tufts joint presentation at AEA on SEIE has more on how learning from post-project evaluation lessons can change sustainability of future projects for the better [6]! Further, what capacities and systems have been built in-country to sustain the results after our funding and expertise leaves? What can we do differently?
Can it be done? Demand is rising. I recently presented at a conference about embedding sustainability in programming now, and an enterprising NGO took this idea to heart. They proposed joint design with communities, which is the very bedrock of designing for sustained impact. Kudos!
2) Secondly, donors are willing to pay for sustainability. Sustainability is ensconced in USAID/ Food For Peace’s (FFP) 2016 Strategy and CLA (above) [7]. In the section called New learning and Implications for FFP Programming, they present findings such as “actions that drive big results during the life of the project may actually undermine sustainability in the long run. It raises the question as to whether FFP is willing to accept more modest results in the near term if they can be delivered in a way that will yield more sustainable gains over time….”
They also point us in the direction of country-led development: “Sustained capacity, resources, motivation, and linkages all require a focus on catalysts for change beyond FFP. Facilitative approaches that rely on and strengthen local actors help ensure that resource and knowledge transfers, and the incentives and linkages that support them, will be self-perpetuating beyond project end”. Notably, while UK’s DFID focuses on maximizing impact through Value-for-Money, it is a shorter-term economy, efficiency and effectiveness rather than sustained impact for the end-users and DFID’s exit strategies have recently been critiqued [5].
There is an issue of disincentives for the new administration to heed (if the agricultural lobby for US food exports does not prevail): “In [USAID’s exit] study evaluating how sustainable the results of Title II development programs are 2–3 years after project closure, FFP found that “providing free resources can threaten sustainability, unless replacement of those resources both as project inputs and as incentives has been addressed” [8]. As the Natural Resources section notes, “Whether entirely in the hands of the community or linked to a formal institution, the incentives and resources necessary to maintain a community asset are part of the system that will sustain it. The lack of such systems is visible in rusted irrigation pumps, failed mangrove plantations, abandoned bore wells, eroded dikes, and silted-in fish ponds around the world” [7]. Yet the private and public sectors are important: “Sustainable, broad-based change is more likely to be achieved by supporting and strengthening existing community, private sector, and public sector mechanisms for product and service delivery, and by supporting the capacity, quality, and accountability of government institutions” [7].
FFP’s new strategy calls for taking a systems approach to change that emphasizes sustainable long-term gains over unsustainable short-term wins. Even more delightful, in a small meeting at MFAN with Dina Esposito, Director of Food For Peace in November, she announced that they were looking to do pilot funding of an additional three years to typical five-year DFAP development projects. One year would involve collaborative participatory design between partners, communities and FFP, the second additional years being evaluation post-project of sustained and emerging impact!
This is a sea shift that can hopefully withstand political winds. After all, US foreign aid accounts for less than 1% of our federal budget, even though many Americans believe it is over 15% (hence easy to cut)…. but fingers crossed the aid effectiveness value of our work is… Valued.
[4] Food and Nutrition Technical Assistance (FANTA). (n.d.). Effective Sustainability and Exit Strategies for USAID FFP Development Food Assistance Projects. Retrieved from https://www.fantaproject.org/research/exit-strategies-ffp
[5] Del Mese, F. (2016, November 16). When aid relationships change: DFID’s approach to managing exit and transition in its development partnerships. Retrieved from https://icai.independent.gov.uk/report/transition/
[8] Rogers, B. L., & Coates, J. (2015, December). Sustaining Development: A Synthesis of Results from a Four-Country Study of Sustainability and Exit Strategies among Development Food Assistance Projects. Retrieved from https://www.fantaproject.org/research/exit-strategies-ffp
Presenting Lessons on (post-project) Sustained and Emerging Impact Evaluations from the U.S. AEA Conference
Dear readers, attached please find the Barking up a Better Tree: Lessons about SEIE Sustained and Emerging Impact Evaluation presentation we did last week at the American Evaluation Association (AEA) conference in Atlanta GA [1]. I had the pleasure of co-presenting with Beatrice Lorge Rogers PhD, Professor, Friedman Nutrition School, Tufts University (aka the famous Food for Peace/ Tufts Exit Strategy study [2]), Patricia Rogers PhD, Director, BetterEvaluation, Professor, Australia and New Zealand School of Government (where we recently published guidance on SEIE [3]) and Laurie Zivetz PhD, International Development Consultant and Valuing Voices evaluator.
We integrated our presentations from Africa, Asia and Latin America into this fascinating overview:
1.Sustained and Emerging Impact Evaluation: global context
2.SEIE: definitions and methods
3.Case studies: findings from post-project evaluations
4.Designing an SEIE: Considerations
5.Q&A — which fostered super comments, but since you couldn’t come, please tell us what you think and what questions you have…
There are amazing lessons to learn about design, implementation, M&E from doing post-project evaluation. We have also grown in appreciating that sustainability can be tracked throughout the project cycle, not just during post-project SEIE evaluation.
We’ll be building this into a white paper or a … (toolkit? webinar series? training? something else?). What’s your vote ___? (I know in this US election season, so… :)).
What would you like to get to support your learning about Sustained and Emerging Impact Evaluations? Look forward to hearing from you- Jindra@ValuingVoices.com
[2] Food and Nutrition Technical Assistance (FANTA). (n.d.). Effective Sustainability and Exit Strategies for USAID FFP Development Food Assistance Projects. Retrieved from https://www.fantaproject.org/research/exit-strategies-ffp
I am delighted to repost the blog on Responsible Donor Exit from Tshikululu, a Social Investment advisory firm in South Africa that I met at the European Evaluation Society conference last week in Holland [1]. The short report outlines different choices of Phase down, Phase out, or Phase over for Corporate Social Responsibility (CSR). As we in foreign aid evaluation have noted, donors should have set criteria for engaging with grantees to facilitate the transition and exit from programs, including exit plans and designing exit in from the beginning. What this report adds is open communication and joint agreements on timelines, exit grants and post-exit ‘scans’ that may foster further partnerships. Especially pleased to see them recommending “programme beneficiaries should therefore be empowered to direct the development processes that affect them.” Enjoy…
Towards responsible donor exiting strategies and practices
10 May 2016 | Silvester Hwenha |
Social investment has evolved as the result of a number of factors, including a growing interest by high net worth individuals and institutional investors in tackling social issues at the local, national or global level. Social investors have also become increasingly relevant in many countries as a result of mounting social challenges amid declining public funds to provide social services. The rationale for social investment is based on the realisation that social or environmental factors can impact a company’s bottom line and therefore are important factors in business. Besides, it has long been acknowledged by civic society and business that government alone cannot confront and solve all of society’s problems.
Social investors typically channel their funds through non-profit entities including non-governmental organisations (NGOs) and community based organisations (CBOs) to deliver social and environmental programmes in communities where such programmes are required. However, while social challenges require long term interventions to address, social investments often support programmes in short funding cycles. In many instances social investment funds are redirected to other social challenges thus necessitating exiting of programmes.
Exiting programmes is usually a highly sensitive and difficult process for donors, grantees and beneficiaries. For most donors, the reasons for exiting programmes include changing priorities and/or leadership, dwindling resources and the potential threat on programmes by the emergence of political instability. Despite having legitimate cause to exit programmes, donor agencies, foundations, trusts and corporate donors often do so with little advance notice, communication and consultation with programme partners.
Jindra Cekan, Ph.D. has used participatory methods for 30 years to connect with participants, ranging from villagers in Africa, Central/ Latin America and the Balkans to policy makers and Ministers around the world for her international clients. Their voices have informed the new Sustained and Emerging Impacts Evaluation, other M&E, stakeholder analysis, strategic planning, knowledge management and organizational learning.
If you don’t find what you are looking for via the search, categories, or posts above, you can go to the Blog page, scroll to the bottom, and click “previous posts” to go through all of the posts (newest–>oldest).
Cookie Consent
We use cookies to improve your experience on our site. By using our site, you consent to cookies.
ID used to identify users for 24 hours after last activity
24 hours
_gat
Used to monitor number of Google Analytics server requests when using Google Tag Manager
1 minute
_gac_
Contains information related to marketing campaigns of the user. These are shared with Google AdWords / Google Ads when the Google Ads and Google Analytics accounts are linked together.
90 days
__utma
ID used to identify users and sessions
2 years after last activity
__utmt
Used to monitor number of Google Analytics server requests
10 minutes
__utmb
Used to distinguish new sessions and visits. This cookie is set when the GA.js javascript library is loaded and there is no existing __utmb cookie. The cookie is updated every time data is sent to the Google Analytics server.
30 minutes after last activity
__utmc
Used only with old Urchin versions of Google Analytics and not with GA.js. Was used to distinguish between new sessions and visits at the end of a session.
End of session (browser)
__utmz
Contains information about the traffic source or campaign that directed user to the website. The cookie is set when the GA.js javascript is loaded and updated when data is sent to the Google Anaytics server
6 months after last activity
__utmv
Contains custom information set by the web developer via the _setCustomVar method in Google Analytics. This cookie is updated every time new data is sent to the Google Analytics server.
2 years after last activity
__utmx
Used to determine whether a user is included in an A / B or Multivariate test.
18 months
_ga
ID used to identify users
2 years
_gali
Used by Google Analytics to determine which links on a page are being clicked
1) Sustainability can be cheap. Far cheaper, in fact to design for sustainability, create feedback loops checking on sustainability through the eyes of our partners and participants, monitor for sustainability than to assume it’ll happen and far cheaper than finding out funds could have had far greater impact if we had valued their voices in the first place.
