Who is accountable for the ‘sustained development’ of those who suffer, and for how long?
Some of the rarely discussed myths of ‘sustainable development is that the aid we donors and implementers bring will help everyone, and that recipient governments can take over once donors leave and that what we left when the project ended was sustainable. The fact that our ‘global development aid’ helps a small fraction of all those who have equally ‘worthy’ needs in the countries we target is unspoken, as is the fact that these projects aren’t often built to sustainably withstand shocks such as climate changes bearing down now. As evaluator Michael Quinn Patton writes, “Effective programs… create islands of protected effectiveness in a sea of need and suffering… [we must] assess sustainability over time for Adaptive Resilience.”
More often, countries need to be: a) so poor, b) so ‘fragile’, or c) so geopolitically important to warrant our aid. The fact that most USAID and EU bilateral funding (not necessarily multilateral funding to IMF, World Bank, African or Asian or other Development Banks) is focused on fragile/ war-torn or the strategically important countries edges out what is left for the impoverished people of the world. Most of our taxpayers have little idea of this and believe it is needs-based. So, many of us assume that: d) our mostly short-term aid is either only the brief support that they need (the ‘shot in the arm school’) as might be true of emergency or humanitarian aid post-emergencies or e) the results will be so excellent as to spontaneously spread (scale) ‘forever’ that no more aid will be needed or f) that national governments – not donors- need to carry the accountability for sustained improvements forward. Yet those stakeholders in poor or fragile countries (governments, non-profit NGOs) often have the weakest capacities to sustain results. Rutere Kagendo, a fellow Kenyan on the Valuing Voices team, wrote a moving blog about who ends up tasked with sustaining project activities and results: communities, especially women.
This is in spite of the fact that ex-post project evaluations done 1+ to 30 years after completion who asked participants and partners about sustainability, show ‘mixed’ results, namely that results fall off as early as 2 years after closure, by between 10-100%. Rarely do ex-posts have results improve or be as sustained as we assume. Some project activities do show somewhat lasting results, ranging from those that provide credit to those generating agriculture. Mostly lackluster results are because global development projects aim for success by closure over sustainability over decades by exiting better with accountability to our participants and partners over time. We do not design what can be locally sustained. Imagine how much would be saved if we did!
Jindra Čekan/ova of the USA/ Czech Republic and Peter Kimeu of Kenya offer our perspectives about who is accountable for aid, and for how long.
I have worked for international NGOs (e.g. Catholic Relief Services, the international division of the American Red Cross, large INGOs such as CARE, World Vision, Lutheran World Relief, and others), including bilaterals (USAID) and foundations (the Bill & Melinda Gates Foundation, Aga Khan Foundation) for over 30 years. Most of our projects strove to fulfill objectives of the grants and were successful. Yet rarely did we ask ‘for how long?’ At one point, I worked for a huge International NGO with a program that had been feeding 50,000 West African children breakfast for 30 years. This was part of a bilateral aid education support + ‘safety net’ program. Many studies, including this terrific one from the UN’s World Food Program, show that such school feeding improves “school participation (enrolment, attendance, completion) and learning (scores on cognitive, language and mathematics tests)… [and] decrease child marriages, etc.“
Clearly, they do good. Yet the donor suddenly wanted proof that the effects of this long-term aid had improved national GDP rather than just the nutritional and learning outcomes of assisted individuals, otherwise, the project would be canceled. Letters from the impoverished country– including the country’s president and cabinet -stating they themselves had benefitted from the program which in turn led them to great educational outcomes and leadership had some effect. Yet without lobbying from the donor country’s agriculture and food aid industry that cutting food aid exports would harm them, it would have been canceled. Did they care about the schoolchildren’s learning or only providing outlets to US agricultural surplus producers? Was 30 years too long to keep helping? How do we have these discussions as equal partners? As evaluator Zenda Offir notes regarding the SDG’s No One Left Behind, “the burden of supporting and sustaining a majority of ‘leaving no-one behind’ efforts fall inevitably on many of the poorest (low-income) countries in the Global South. The problem is that they cannot afford it, nor can they sustain it. It will therefore be unfair to hold such countries accountable for ‘leaving no-one behind’ strategies. “
This brings up the questions of sustainability and accountability ‘to whom’ and ‘for how long’? You may have other questions, including ‘why’ and ‘how we know, which I look forward to addressing, but for now, these two are the focus of this blog. While 65% of Americans favor foreign aid, believing we spend up to 25% of our GDP abroad, the US spends just over 1%. Most US aid goes to the fragile and geopolitically strategic: “More than two hundred countries receive U.S. aid. It disproportionately goes to a few, however, with the top five all receiving over $1 billion per year as of 2016: Iraq ($5.3 billion), Afghanistan ($5.1 billion), Israel ($3.1 billion), Egypt ($1.2 billion), and Jordan ($1.2 billion).” In Europe, only 3 countries met the OECD goal of giving 0.7% of GNI: Norway, Sweden, and Denmark while the Czech Republic was at 0.13%, just below the USA 0.16%(2019). A fascinating measure of ‘commitment to development” (CDI) looking at the ‘quality of aid’ found that in the Czech Republic aid performance was very poor: “Adding up both quality-adjusted aid (95.7 million USD) and quality-adjusted charity induced by public policy (1.1 million USD), we arrive at 96.8 million USD for 2009 which amounts to 0.054% of GNI. Translating the percentage onto the standardised CDI scale the Czech Republic… [has] the third least favorable aid policy towards developing countries among DAC countries… [and] Aid allocation is not primarily focused on low-income countries” which is in part explained by the recent shift from aid recipient to aid donor (Syrovatka and Krylova 2012). Even, worse now, donors and international NGOs distribute aid (especially what is left for the poor countries of the world) that is annually allocated, but as the pandemic has led more spending to be domestic, aid to the poorest has decreased among almost all donor countries, bad news as Covid-economic downturns continue and climate change ramps up.
Is there proof there is no more need for aid to places or people? We do not know as we return to evaluate the sustainability after projects close far less than 1% of the time. Mostly this is because we assume that the recipient governments have ‘taken over’, that there is funding from elsewhere, or that the communities and organizations helped have become as resilient as to keep up the good work themselves. Yet the ex-post evaluation data does not bear this out. Few such evaluations are done or done well and many assumptions of positive trajectories are unproven. Donors and INGOs want to help, must leave after money is spent, and assume the best. Local participants implement the projects but they do not design or lead their implementation, which limits continuation after donor support exits.
So who is accountable to the poor whom we help? Peter comments on that from the perspective of the CEO of a local Kenyan NGO targeting 15,000 farmers.
I have over forty years of experience in development; 8 years lead in Community Initiated (Harambee) High schools, 35 years with Catholic Relief Services in Emergency Relief, Sustainable Development and Justice and Peace Programs, and currently 6 years with the (Kenyan) county-devolved sustainable development. I am the Founder and CEO of Decent Living Institute of Organic Farming promoting avocado farming, aquaculture, and apiculture for improved decent livelihoods. My early life as a young boy makes me a living witness of a life in deep poverty, which the New York Times featured.
The question ‘who is accountable for sustained development’ and ‘for how long’ has an assumption that it is possible to attain sustainable development without the continued involvement of those who suffer. I don’t think so. Sustained development occurs as a process to a transformed situation from abject poverty, a condition of want without the capacity to satisfy even the most basic needs, a position of lacking continuously leading to untold suffering and living in dehumanized conditions for the sufferer and the generations to come to the desired decency of fulfilled living. Living as a pauper in my first 30 years of life, having been born in a paupers’ family, I accepted the conditions of poverty and hunger as a way of life. After all, you know nothing better and when you see wealth around you, it is meant for the lucky few, and not for you. The situation limits the poor to survival conditions, eating from hand to mouth and everything is left to luck.
Aid to the poor would make sense if it is used as a catalyst to motivate and enable the poor depart from the circle of poverty (the poor giving birth to more poor) and is able to sustain the conditions of being above the poverty line of US$2 a day. Such aid would enable them to have enough to take care of their daily basic needs and create wealth without falling back below the poverty line repeatedly, for generations to come. The impact measure for aid should therefore be participatory learning from and measuring the extent to which success is sustained documenting representative success stories by participants who have left the circle of poverty sustainably. Such would include ‘in the past I couldn’t to find enough to eat occasionally slept without food and now my family has no idea of how it feels like to be hungry.
Unfortunately, the manner of delivering aid is seen as pure luck by the targeted poor for it comes without involving the poor as to strategically plan long-term impact that they can sustain. The aid donors and implementing agencies will target a given county, while the identification of targeted community cluster location for aid will be influenced by either by powerful persons from the locality or larger numbers in the public participation, so those with greater authority or louder voice will take the day. The decision on who will participate in the project finally will be determined by the same criteria and not the poverty levels. One example is the aid fund for COVID-19 response in Kenya which was distributed to the well connected to persons of authority and not to those who championed the control of the coronavirus. A decision was made at the county governments to disperse one million Kenya shillings to every cluster of villages to pay the youth for engaging in communal work such as community road works, terracing a degraded land, or even constructing an earth dam and paid per piece work completed – termed employment -to cushion the youth who have lost job opportunities due to the COVID -19 effects. A million shillings in a cluster of ten villages would perhaps engage 100 young people for a week earning Kenya shillings 500 (US$5) a day or $30 a week. The rest of the money – 50% of the total or more – does not go for wages as planned but is used to cover the management of the program by the county officials. The youth will spend this money like it is good luck for it is too little to ever think of the future investments.
However, the same amount is what it costs to support a member of the self-help group and collective community-led development in our Decent Living NGO per family of a vulnerable child to grow a vegetable garden, keep six chicks and grow three Hass avocado seedlings. Further, the participants commit to support another poor family with six chicks in a years’ time. From the onset, the poor are involved in ‘planning in advance’ to help others. Their developing vision is guided by the long-term impact they hope for, such as the family economic boost that will cover the full cost of schooling, medical expenses, and family meals, clothing, and shelter for all the children including the most vulnerable. Other long-term indicators will be the percentage of poor families that are above the poverty line meeting the family basic needs sustainably.
I see the role of the aid donor as to holding the aid receiver (local government and recipient communities) to their goals of sustainable development and to account for the funds given by reaching their goals and targets that must be time-bound. The aid receivers are also responsible to account for their aid distribution to their intermediary implementing partners (often local non-profits/ NGOs) to meet their targets, goals and should track expected and measurable long-term outcomes within three years after the closure of the project. This means to deliver not only the aid funds but also through the funding the systems established or improved at the conception of the project should be accounted for during the project implementation period and will be impacting long term results transforming the community to the desired state long after the project activities.
It could be building sustainable infrastructure for long-term support to the poor. The sustained impact would then be numbers of poor that have transformed their poverty and created wealth through the developed infrastructure in an intergenerational, long-lasting way that could be measured in later years. Sadly, most of the aid givers do not see their role beyond the performance short-term outputs such as trainings given or outcomes leading to a change in farming practices, like the deliverables for the specific objectives in an agriculture project. Hence the success of the project is defined by these short-term indicators that measure outputs such as the target number reached with food aid, or even some changes in practices leading to improved yields, but once the project ends, all tracking of results end. The national stakeholders are – or should be- responsible to demonstrate how the results of the project will be assumed by the community’s self-help groups so that the impacts become intergenerational. For it is vital to see that the project does not end with the implementing agencies. It is not only short-sighted by aid donors to believe that it ends, but national stakeholders are absconding their responsibility of accountability to the long-term impacts that are related to relationships and behavior change sustainably when they do not sustain them.
The UN’s ‘Sustainable Development Goals’ are merely a dream for most poor until the individual struggling with conditions of want is able to take steps towards permanent solutions for themselves and their future generations. It takes an oppressed dreamer (the poor with empty stomachs) who believes a change is possible to demand accountability. It also takes a progressive facilitator (donors and national stakeholders) who believes in creating enabling conditions for the oppressed to succeed. Both the oppressed dreamers (project participants, local implementing NGO agencies and the progressive facilitator (donor) are accountable to the transformed conditions. For the ‘sustained development’ to occur it must be intentionally dreamed of by all parties engaged in the process of development.
I dream of “a just world where everyone is fully a participant and celebrates sustainable development for all” wrote Pope Francis in his 2015 Laudato Si encyclical. He calls for all humanity to take care of our Mother Earth and in return, she will provide for all, including addressing issues of global warming. In my world dream, I see a time when a transformation of the sufferer from distressful and oppressive conditions of living is eased by putting future dreams into action, for those who suffer with deprivation today and are thirsty for change. I wish to make reference to a story told in the bible Jesus meeting a blind beggar (Mark 10:51) shouting to Jesus for help. Jesus asked the beggar to identify what type of aid he needed. And the blind beggar’s request for the power to see was heard and his sight was fully restored, emancipating him from the bondage of begging. We are told he transformed from a beggar into a disciple of Jesus. The transformation of conditions to sustainable options starts with the bilateral donor engaging a poor government to undertake a particular development agenda that in return facilitates its citizens to enjoy sustained development. The donor government should hold the recipient government responsible and accountable of delivering sustainable options for its citizen as per the grant agreements with evaluation two to three years after the project closure.
The poor who may be targeted with the aid may seem passive, not having been involved right from the beginning of design, and may have limitations of identifying what to ask for, perhaps those with intermediary implementing INGOs may be aware of how well what is being offered can meet their needs. Setting up the appropriate structures, they may dictate and demand sustained development options for themselves and those who are suppressed in poverty. The major issue is that most often the victims of poverty are never engaged in aid’s design and only implement what is offered. The situation creates room for corrupt national governments, INGOs, and NGOs to make quick money. The donors should hold recipient countries and INGOs accountable for tangible results toward the Sustainable Development Goals indicators for every grant in aid for as long as it takes, not just reporting at the national level.
I see a world where what matters most, is how engaged those who have empty stomachs are in the development aid agenda, and how the aid is administered and accounted for themselves and the neighboring suffering households. That development is all about a sustained transformation for empty stomachs of our project participants, their immediate neighbors, their children, their husbands, and their fathers/in law and mothers /in law, their brothers/ in law and sisters/ in law. It is about a better living standard of their neighbors who lends salt and water, the generosity of their firewood friends, those neighbors who will never turn down an opportunity to offer help no matter what. If these impacts and long term outcomes are not evaluated and accounted for, those who suffer poverty will always consider projects as myths of ‘sustainable development’ and the aid provided by bilateral, multilateral donors or through INGOs/NGOs as beneficial to the lucky few, while recipient governments and participating communities and their future generations have no sustainable impact results.
 MQ Patton in New Directions for Evaluation “Transformation to Global Sustainability: Implications for Evaluation and Evaluators, 2019 (link inaccessible)
Sustainable Exit Strategies: USAID vs. EU
Once malnutrition has decreased, students’ attendance has risen, or the number of small businesses has doubled, program implementers may be quick to pack their bags and leave. But the impact of their work may be undermined if not undone if they leave before implementing context-specific, comprehensive plans for post-project sustainability. It is for this reason that exit strategies – and particularly institution-wide mandates by donors – are crucial components of effective development programming. The United States Agency for International Development (USAID and the European Union (EU) Commission both have close-out protocols in place, but much to learn from one another in ensuring long-term progress in the wake of their close-outs.
Recommendations for Project Close-Out
Although every program should develop and follow context-specific exit measures, non-profits and ex-post evaluators have found that some exit strategies consistently lead to long-term, sustainable impact. The International NGO Training and Research Centre (INTRAC), for example, states that donors and implementers should actively plan for close-out throughout the project – from program design to the post-exit stage . It recommends the following:
- Implementers should “build exit thinking into the design of the project,” and during the program, representatives should be monitoring pre-determined exit indicators 
- “Senior staff and management need to prioritize resources for exits in order to do them well, embed learning within organizations and across teams, and ensure mistakes are not repeated” 
- “Practitioners should also consider going the extra mile to look after staff at headquarters and within partner organizations” 
- “Document experiences and share them externally” 
- “Consider whether a timeline approach would work for you” 
The USAID Approach to Project Close-Out
According to USAID’s 2020 Program Cycle Operational Policy, the formal close-out process for specific projects are carried out by the USAID Mission team in the recipient country and Operating Units based in Washington, DC . However, the Program Cycle does not reference the close-out process, let alone offer guidance . Unlike country exits, which require thorough evaluation and international cooperation ,ᴬ the sustainability measures of close-out procedures for activities – components of country-wide programs – seem to be delegated to implementing agencies.
Tellingly, the USAID formally defines “sustainability” as “the ability of a local system to produce desired outcomes” after its projects end, which the Agency may “contribute to” by strengthening capacity in their respective realms . After close-out, USAID currently considers the sustainability of its projects to be a matter of “the country and/or targeted community’s commitment and capacity to achieve development solutions” . Commitment and capacity, in turn, are gauged by 17 country-level “self-reliance metrics” ranging from “liberal democracy” to “export sophistication” [6,7].ᴮ Even post-project monitoring and evaluation are explicitly meant to “reveal whether implementation is on track and results are being achieved” with no mention of sustainability or long-term follow-up .
In effect, this framework not only puts the onus of project sustainability on recipients, but evaluates it based on large-scale and potentially tangential metrics. The stakes of this practice are raised by the fact that the resultant “Country Roadmaps” inform when USAID ends its programming in a given country, endangering a range of populations that these metrics do not account for, like seniors and prisoners .
USAID’s official Program Cycle 
During close-out, USAID representatives seem to focus on administrative tasks. When a Mission is preparing for activity close-out, for example, Mission Directors are responsible for developing plans that include the following:
- “The retention of sufficient and suitable staff members and the delegation of authority and assignment of specific responsibility to each to carry to completion the required close-out action” 
- “Estimates of the personnel, space, and funds required to complete the close-out” 
- “The timetable for phasing out, transferring, or terminating U.S. direct-hire, participating agency, and contract personnel and the replacement of assistance from other resources” 
- “Actions for terminating the services of cooperating country and third country national staff” 
- In its 2003 version, but since removed:
- “Missions must think in terms of leaving sustainable and useful units of assistance” 
- “A monthly timeframe/activity projection for the entire close-out period [approximately six months] of major activities that must occur” 
Similarly, in the case of Interagency Agreements, USAID Agreement Officers’ (AO) personal and delegated close-out responsibilities are limited to the collection of financial reports and compliance-related documents . AOs are expected to work in this same, limited capacity as they terminate Assistance and Acquisition Awards .
By contrast, when USAID gives grants to NGOs and has “little direct involvement,” recipients are strongly encouraged to adhere to six substantial, well-rounded “tips,” which include [12,13]:
*Staff refers to each NGO’s own staff; based on 
Among all of the terms and conditions reviewed, USAID’s exit strategies for foreign Missions, Agreements, and Awards have been almost exclusively administrative in nature. Although the Agency should not be solely responsible for ensuring the sustainability of its programming, its lack of regulation in this realm risks both undue pressure on implementing partners to do so and short-lived progress in the wake of its exit .
Fortunately, USAID has demonstrated its interest in sustainable close-out by commissioning studies and reports. In March 2020, USAID’s three-year “collaborative learning project” called Stopping as Success published a checklist for sustainable project transition [15,16]. The publication offers 16 concrete suggestions ranging from “build[ing] trust” with local communities to requiring implementers to report their progress in carrying out a pre-determined sustainability plan . Similarly, the Food and Nutrition Technical Assistance III Project – a partnership between USAID and FHI 360 – studied the consequences of four food assistance programs and made thorough recommendations for ensuring the long-term impact of future projects . These recommendations include accounting for “predictable external shocks and stressors” like natural disasters, and building “vertical linkages […] between community-based organizations or individuals and existing public or private sector institutions” . Armed with this impressive base, USAID is well-equipped to account for new, evidence-based close-out practices.
UPDATE: A NEW DISCUSSION NOTE BY USAID ON EX-POST EVALUATION WHERE EXIT IS MENTIONED.
The EU Approach to Project Close-Out
Given the sheer scale of EU development programming – 28 countries providing 47% of the world’s development aid – no specific, all-encompassing mandates for the close-out process was found .ᶜ Perhaps as a result, there is notoriously little development policy coordination among Member States:
- Due to inconsistencies among the eight documents outlining the EU’s development objectives and indicators, an internal audit characterized its collective approach as “mixed – at best” 
- High-level meetings have explicitly prioritized “consensus on development” and “working better together,” producing appropriately vague guidelines that do not specifically address closure [19,20]
- e.g. EU staff should “help to promote more inclusive, sustainable growth that does not compromise the ability of future generations to meet their needs” 
- Its main development finance body, the Development Cooperation Instrument, has been critiqued for its “ambitio[us] desig[n] to address a very broad range of issues, from policy priorities in the agenda for change to EU policy concerns” 
For insight into the EU’s exit strategies, one must look to guidelines for specific types of development programming.
In the case of joint programming, when Member States work together to design and implement a country-wide, multi-sectorial project, donors must “reach agreement on a responsible exit strategy” for “phasing out sectors outside focal/centration areas, without creating a financing gap and with minimum disruption to the partner country” . The policy offers two examples of such a strategy: “including scenarios in which other participating partners ‘plug the gap’ and/or other forms of cooperation beyond ODA [official development assistance] (e.g. the private sector)” . The strategy must be drafted with input from the recipient country “as much as possible,” and may include either the intervention of other stakeholders or the private sector . These discussions make take place at any of the following four “entry points” :
- “At the start of a new national development plan cycle, or a major change in the country” 
- “At the start of a new strategic programming period or a review process for several important development partners” 
- “During high-level country meetings […] where partner governments present their plans and solicit development partners support” 
- “In situations of fragility or transition from humanitarian aid to more structured programmatic aid and more development partners” 
The development of context-specific exit strategies by Member States and recipients allows for practicable, fairly delegated, and concrete steps to exiting. However, this provision omits mention of enforcement or evaluation, may take place at the end of the implementation period, and does not encompass best practices as defined by INTRAC .
Outside of joint programming, the EU oversees projects funded by its executive implementing agencies, such as the Consumers, Health, Agriculture and Food Executive Agency and the Executive Agency for Small and Medium-sized Enterprises (EASME). In those cases, agency-wide final report guidelines could shed light on collective guidance but these are publicly inaccessible .ᴰ However, an early iteration of the Horizon 2020 Programme’s final report guide is still available online. The EU had required that implementers submit 20-30 lines of text addressing the following:
“Inform about activities undertaken during the project to ensure that your action
continues and reaches out further. Inform also about plans and intentions to continue
and sustain the project activities after the end of the project. Indicate the state
of advancement (e.g. agreed action, concrete proposal, idea)” .
Although field staff may not have been required to tangibly ensure the sustainability of their projects – and may not even make these assessments anymore – it is clear that at one time, the EU mandated assessments of expected sustainability at the end of program-specific project implementation.
With no overarching guidance for EU project exits, countries and individual NGOs have resorted to choosing or developing their own. The think tank E3G – funded in part by EASME  – has issued both internal and guidance for sustainable programming [25,26]. Its key points include “consultative engagement[s with] a broad range of stakeholders,” “integrat[ing programs] with long term national development plans,” and “strengthening city and local-level planning and institutions” . These recommendations, however, are scattered among legal studies, commissioned reports, and briefing papers. By contrast, another EASME-funded program called Carbon Market Watch adheres to the project cycle guidelines of UN committees [27,28,29].ᴱ Although their Project Design Document and methodology forms request specific information about monitoring and evaluation, they do not ask for implementers’ plans for program close-out or long-term impact .
Other EU member-country NGOs end up developing sustainable exit measures mid-project. One example is a 2011-2013 agricultural project in Ethiopia implemented by People in Need, an international NGO based in the Czech Republic and funded in part by the EU’s Commission for Civil Protection and Humanitarian Aid . Two months before its end, an evaluation of the program’s sustainability was conducted, receiving such recommendations as: 1) “increasing [recipients’] financial sustainability;” 2) “ensuring that FTCs’ [Farmer Training Centers] services benefit all, not only model farmers;” and 3) “providing books in Amharic language, not English” . In the remaining time, 75% of these recommendations were met, leading to the internal assessment that the “sustainability of the project’s outcomes is likely to be largely positive” .
Learning from One Another
USAID and the EU operate at starkly different scales – one a country-specific bilateral donor, and the other an economic union of dozens of donor countries – but each of their exit strategies could be strengthened by incorporating elements from the other’s.
The EU could learn from the USAID’s multifaceted guidance for NGOs; the considerations listed in its final report guide are limited to the post-project involvement of EU field staff. It should explicitly promote local participation for sustainability during project close-out and beyond, such as transferring operations to local organizations or businesses.
In turn, USAID’s administration-centric strategy could benefit from the large-scale considerations of EU policies. Discussions with recipient governments, particularly in turbulent circumstances, could strengthen diplomatic relations with the US in addition to supporting reasonable transition plans. Formally soliciting exit plans from project implementers could also produce more sustainable project results.
NOTE: Valuing Voices thanks our excellent Harvard intern Rachel Sadoff for this research! We have a question: we found no EU/EC sites discussing exit strategies recommendations or guidance for global development projects. Do you have some? Please share in the comments below. Thank you.
A) Projects and activities need to explain and demonstrate how their programming advances the objectives of USAID’s Country Development Cooperation Strategies for the given state (see ADS Chapter 201).
B) Current totals are publicly available on the Agency’s Country Roadmap website.
C) VV’s search spanned over five hours.
D) As of writing, final report guidelines are only available through an exclusive portal.
E) These are the Clean Development Mechanism and Joint Implementation Committee of the United Nations Framework Convention on Climate Change (UNFCCC).
 Lewis, S. (2016, January). Developing a Timeline for Exit Strategies. Retrieved from https://www.intrac.org/resources/praxis-paper-31-developing-timeline-exit-strategies/
 USAID. (2020, June 16). ADS Chapter 201: Program Cycle Operational Policy. Retrieved from https://www.usaid.gov/sites/default/files/documents/1870/201.pdf
 USAID. (2016, December 8). ADS 201: Program Cycle Operational Policy – Fact Sheet. Retrieved from https://usaidlearninglab.org/library/ads-201-program-cycle-operational-policy-fact-sheet
 USAID. (2020, February 26). Strategic Transitions Fact Sheet. Retrieved from https://www.usaid.gov/documents/1870/strategic-transitions-fact-sheet
 USAID. (2020, May). Discussion Note: Ex-Post Evaluations. Retrieved from https://usaidlearninglab.org/library/discussion-note-ex-post-evaluations
 USAID. (2020, June 3). USAID’S Country Roadmaps Fact Sheet. Retrieved from https://www.usaid.gov/documents/1870/usaids-country-roadmaps-fact-sheet
 USAID. (n.d.). The Journey To Self-Reliance: Country Roadmaps. Retrieved June 2020 from https://selfreliance.usaid.gov/
 USAID. (2018, July 17). Mandatory Reference for ADS Chapter 527: Administrative Guidance on How to Close a USAID Operating Unit – Checklists. Retrieved from https://www.usaid.gov/sites/default/files/documents/1868/527mab.pdf
 USAID. (2003, October 1). ADS 527 Mandatory Reference: Guidance on How to Close a USAID Mission – Checklist. Retrieved from https://2012-2017.usaid.gov/sites/default/files/documents/1868/527mab.pdf
 USAID. (2015, November 30). ADS Chapter 306: Interagency Agreements. Retrieved from https://2012-2017.usaid.gov/sites/default/files/documents/1868/306.pdf
 USAID. (2010, October 6). Guidance on Closeout Procedures for A&A Awards. Retrieved from https://2012-2017.usaid.gov/sites/default/files/documents/1868/302sat.pdf
 USAID. (2020, February 13). Doing Business with USAID. Retrieved from https://www.usaid.gov/work-usaid/how-to-work-with-usaid
 USAID, & FHI 360. (2012). The Essential NGO Guide to Managing Your USAID Award. Retrieved from https://www.fhi360.org/resource/essential-ngo-guide-managing-your-usaid-award
 Food and Nutrition Technical Assistance. (2015, December). Effective Sustainability and Exit Strategies for USAID FFP Development Food Assistance Projects. Retrieved from https://www.fantaproject.org/research/exit-strategies-ffp
 Stopping as Success. (n.d.). About Us. Retrieved June 2020 from https://www.stoppingassuccess.org/about-us
 Stopping as Success. (2020, March 17). USAID Mission Checklist for Sustainable Transitions. Retrieved from https://www.stoppingassuccess.org/resources/usaid-mission-checklist/
 European Commission. (2015, December). EU Site Explorer: Overview. Retrieved June 2020 from https://euaidexplorer.ec.europa.eu/content/overview_en
 Froidure, P., & Devillé, H. (2015). Review of the risks related to a results oriented approach for EU development and cooperation action. Retrieved from https://op.europa.eu/en/publication-detail/-/publication/b2921ec1-c589-11e5-a4b5-01aa75ed71a1
 European Commission Directorate-General for International Cooperation and Development. (2018, November 19). The New European Consensus on Development: ‘Our World, Our Dignity, Our Future’. Retrieved from https://op.europa.eu/en/publication-detail/-/publication/5a95e892-ec76-11e8-b690-01aa75ed71a1
 European Commission. (2018, June). Joint Programming Guidance: Supporting EU Delegations to Work Better Together with Member States, Like-Minded Partners and Country Stakeholders. Retrieved from https://knowledge.effectivecooperation.org/node/77
 Parry, M., & Segantini, E. (2017, October). How the EU Budget is Spent: Development Cooperation Instrument. Retrieved from https://www.europarl.europa.eu/RegData/etudes/BRIE/2017/608764/EPRS_BRI(2017)608764_EN.pdf
 Linköping University. (2020, May 25). Reporting to the European Commission. Retrieved June 2020 from https://insidan.liu.se/forskningsfinansiering/horizon-2020/rapportera-till-eu-kommissionen?l=en
 European Commission: Intelligent Energy Europe. (2012). Guidelines for Drafting the Final Technical Implementation Report. Retrieved June 2020 from https://ec.europa.eu/easme/en/technical-reporting
 LIFE Program. (2019, June 3). 2019 Operating Grants for NGOs. European Commission. Retrieved from https://ec.europa.eu/easme/en/section/life/2019-life-call-proposals-ngos
 Wright, H., Dimsdale, T., & Healy, C., et al. (2018, June 26). Sustainable Infrastructure and the Multilateral Development Banks: Changing the Narrative. E3G. https://www.e3g.org/library/sustainable-infrastructure-and-the-multilateral-development-banks-changing
 Naidoo, C., Amin, A., & Dimsdale, T., et al. (2014, April 9). Strategic national approaches to climate finance: Report on scoping work in Peru, Chile and Colombia on national climate finance pathways and strategies. Retrieved from https://www.e3g.org/news/media-room/strategic-national-approaches-to-climate-finance
 LIFE Program. (2019, June 3). 2019 Operating Grants for NGOs. European Commission. https://ec.europa.eu/easme/en/section/life/2019-life-call-proposals-ngos
 Carbon Market Watch. (2014, June 25). The Joint Implementation offsetting mechanism is deeply flawed – time to consign it to history? Retrieved from https://carbonmarketwatch.org/2014/06/25/the-joint-implementation-offsetting-mechanism-deeply-flawed-time-to-consign-it-to-history/
 Carbon Market Watch. (n.d.) CDM Project Cycle. Retrieved 2019, May 10 from
 United Nations Framework Convention on Climate Change. (n.d.). Guidelines for Completing the Project Design Document (CDM-PDD), the Proposed New Methodology: Baseline (CDM-NMB) and the Proposed New Methodology: Monitoring (CDM-NMM). Retrieved June 2020 from https://cdm.unfccc.int/Reference/Documents/Guidel_Pdd/English/Guidelines_CDMPDD_NMB_NMM.pdf
 Directorate-General for European Civil Protection and Humanitarian Aid Operations. (n.d.). Agreements for Humanitarian Aid and Emergency Support Within the EU Awarded in 2017 by DG ECHO. Retrieved June 2020 from https://ec.europa.eu/echo/files/funding/agreements/agreements_2017.pdf
 People in Need Ethiopia. (n.d.). Final Evaluation: PIN’s Support to Strengthening Agricultural Extension Services in SNNPR, Ethiopia. Retrieved from https://www.clovekvtisni.cz/en/learning-series-on-agricultural-extension-738pub
Assuming Sustainability and Impact is Dangerous to Development
(+ OECD/ DAC evaluation criteria)
We all do it; well, I used to do it too. I used to assume that if I helped my field staff and partners target and design funded projects well enough, and try to ensure a high quality of implementation and M&E, then it would result in sustainable programming. I assumed we would have moved our participants and partners toward projected long-term, top-of-logical-framework’s aspirational impact such as “vibrant agriculture leading to no hunger”, “locally sustained maternal child health and nutrition”, “self-sustained ecosystems”.
INTRAC nicely differentiates between what is typically measured (“outputs can only ever be the deliverables of a project or programme…that are largely within the control of an agency”) and what is not: “impact as the lasting or significant changes in people’s lives brought about by an intervention or interventions” . They continue: “as few organisations are really judged on their impact, the OECD DAC impact definition (“positive and negative, primary and secondary long-term effects produced by a development intervention, directly or indirectly, intended or unintended“) allows for long-term changes in institutional capacity or policy change to be classed as impact” . Do we do this? Virtually never. 99% of the time we only evaluate what happened while the project and its results is under the control of the aid implementer. Yet the five OECD/DAC evaluation criteria asks us to evaluate relevance, effectiveness, efficiency (fair enough, this is important to know if a project was good) and also impact and sustainability. So in addition to the prescription to evaluate ‘long-term effects’ (impact), evaluators are to measure “whether the benefits of an activity are likely to continue after donor funding has been withdrawn… [including being] environmentally as well as financially sustainable” .
How do we know we are getting to sustained outcomes and impacts? We ask people on the receiving end ideally after projects end. It is dangerous to assume sustainability and impact, and assume positive development trajectories (Sridharan) unless we consistently do “ex-post” project evaluations such as these from our research or catalytic organizations that have done at least one ex-post. At very minimum we should evaluate projected sustainability at end of project with those tasked to sustain it before the same project is repeated. Unfortunately we rarely do so and the assumed sustainability is so often not borne out, as I presented at the European Evaluation Society conference Sustainability panel two weeks ago along with AusAid’s DFAT, the World Bank, University College London and UNFEM.
Will we ever know if we have gotten to sustained impacts? Not unless the OECD/DAC criteria are drastically updated and organizations evaluate most projects ex-post (not just good ones :)), learn from the results and fund and implement for country-led sustainability with the country nationals. We must, as Sanjeev Sridharan tells us in a forthcoming paper embed sustainability into our Theories of Change from the onset (“Till time (and poor planning) do us part: Programs as dynamic systems — Incorporating planning of sustainability into theories of change” (Canadian Journal of Program Evaluation, 2018).*
There are remarkable assumptions routinely made. Many projects put sustainability into the proposal, yet most close out projects in the last 6 months. Rarely do projects take the time to properly phase down or phase over (unlike CRS Niger); many exit ceremonially ‘handing over’ projects to country-nationals, disposing project assets, and leaving only a final report behind. Alternatively, this USAID Uganda CDCS Country Transition Plan which looks over 20 years in the future by when it assumes to have accomplished sustained impact for exit . Maybe they will measure progress towards that goal and orient programs toward handover, as in the new USAID “Journey to Self-Reliance” – we hope! Truly, we can plan to exit, but only when data bears out our sustained impact, not when the money or political will runs out.
As OXFAM’s blog today on the evaluation criteria says, “Sustainability is often treated as an assessment of whether an output is likely to be sustained after the end of the project. No one, well, hardly anyone, ever measures sustainability in terms of understanding whether we are meeting the needs of the present, without compromising the ability of future generations to meet their own need” and “too often in development we evaluate a project or programme and claim impact in a very narrow sense rather than the broader ecology beyond project or programme parameters” . In fact, most ‘impact evaluations’ actually test effectiveness rather than long-term impact. Too rarely do we test impact assumptions by returning 2-10 years later and gather proof of what impacted locals’ lives sustainably, much less – importantly – what emerged from their own efforts once we left (SEIEs)! Oh, our hubris.
if you’re interested in the European Evaluation Society’s DAC criteria update discussion, see flagship discussion and Zenda Offir’s blog which stresses the need for better design that include ownership, inclusivity, empowerment . These new evaluation criteria need to be updated, including Florence Etta’s and AGDEN‘s additional criteria participation, non-discrimination and accountability!
We can no longer afford to spend resources without listening to our true clients – those tasked with sustaining the impacts after we pack up – our partners and participants. We can no longer fund what cannot be proven to be sustained that is impactful. We talk about effectiveness and country ownership (which is paramount for sustainability and long-term impact), with an OECD report (2018) found “increases [in[ aid effectiveness by reducing transaction costs and improving recipient countries ownership” . Yet donor governments who ‘tie’ aid to their own country national’s contracts benefit a staggering amount from ‘aid’ given. “Australia and the United Kingdom both reported … 93 percent and 90 percent of the value of their contracts respectively went to their own firms” . It is not so different in the USA where aid is becoming bureaucratically centralized in the hands of a few for-profit contractors and centralized hundreds of millions in a handful of contracts. We must Do Development Differently. We can’t be the prime beneficiaries of our own aid; accountability must be to our participants; is it their countries, not our projects, and we cannot keep dangerously assuming sustained impact. Please let us know what you think…
[*] This paper is now available at https://journalhosting.ucalgary.ca/index.php/cjpe/article/view/53055
 Simister, N. (2015). Monitoring and Evaluation Series: Outcomes Outputs and Impact. Retrieved from https://www.intrac.org/wpcms/wp-content/uploads/2016/06/Monitoring-and-Evaluation-Series-Outcomes-Outputs-and-Impact-7.pdf
 OECD. (n.d.). DAC Criteria for Evaluating Development Assistance. Retrieved September, 2018, from https://web.archive.org/web/20180919035910/http://www.oecd.org/dac/evaluation/daccriteriaforevaluatingdevelopmentassistance.htm
 USAID. (2016, December 6). USAID Uganda Country Development Cooperation Strategy 2016-2021. Retrieved October, 2018, from https://www.usaid.gov/uganda/cdcs
 Porter, S. (2018, October 18). DAC Criteria: The Hand That Rocks the Cradle. Retrieved from https://views-voices.oxfam.org.uk/2018/10/dac-criteria-the-hand-that-rocks-the-cradle/
 European Evaluation Society Biennial Conference: Flagship Symposia. (2018). Retrieved from http://www.ees2018.eu/1539782596-flagship-symposia.htm
 Ofir, Z. (2018, October 13). Updating the DAC Criteria, Part 11 (FINAL). From Evaluation Criteria to Design Principles. Retrieved from https://zendaofir.com/dac-criteria-part-11/
 OECD. (2018, June 11). 2018 Report On The DAC Untying Recommendation. Retrieved from http://www.oecd.org/dac/financing-sustainable-development/development-finance-standards/DCD-DAC(2018)12-REV2.en.pdf