Learning about Sustainability and Exit Strategies from USAID’s Food Assistance Projects
USAID overall and Food for Peace (FFP) specifically have become far more progressive in the Obama Administration and under Administrator Rajiv Shah, with a much greater focus on accountability and results. Those of you unfamiliar with USAID’s Food For Peace will learn it has been a large channel of international assistance for over 60 years and is not a small funding instrument. For 2016 alone, they have proposed spending $1.75 billion to feed 47 million people through humanitarian and development programs implemented by non-profits, for-profits and the UN’s World Food Program. Given this scale of resources, it is highly surprising that while many documents in their archive ask for post-project evaluation and there are a handful of desk reviews, they have done only two actual ones with new fieldwork in the last 30 years (these four countries and a recently published one on Uganda, see catalysts). This recent and excellent 2015 synthesis report by authors Rogers and Coates is presented below.
Commissioned by USAID, Tufts University and FHI360 have done a remarkably thorough two to three year post-project evaluation of four (Title II) food-assisted programs containing 12 projects in Bolivia, Honduras, India, and Kenya that closed out in 2009. The methodologies used are clearly outlined (itself a boon to our fledgling field) as are limitations and comments on context, findings and recommendations. It was no small feat to compare activities across four countries and so many sectors (some of which were supported by provision of US food aid resources, others with in-kind or cash inputs): maternal and child health and nutrition; water and sanitation; agriculture, livestock, and rural income generation; natural resource management; school feeding; and micro-savings and loans. Also this covered many implementers, from CARE, ADRA and Save the Children to World Vision, CRS and Feed the Hungry.
In this document, Dina Esposito, the Director of FFP states “We commissioned this report with the objective of determining what factors enhanced the likelihood of sustained project benefits, in order to improve our guidance for future food assistance development projects.… FFP development projects are designed to reduce the long-term need for food assistance by strengthening the capacity of developing societies to ensure access to nutritious food for their most vulnerable communities and individuals, especially women and children. The study team looked at 12 FFP development projects across four countries and asked not only what was achieved by each project’s end?, but also, what of those achievements remained one year after project close-out? and two years after? This rigorous, retrospective approach is not widely done, but is essential if we are to understand the true impacts of our investments. To be effective, development projects must result in changes that last beyond the duration of the project themselves.”
Process and findings:
The researchers compared baseline, midline and endline evaluations and exit strategy documents to new mixed-method data collection. There were four main findings:
1) Impact vs. Sustainability trade-offs: Evidence of project success at the time of exit (as assessed by impact indicators) did not necessarily imply sustained benefit over time. Just because projects were deemed successful at exit does not mean that those continued after closeout. “Moreover, the study found that focusing exclusively on demonstrating impact at exit may jeopardize investment in longer-term sustainability.” Valuing Voices found the same in Ethiopia in research done in 2013.
2) Preconditions to successful sustainability: In addition to an ongoing source of resources, good technical and managerial capacity, and sustained motivation of participants and partners, linkages to governmental organizations and/or other entities were key to continuity and sustainability of outcomes and new impacts. “No project in this study achieved sustainability without [the first] three of them in place before the project ended,” and linkages between community partners and the public/private sector were critical for handover (Figure 1, below). Further, a gradual transition from project-supported activities to independent operation was important for sustainability. “Sustainability was more likely when projects withdrew gradually, allowing community-based organizations to develop the capacity to operate independently.”
3) Free resources can threaten sustainability, unless replaced while there is no one-size fits all for resources: Using incentives has costs. “Free supplementary food in maternal and child health and nutrition projects or free marketing services in agriculture projects created expectations in many projects that could not be sustained once resources were withdrawn”. Valuing Voices found the same in research in Niger (report imminent). But other financing options, free health care or fee for service are still unsystematically studied regarding fostering sustainability in differing sectors.
4) External factors (climate, economy) can affect sustainability: The operating context and exogenous shocks (e.g., economic, legal and climatic) also affected the sustainability of project benefits, positively or negatively.
Most tellingly, the authors warned that “sustainability plans cannot be based on the hope that activities and benefits will continue in the absence of the key factors identified in this study.” Throughout the report and in pending country-specific studies, they outlined the assumptions that projects made about sustainability in order to exit and closeout, which were variably disproved, such as:
- Community health workers would continue to provide services although without remuneration,
- Households could continue to access nutritious food from their own (increased) production or purchases and have time, and know how to prepare such food,
- Farmers will pay for inputs with profits from increased production and commercialization and can meet the quantity and quality requirements of long-term contracts
- Community members will recognize the tangible benefit of Natural Resource Management activities and will be motivated to continue them without further inputs or remuneration
- Water committees will have sufficient administrative capacity and resources to manage their budgets effectively
- Community-based organizations have strong institutional capacity
- Partner organizations will continue to provide teacher training
- Government will have the resources and commitment to support future needs
The country studies with detailed findings are still forthcoming but these examples may illustrate the range of sustainability. There were some very well-sustained positive results in Food Production (India by area) and Child Health Growth Monitoring (Bolivia by consortium implementers) between baseline or enline and followup 2-3 years later:
As well as some far more mixed or negative results in examples across all the Water and Sanitation projects:
and far less stellar results in Maternal Child Health’s Community Health Workers (Kenya):
The authors recommended not only ensuring resources, capacity, motivation and linkages are present before exiting but also institutionalizing sustainable approaches to project design and evaluation including in solicitations and applications, project assessments, project management and knowledge management. They also recommended not only phasing down exit but also extending more such evaluations beyond the 5 years of implementation and assessing impacts as long as 10 years after. This requires some sizeable revisions to how development is done at Food For Peace.
All of these findings recommendations are near and dear to those of us at Valuing Voices. We strongly commend Food For Peace and ask for many more such studies, for unless we know what worked best and why, how do we know what to design next together with our partners and participants for real sustainability?