Who is accountable for the ‘sustained development’ of those who suffer, and for how long?
Some of the rarely discussed myths of ‘sustainable development is that the aid we donors and implementers bring will help everyone, and that recipient governments can take over once donors leave and that what we left when the project ended was sustainable. The fact that our ‘global development aid’ helps a small fraction of all those who have equally ‘worthy’ needs in the countries we target is unspoken, as is the fact that these projects aren’t often built to sustainably withstand shocks such as climate changes bearing down now. As evaluator Michael Quinn Patton writes, “Effective programs… create islands of protected effectiveness in a sea of need and suffering… [we must] assess sustainability over time for Adaptive Resilience.”[1]
More often, countries need to be: a) so poor, b) so ‘fragile’, or c) so geopolitically important to warrant our aid. The fact that most USAID and EU bilateral funding (not necessarily multilateral funding to IMF, World Bank, African or Asian or other Development Banks) is focused on fragile/ war-torn or the strategically important countries edges out what is left for the impoverished people of the world. Most of our taxpayers have little idea of this and believe it is needs-based. So, many of us assume that: d) our mostly short-term aid is either only the brief support that they need (the ‘shot in the arm school’) as might be true of emergency or humanitarian aid post-emergencies or e) the results will be so excellent as to spontaneously spread (scale) ‘forever’ that no more aid will be needed or f) that national governments – not donors- need to carry the accountability for sustained improvements forward. Yet those stakeholders in poor or fragile countries (governments, non-profit NGOs) often have the weakest capacities to sustain results. Rutere Kagendo, a fellow Kenyan on the Valuing Voices team, wrote a moving blog about who ends up tasked with sustaining project activities and results: communities, especially women.
Jindra Čekan/ova of the USA/ Czech Republic and Peter Kimeu of Kenya offer our perspectives about who is accountable for aid, and for how long.
Jindra
I have worked for international NGOs (e.g. Catholic Relief Services, the international division of the American Red Cross, large INGOs such as CARE, World Vision, Lutheran World Relief, and others), including bilaterals (USAID) and foundations (the Bill & Melinda Gates Foundation, Aga Khan Foundation) for over 30 years. Most of our projects strove to fulfill objectives of the grants and were successful. Yet rarely did we ask ‘for how long?’ At one point, I worked for a huge International NGO with a program that had been feeding 50,000 West African children breakfast for 30 years. This was part of a bilateral aid education support + ‘safety net’ program. Many studies, including this terrific one from the UN’s World Food Program, show that such school feeding improves “school participation (enrolment, attendance, completion) and learning (scores on cognitive, language and mathematics tests)… [and] decrease child marriages, etc.“
Clearly, they do good. Yet the donor suddenly wanted proof that the effects of this long-term aid had improved national GDP rather than just the nutritional and learning outcomes of assisted individuals, otherwise, the project would be canceled. Letters from the impoverished country– including the country’s president and cabinet -stating they themselves had benefitted from the program which in turn led them to great educational outcomes and leadership had some effect. Yet without lobbying from the donor country’s agriculture and food aid industry that cutting food aid exports would harm them, it would have been canceled. Did they care about the schoolchildren’s learning or only providing outlets to US agricultural surplus producers? Was 30 years too long to keep helping? How do we have these discussions as equal partners? As evaluator Zenda Offir notes regarding the SDG’s No One Left Behind, “the burden of supporting and sustaining a majority of ‘leaving no-one behind’ efforts fall inevitably on many of the poorest (low-income) countries in the Global South. The problem is that they cannot afford it, nor can they sustain it. It will therefore be unfair to hold such countries accountable for ‘leaving no-one behind’ strategies. “
This brings up the questions of sustainability and accountability ‘to whom’ and ‘for how long’? You may have other questions, including ‘why’ and ‘how we know, which I look forward to addressing, but for now, these two are the focus of this blog. While 65% of Americans favor foreign aid, believing we spend up to 25% of our GDP abroad, the US spends just over 1%. Most US aid goes to the fragile and geopolitically strategic: “More than two hundred countries receive U.S. aid. It disproportionately goes to a few, however, with the top five all receiving over $1 billion per year as of 2016: Iraq ($5.3 billion), Afghanistan ($5.1 billion), Israel ($3.1 billion), Egypt ($1.2 billion), and Jordan ($1.2 billion).” In Europe, only 3 countries met the OECD goal of giving 0.7% of GNI: Norway, Sweden, and Denmark while the Czech Republic was at 0.13%, just below the USA 0.16%(2019). A fascinating measure of ‘commitment to development” (CDI) looking at the ‘quality of aid’ found that in the Czech Republic aid performance was very poor: “Adding up both quality-adjusted aid (95.7 million USD) and quality-adjusted charity induced by public policy (1.1 million USD), we arrive at 96.8 million USD for 2009 which amounts to 0.054% of GNI. Translating the percentage onto the standardised CDI scale the Czech Republic… [has] the third least favorable aid policy towards developing countries among DAC countries… [and] Aid allocation is not primarily focused on low-income countries” which is in part explained by the recent shift from aid recipient to aid donor (Syrovatka and Krylova 2012). Even, worse now, donors and international NGOs distribute aid (especially what is left for the poor countries of the world) that is annually allocated, but as the pandemic has led more spending to be domestic, aid to the poorest has decreased among almost all donor countries, bad news as Covid-economic downturns continue and climate change ramps up.
So who is accountable to the poor whom we help? Peter comments on that from the perspective of the CEO of a local Kenyan NGO targeting 15,000 farmers.
Peter
I have over forty years of experience in development; 8 years lead in Community Initiated (Harambee) High schools, 35 years with Catholic Relief Services in Emergency Relief, Sustainable Development and Justice and Peace Programs, and currently 6 years with the (Kenyan) county-devolved sustainable development. I am the Founder and CEO of Decent Living Institute of Organic Farming promoting avocado farming, aquaculture, and apiculture for improved decent livelihoods. My early life as a young boy makes me a living witness of a life in deep poverty, which the New York Times featured.
The question ‘who is accountable for sustained development’ and ‘for how long’ has an assumption that it is possible to attain sustainable development without the continued involvement of those who suffer. I don’t think so. Sustained development occurs as a process to a transformed situation from abject poverty, a condition of want without the capacity to satisfy even the most basic needs, a position of lacking continuously leading to untold suffering and living in dehumanized conditions for the sufferer and the generations to come to the desired decency of fulfilled living. Living as a pauper in my first 30 years of life, having been born in a paupers’ family, I accepted the conditions of poverty and hunger as a way of life. After all, you know nothing better and when you see wealth around you, it is meant for the lucky few, and not for you. The situation limits the poor to survival conditions, eating from hand to mouth and everything is left to luck.
Aid to the poor would make sense if it is used as a catalyst to motivate and enable the poor depart from the circle of poverty (the poor giving birth to more poor) and is able to sustain the conditions of being above the poverty line of US$2 a day. Such aid would enable them to have enough to take care of their daily basic needs and create wealth without falling back below the poverty line repeatedly, for generations to come. The impact measure for aid should therefore be participatory learning from and measuring the extent to which success is sustained documenting representative success stories by participants who have left the circle of poverty sustainably. Such would include ‘in the past I couldn’t to find enough to eat occasionally slept without food and now my family has no idea of how it feels like to be hungry.
Unfortunately, the manner of delivering aid is seen as pure luck by the targeted poor for it comes without involving the poor as to strategically plan long-term impact that they can sustain. The aid donors and implementing agencies will target a given county, while the identification of targeted community cluster location for aid will be influenced by either by powerful persons from the locality or larger numbers in the public participation, so those with greater authority or louder voice will take the day. The decision on who will participate in the project finally will be determined by the same criteria and not the poverty levels. One example is the aid fund for COVID-19 response in Kenya which was distributed to the well connected to persons of authority and not to those who championed the control of the coronavirus. A decision was made at the county governments to disperse one million Kenya shillings to every cluster of villages to pay the youth for engaging in communal work such as community road works, terracing a degraded land, or even constructing an earth dam and paid per piece work completed – termed employment -to cushion the youth who have lost job opportunities due to the COVID -19 effects. A million shillings in a cluster of ten villages would perhaps engage 100 young people for a week earning Kenya shillings 500 (US$5) a day or $30 a week. The rest of the money – 50% of the total or more – does not go for wages as planned but is used to cover the management of the program by the county officials. The youth will spend this money like it is good luck for it is too little to ever think of the future investments.
However, the same amount is what it costs to support a member of the self-help group and collective community-led development in our Decent Living NGO per family of a vulnerable child to grow a vegetable garden, keep six chicks and grow three Hass avocado seedlings. Further, the participants commit to support another poor family with six chicks in a years’ time. From the onset, the poor are involved in ‘planning in advance’ to help others. Their developing vision is guided by the long-term impact they hope for, such as the family economic boost that will cover the full cost of schooling, medical expenses, and family meals, clothing, and shelter for all the children including the most vulnerable. Other long-term indicators will be the percentage of poor families that are above the poverty line meeting the family basic needs sustainably.
I see the role of the aid donor as to holding the aid receiver (local government and recipient communities) to their goals of sustainable development and to account for the funds given by reaching their goals and targets that must be time-bound.The aid receivers are also responsible to account for their aid distribution to their intermediary implementing partners (often local non-profits/ NGOs) to meet their targets, goals and should track expected and measurable long-term outcomes within three years after the closure of the project. This means to deliver not only the aid funds but also through the funding the systems established or improved at the conception of the project should be accounted for during the project implementation period and will be impacting long term results transforming the community to the desired state long after the project activities.
It could be building sustainable infrastructure for long-term support to the poor. The sustained impact would then be numbers of poor that have transformed their poverty and created wealth through the developed infrastructure in an intergenerational, long-lasting way that could be measured in later years. Sadly, most of the aid givers do not see their role beyond the performance short-term outputs such as trainings given or outcomes leading to a change in farming practices, like the deliverables for the specific objectives in an agriculture project. Hence the success of the project is defined by these short-term indicators that measure outputs such as the target number reached with food aid, or even some changes in practices leading to improved yields, but once the project ends, all tracking of results end. The national stakeholders are – or should be- responsible to demonstrate how the results of the project will be assumed by the community’s self-help groups so that the impacts become intergenerational. For it is vital to see that the project does not end with the implementing agencies. It is not only short-sighted by aid donors to believe that it ends, but national stakeholders are absconding their responsibility of accountability to the long-term impacts that are related to relationships and behavior change sustainably when they do not sustain them.
The UN’s ‘Sustainable Development Goals’ are merely a dream for most poor until the individual struggling with conditions of want is able to take steps towards permanent solutions for themselves and their future generations. It takes an oppressed dreamer (the poor with empty stomachs) who believes a change is possible to demand accountability. It also takes a progressive facilitator (donors and national stakeholders) who believes in creating enabling conditions for the oppressed to succeed. Both the oppressed dreamers (project participants, local implementing NGO agencies and the progressive facilitator (donor) are accountable to the transformed conditions. For the ‘sustained development’ to occur it must be intentionally dreamed of by all parties engaged in the process of development.
I dream of “a just world where everyone is fully a participant and celebrates sustainable development for all” wrote Pope Francis in his 2015 Laudato Si encyclical. He calls for all humanity to take care of our Mother Earth and in return, she will provide for all, including addressing issues of global warming. In my world dream, I see a time when a transformation of the sufferer from distressful and oppressive conditions of living is eased by putting future dreams into action, for those who suffer with deprivation today and are thirsty for change. I wish to make reference to a story told in the bible Jesus meeting a blind beggar (Mark 10:51) shouting to Jesus for help. Jesus asked the beggar to identify what type of aid he needed. And the blind beggar’s request for the power to see was heard and his sight was fully restored, emancipating him from the bondage of begging. We are told he transformed from a beggar into a disciple of Jesus. The transformation of conditions to sustainable options starts with the bilateral donor engaging a poor government to undertake a particular development agenda that in return facilitates its citizens to enjoy sustained development.The donor government should hold the recipient government responsible and accountable of delivering sustainable options for its citizen as per the grant agreements with evaluation two to three years after the project closure.
The poor who may be targeted with the aid may seem passive, not having been involved right from the beginning of design, and may have limitations of identifying what to ask for, perhaps those with intermediary implementing INGOs may be aware of how well what is being offered can meet their needs. Setting up the appropriate structures, they may dictate and demand sustained development options for themselves and those who are suppressed in poverty. The major issue is that most often the victims of poverty are never engaged in aid’s design and only implement what is offered. The situation creates room for corrupt national governments, INGOs, and NGOs to make quick money. The donors should hold recipient countries and INGOs accountable for tangible results toward the Sustainable Development Goals indicators for every grant in aid for as long as it takes, not just reporting at the national level.
I see a world where what matters most, is how engaged those who have empty stomachs are in the development aid agenda, and how the aid is administered and accounted for themselves and the neighboring suffering households. That development is all about a sustained transformation for empty stomachs of our project participants, their immediate neighbors, their children, their husbands, and their fathers/in law and mothers /in law, their brothers/ in law and sisters/ in law. It is about a better living standard of their neighbors who lends salt and water, the generosity of their firewood friends, those neighbors who will never turn down an opportunity to offer help no matter what. If these impacts and long term outcomes are not evaluated and accounted for, those who suffer poverty will always consider projects as myths of ‘sustainable development’ and the aid provided by bilateral, multilateral donors or through INGOs/NGOs as beneficial to the lucky few, while recipient governments and participating communities and their future generations have no sustainable impact results.
Your thoughts?
[1] MQ Patton in New Directions for Evaluation “Transformation to Global Sustainability: Implications for Evaluation and Evaluators, 2019 (link inaccessible)
Reblog Ex-post Eval Week: Evaluating Peace for Sustainability by Peter Kimeu Ngui
Reblogged from https://aea365.org/blog/ex-post-eval-week-evaluating-peace-for-sustainability-by-peter-kimeu-ngui/ January 20, 2021
Peter Kimeu Ngui
Peace is with you. My name is Peter Kimeu Ngui, the Founder and CEO of Decent Living Institute of Organic Farming with extensive experience on reflective peace and partnership building in Africa.
Is there anything like a ‘just world’? Can peace be ‘sustainable’? These two closely related questions must be in mind of evaluators of peace for sustainability in a community long after the funding expires. According to Catholic Relief Services’ Justice Lens, three key conditions that must exist in the vision of a just world and used by all sustainable peace evaluators.:
(1) sustainable right relationship for all humanity and nature
(2) the transformation of injustices in society, structures and policies with
(3) fairness of rights and responsibilities for sustained peace to all creation
The process of reflective peace building evaluation, including participatory creation of sustainability indicators demands a collaborative effort by all the stakeholders. This includes victims, perpetrators, peace builders/ advocacy agents who are necessary to establish and maintain a just environment.
Step one of the process involves reflecting on eight keys principles drawn from Pope Francis’ letter that awaken the conscience and human rights standards which fit into an acronym (CORDSSSS) for easy remembrance; Common Good, Option for the Poor, Rights and Responsibilities, Dignity and Equality, Social nature, Subsidiarity, Stewardship and Solidarity. Each team reflects on:
the definition of each principle
discussing what their world would be like if the principle was fully lived
what would be sustainable peace indicators (e.g. Increased peaceful resolutions administered by the community; peaceful co-existence by people in violent conflicts; reduced rape)
what needs to be different in the world (location) they live in?
Step two of the process is scenario building using the Appreciative Inquiry method that engages the participants in the evaluation. This involves finding out what:
sustainable values, customs, and traditions
policies and structures/ institutions exist in the community that promote/ advocate, promote, establish and maintain, monitor, and evaluate for sustainable peace building
the participants are led to dreaming of a just environment in their community and what needs be improved and put in place for this scenario to be realized.
Lessons Learned:
The evaluation teams are always and immediately energized to pursue the best scenario for peacebuilding, using the resources locally available. The reflective peacebuilding process engages participants on real issues that directly affects the community, that the community is fears, worries, concerned with and they are immediately empowered with knowledge, using familiar traditional approach to resolve the violent conflict, and they self-discover that power to transform is with those engaged in the conflict.
The possibility of a just world becomes a reality amassing greater numbers of individuals to work for peace and work through these processes.
Lessons learned are self-created from local case study evaluation processes and appeal greatly to most of the community members
Sustaining projects during and after Implementation: Does gender count?
Rutere Kagendo from Ronto Research and Valuing Voices
Gender as a concept is one of the many kinds of indicators of change measured in development projects. However, like many others, its reporting in many cases is limited to the number of women and men participating in the project. Whereas participation by gender is an important variable in measuring community engagement and inclusiveness in a project, this measure is more focused at the registration level and the mere fact that women’s or men’s names appear in the registers. However, numbers only are not a clear indicator of participation and the extent to which it happens.
Gender reporting, as shown above, can be too simplified. There is also ongoing misunderstanding surrounding gender, which only separates genders and omits a gendered analysis that would help understand the relationships between men and women, boys and girls within the dynamics of a project. This can include differentiated decisions about how resources are allocated within household expenditures (e.g. women overseeing food consumption, health and children’s expenses versus men overseeing large investments into livelihoods) and division of labor (e.g. men ploughing and planting fields and women weeding and conserving crops leading to a family harvest). Gender negotiations are richer than numbers of men and women participating in projects.
As we consider the various stages of a project, starting from design, implementation to evaluation and sustainability, it is critical to assess the role of gender in each of these stages. It is obvious that women and men play important roles in projects but what is not clear is how these roles build, complement and sustain the project. The questions that come into mind after interacting with communities during project evaluations (mid, endline, impact and sustainability) are:
How do the various roles played by both men and women, and their interaction with each other shape the outcome of a project?
Does gender contribute to project sustainability?
These are important questions that need to be answered through more research and writing but there are early insights I have learned:
1. Resilience in roles during the project life cycle
Studies highlighting the roles played by both women and men in projects shed light on how gender shapes project outcomes. I once attended a dairy project launch exercise in Kenya, my home country, and was surprised by what I saw. All the men sat at the front and the women took a back seat with their young children. The women remained silent the entire time the meeting was held, and efforts to involve them were unsuccessful. I later learned that in that community, women do not share their views in the presence of men. However, I left with one question in my mind, how will this project succeed if only one party is going to be talking and making decisions? Was there no mechanism for women to share their views as well? Is it not going to become a project that benefits men only? Years later, I got an opportunity to evaluate the project. I held both men and women focused group discussions separately. I found that firstly, men may be the gatekeepers but may not necessarily have the resilience needed to push through the whole project cycle. For example in this particular project, men did not continuously stay with the project, as noted by one female group participant:
“When this project started, we were both men and women. However, men decided to pull out because they thought milk was a women’s commodity and there was no money [to be made] in it. The women got organized in groups and started selling the milk in bulk. We made money and within no time, the men came back and took over the milk business. If we women had not stayed in the project, the milk business would not have been realized although we lost it to the men”. In another dairy-related project elsewhere, the men were the cattle and milk owners so they sold all the milk. But during a season when the market was flooded with milk (milk glut), the men abandoned the milk business. The women took it up and tried to sell the milk whenever possible until the milk glut’ season passed, only for the men to get back to the milk business again.
Whether it is a good idea or not for men to have taken the milk business away from women in both of these cases is subject for discussion, but the main issue here is the role played by the women to sustain the project long enough to grow it from an idea and push it into a viable business that attracted the men back into the project. Indeed one wonders if the project would have grown if the women had not remained in the project. Valuing Voices at Cekan Consulting found the same in an all-women and highly successful sesame seed production Catholic Relief Services project in the Gambia. Women had grown the communal agribusiness from microenterprise size to being so successful that they became of interest to local banks. At that point, men in the communities began to take the project, and savings, over. Valuing Voices has documented savings and loans sustained post-project, and even scaled up by women in several countries.
Also thinking about the initial community entry meetings and seeing how unengaged and distant the women appeared, one would have dismissed their role in the project. As it turned out, women became the engine of this project. Surface impressions may not hold true.
2. Saving the project for whose benefit?
In another project, community members were encouraged and supported to start savings and lending groups to run their small businesses. The qualitative discussions in the final evaluation revealed that at the end of the project, it was perceived as more of a women’s project in spite of being targeted to men. This is because almost all the men had left the project after taking loans and failing to repay. The women, especially those whose spouses had taken the loans took over the burden of repaying the loans which saved the project.
Borrowing from these examples one is tempted to ask, what happens in those projects that target men only or where women get demoralized and leave? Do they fail at a higher rate altogether than those that include women? Arguing along these lines one would ask, what are the driving forces for men and women to remain or quit from project activities and how has this been addressed so far in project designs?
In most of the evaluation projects Ronto Research has conducted around Africa, men may take a lead in project implementation depending on their expected quick/ shorter-term gains/ expectations which determine whether they remain active or not. On the other hand, women seem to get into projects with a determined resolve to see their problems solved, no matter how long it takes. As a result, they hold onto the project activities despite the challenges that come along. It is observable though, that as they progress and begin to take hold and show signs of success, the men get attracted back.
In most of the project evaluations I have participated in, there are always few men who are loyal and remain part and parcel of the project. Pivotally, we also need to understand the role of these few men who have remained in these projects with the women the whole time. This can be either because they hold positions or just because they believed in the projects and they support the women with ideas or just their mere presence in the project activities. These men seem to play a critical role in encouraging the women to push on. Their role cannot be ignored and there is need to understand them better in terms of their opinions. Why do they remain with the women even when there seems to be few benefits forthcoming to them, whereas their fellow men pull out? In an interview with women in Tanzania on an agricultural project, some women felt that the few men who had been left in their group were very instrumental in their achievements by giving the project an identity in as far as it being a community and not a ‘women-only’ project was concerned.
3.Gender- Informing evaluation
Thirdly, while conducting evaluation interviews, gender counts. Typically, women and men are asked about results in gender-specific groups. At the household level, the preferred interviewee is the household head. In most cases, they would be the direct ‘beneficiaries’ (participants) of the project information on behalf of their households. Due to the cultural set-ups in much of Africa, in married families, men are automatically the household head, which would mean they would be the target interviewees. However, our experience during interviews either through Focus Group Discussions or on a one-on-one situation is that consulting with women as well as considering differentiated gender roles and gender relations are key missing ingredients. For often during interviews, the men end up involving their spouses for details about projects, in part because men might not consistently follow up the project activities and therefore might not have all the details.
There is also a possibility that although it is the men who are registered as participants, in actual sense it is the women who participate and therefore have the detailed information about some issues being discussed, so it is important to check who is the actual ‘information bank’ (and day-to-day participant) of the household for the project. This issue needs to be explored in-depth both in regular project evaluation and more needs to be learned about how this can be used to inform sustainability studies. Asking only men or only women may limit evaluative learning.
4. Sustaining results – which gender is best?
What can we learn from the projects that have been completed so far years after they close out? When projects come to an end, communities are left to experience and foster the project impacts on their own. They learn a lot on their own post-project and this could be an important source of information on sustainability, but what mechanisms can be put in place for these ‘information banks’ to share the information long after projects end? They may be the only key informants that remain (as partners may have long gone onto other projects far away); before the knowledge they hold that can be eroded with time. It would be interesting to compare the information held by men and women about outcomes and impacts being sustained, stopped or new ones emerging. The question on information especially related to project sustainability is going to be very critical considering that most of this information has to be sought from community members post-project period.
Does gender play a role in holding a project together/implementation?
What role does gender play in project sustainability?
Heading up Food Security for Catholic Relief Services (CRS) was my first international development job in 1995-1999 and I have watched this organization grow in its commitment to program quality and learning/ knowledge management ever since. At the time I oversaw 17 of of CRS' USAID/ Food for Peace (FFP) programs. So I was delighted that not only has CRS done an ex-post evaluation and used the findings for programming (e.g. the effectiveness of investing in a particular sector—for example, the importance of supporting girls’ education within a food security program) and also for advocacy (e.g. evaluation lessons from Rwandan peace-building projects seven years after the genocide informed CRS’ evolving approach to peace and justice strategies), but I get to celebrate FFP learning too. In addition to having consulted to USAID/PPL (Policy, Planning and Learning) and the FANTA project, all featured below, I went to Tufts University’s Fletcher School. Super to see great organizations learning about sustainability!
CRS’ 2007 project package guidance for implementation and guidance (ProPackII), described ex-post evaluation/sustainable impact evaluation’s aim “to determine which project interventions have been continued by project participants on their own [which] may contribute to future program design…. it is fair to say that NGOs rarely evaluate what remains following the withdrawal of project funding [which] is unfortunate [as] important lessons can be generated regarding factors that help to ensure project sustainability.”
A 2004 Catholic Relief Services excellent ex-post evaluation in Ethiopia was featured: “Looking at the past for better programming:dap I Ex-Post Assessment Report”. It assessed sustainability of Agriculture Natural Resource Management, and Food-Assisted Child Survival/Community Based Health Care programming, done as an internal evaluation by CRS staff and partners with document review, partner, government and community interviews. Results were mixed.
Some activities generated enough food and income that households could eat throughout the year and have some savings, making them more resilient against drought
Almost 100% of cropland bunding and irrigation practices for improved crop production were still being applied and buffered them during a subsequent drought
Health practices had also continued (e.g. trained traditional birth attendants had continued to provide services with high levels of enthusiasm and commitment, and increased levels of health care-seeking behaviors existed).
However, many other benefits and services had severely deteriorated:
Nearly all water committees had dissolved, fee collection was irregular or had been discontinued
Many water schemes were not operational
The centrally managed [tree] nurseries had been abandoned (given the existing management capacity of communities and government).
CRS/Ethiopia and its partners came to see that:
“The potable water strategy had over-focused on the technical aspects (“hardware”) while not paying enough attention to the community organizing dimensions and support by existing government services (“software”).
Even limited post-project follow-up by partners and government staff might have gone a long way towards mitigating the deterioration of project benefits and services.
What was terrific was that they “went on to use these findings and lessons learned from this ex-post evaluation to inform the design of similar projects in Ethiopia… while also raising awareness of these issues among partner staff”. The ex-post recommendedincreased planning for sustainability, setting up village management for post-project and incentive maintenance. Great learning, yet we have found few ex-posts at CRS or elsewhere. Our industry needs to explore issues such as those the evaluators posed: Was the lack of sustainability due to technical, institutional or financial faults in the programming? In other words, was the lack of self-sustainability due to the design/ aim of the activity itself or how it was implemented?
In 2013, USAID’s Food For Peace commissioned fascinating research on Exit Strategies. Tufts University went to Bolivia, Honduras, India and Kenya which were phasing out of Title II food aid to look at how to “ensure that the benefits of interventions are sustained after they end, [as] there is little rigorous evidence on the effectiveness of different types of exit strategies used in USAID Office of Food for Peace Title II development food aid prog
rams.” The research is to “assess the extent to which the programs’ impacts were maintained or improved and to help understand factors of success or failure in the specific exit strategies that were used.” They have made the important discernment that the effectiveness of Title II programs depends on both short-term impact and long-term sustainability.
The FANTA project (contractor) made the following preliminary results available:
Impact assessment at exit does not consistently predict sustained impact two years later…. It can be misleading.
Many activities, practices, and impacts across sectors declined over the two years after exit. These declines are related to inadequate design and implementation of sustainability strategies and exit processes.
There are specific ways to increase the likelihood of sustainability: Sustaining service provision and beneficiary utilization of services and practices depends on three critical factors: Resources, Technical and Management Capacity, Motivation
Withdrawal of free food rations or any other free input (as incentive) jeopardizes sustainability without consideration of substitute incentives. For instance,
Withdrawal of food was a disincentive for participation in and provision of [child] growth monitoring…. Resources and health system linkages are needed to sustain health activities
Motivation, capacity and resources are all needed to maintain water systems
Agriculture and Natural Resource Management suffered greatly when resource incentives disappeared
Their main recommendations are that sustainability should be built into the design from the beginning, program cycles are longer and exit is gradual.
CRS found the same issue of incentives as a barrier, as they did technical and (institutional) capacity/ motivation/ management issues. We have much to learn… at least we’ve started Valuing Voices and asking… and eventually designing for sustainability!
So how are we to get there? A Sustainable Brands Conference this year gets us there through being clear about their own consumption, and USAID is no different. USAID Forward is putting their money where their keyboards are (so to speak), toward more sustainable local delivery by directing a huge 30 percent of its funding to “local solutions” through procurement in coming years. This framework is to “support the ‘new model of development’ that USAID Administrator Rajiv Shah has touted, which entails a shift away from hiring U.S.-based development contractors and NGOs to implement projects, and toward channeling money through host-country governments and local organizations to build their capacity to do the work themselves and sustain programs after funding dries up. I, and others celebrate the investments this will enable local firms to make in their own capacity, in leading development!
Of course all sorts of safeguards are needed, and ideally US firms would be providing capacity development, but shouldn’t we have been doing this all along, to move toward transferring ‘development’ to the countries themselves?
Also vital to sustainable development is learning from what works and doing more of it. USAID is finally planning to incorporate more ex-post evaluations into its toolkit of evaluating sustainability! Two weeks ago, PPL/LER shared their great new policy document- “Local systems: A framework for supporting sustained development” on how they can better incorporate local systems thinking into policy as well as DIME (Design, Implementation, Monitoring and Evaluation). Industry insider DevEx tells us "even though the agency plans to use ex-post evaluations to measure whether development projects are successful or not, these evaluations will not focus on “specific contractor performance” but instead consider the “types of approaches that contribute to more sustainable outcomes…to inform USAID’s country strategies and project design." While PVO implementing partners will not [yet?] be required to do ex-post evaluations as part of their projects, having this door cracked open is excitingly opening. Notably, it is a ‘back to the future’ moment, as 30 years ago USAID led the development world in post-project evaluations, yet in the last 24 years has done none (or at least not published any) except for the Food for Peace retrospective below, as I found in our Valuing Voices research of USAID's Development Experience Clearinghouse.
There is far more to watch. In our view, the whole development industry needs to grapple with the perceived barrier that funding ends with projects (note: a trust could be set up to document post-project impact 1, 3, 5 years later and results retained, much as 3ie does now for impact evaluations) and the view that one cannot discern attributable project impact with a time-lag of several years. Yet even the General Accounting Office is asking for longitudinal data; they reviewed USAID’s document and wants to see clear measures of success at Mission and HQ level by different indicators of local institutional sustainability and impact four years on.
Why should we care? As Chelsea Clinton of the Clinton Global Initiative puts it, "you can't measure everything, but you can measure almost everything through quantitative or qualitative means, so that we know what we're disproportionately good at. And, candidly, what we're not so good at, so we can stop doing that.
Yes! Development should be about doing more of what works, sustainably, and less of what doesn’t. USAID’s Local Systems Framework found the best could also be free, as in this one Food For Peace evaluation shows:
Returning to Chelsea Clinton, I’ll conclude by stating something obvious. She "wants to see some evidence of why we're making decisions, as opposed to the anecdotes” which is what getting post-project evaluation data from our true clients, our participants, is all about. Clinton says this will transform CGI into a smart, accountable, and sustainable support system for philanthropic disrupters around the world. USAID is radical for me, today, with their Local Systems investments… my neighborhood disrupter.
Are you such a disrupter too? Who else is one whom we can celebrate together?
Blog Categories
Jindra Cekan, Ph.D. has used participatory methods for 30 years to connect with participants, ranging from villagers in Africa, Central/ Latin America and the Balkans to policy makers and Ministers around the world for her international clients. Their voices have informed the new Sustained and Emerging Impacts Evaluation, other M&E, stakeholder analysis, strategic planning, knowledge management and organizational learning.
If you don’t find what you are looking for via the search, categories, or posts above, you can go to the Blog page, scroll to the bottom, and click “previous posts” to go through all of the posts (newest–>oldest).