Who is accountable for the ‘sustained development’ of those who suffer, and for how long?
Some of the rarely discussed myths of ‘sustainable development is that the aid we donors and implementers bring will help everyone, and that recipient governments can take over once donors leave and that what we left when the project ended was sustainable. The fact that our ‘global development aid’ helps a small fraction of all those who have equally ‘worthy’ needs in the countries we target is unspoken, as is the fact that these projects aren’t often built to sustainably withstand shocks such as climate changes bearing down now. As evaluator Michael Quinn Patton writes, “Effective programs… create islands of protected effectiveness in a sea of need and suffering… [we must] assess sustainability over time for Adaptive Resilience.”
More often, countries need to be: a) so poor, b) so ‘fragile’, or c) so geopolitically important to warrant our aid. The fact that most USAID and EU bilateral funding (not necessarily multilateral funding to IMF, World Bank, African or Asian or other Development Banks) is focused on fragile/ war-torn or the strategically important countries edges out what is left for the impoverished people of the world. Most of our taxpayers have little idea of this and believe it is needs-based. So, many of us assume that: d) our mostly short-term aid is either only the brief support that they need (the ‘shot in the arm school’) as might be true of emergency or humanitarian aid post-emergencies or e) the results will be so excellent as to spontaneously spread (scale) ‘forever’ that no more aid will be needed or f) that national governments – not donors- need to carry the accountability for sustained improvements forward. Yet those stakeholders in poor or fragile countries (governments, non-profit NGOs) often have the weakest capacities to sustain results. Rutere Kagendo, a fellow Kenyan on the Valuing Voices team, wrote a moving blog about who ends up tasked with sustaining project activities and results: communities, especially women.
This is in spite of the fact that ex-post project evaluations done 1+ to 30 years after completion who asked participants and partners about sustainability, show ‘mixed’ results, namely that results fall off as early as 2 years after closure, by between 10-100%. Rarely do ex-posts have results improve or be as sustained as we assume. Some project activities do show somewhat lasting results, ranging from those that provide credit to those generating agriculture. Mostly lackluster results are because global development projects aim for success by closure over sustainability over decades by exiting better with accountability to our participants and partners over time. We do not design what can be locally sustained. Imagine how much would be saved if we did!
Jindra Čekan/ova of the USA/ Czech Republic and Peter Kimeu of Kenya offer our perspectives about who is accountable for aid, and for how long.
I have worked for international NGOs (e.g. Catholic Relief Services, the international division of the American Red Cross, large INGOs such as CARE, World Vision, Lutheran World Relief, and others), including bilaterals (USAID) and foundations (the Bill & Melinda Gates Foundation, Aga Khan Foundation) for over 30 years. Most of our projects strove to fulfill objectives of the grants and were successful. Yet rarely did we ask ‘for how long?’ At one point, I worked for a huge International NGO with a program that had been feeding 50,000 West African children breakfast for 30 years. This was part of a bilateral aid education support + ‘safety net’ program. Many studies, including this terrific one from the UN’s World Food Program, show that such school feeding improves “school participation (enrolment, attendance, completion) and learning (scores on cognitive, language and mathematics tests)… [and] decrease child marriages, etc.“
Clearly, they do good. Yet the donor suddenly wanted proof that the effects of this long-term aid had improved national GDP rather than just the nutritional and learning outcomes of assisted individuals, otherwise, the project would be canceled. Letters from the impoverished country– including the country’s president and cabinet -stating they themselves had benefitted from the program which in turn led them to great educational outcomes and leadership had some effect. Yet without lobbying from the donor country’s agriculture and food aid industry that cutting food aid exports would harm them, it would have been canceled. Did they care about the schoolchildren’s learning or only providing outlets to US agricultural surplus producers? Was 30 years too long to keep helping? How do we have these discussions as equal partners? As evaluator Zenda Offir notes regarding the SDG’s No One Left Behind, “the burden of supporting and sustaining a majority of ‘leaving no-one behind’ efforts fall inevitably on many of the poorest (low-income) countries in the Global South. The problem is that they cannot afford it, nor can they sustain it. It will therefore be unfair to hold such countries accountable for ‘leaving no-one behind’ strategies. “
This brings up the questions of sustainability and accountability ‘to whom’ and ‘for how long’? You may have other questions, including ‘why’ and ‘how we know, which I look forward to addressing, but for now, these two are the focus of this blog. While 65% of Americans favor foreign aid, believing we spend up to 25% of our GDP abroad, the US spends just over 1%. Most US aid goes to the fragile and geopolitically strategic: “More than two hundred countries receive U.S. aid. It disproportionately goes to a few, however, with the top five all receiving over $1 billion per year as of 2016: Iraq ($5.3 billion), Afghanistan ($5.1 billion), Israel ($3.1 billion), Egypt ($1.2 billion), and Jordan ($1.2 billion).” In Europe, only 3 countries met the OECD goal of giving 0.7% of GNI: Norway, Sweden, and Denmark while the Czech Republic was at 0.13%, just below the USA 0.16%(2019). A fascinating measure of ‘commitment to development” (CDI) looking at the ‘quality of aid’ found that in the Czech Republic aid performance was very poor: “Adding up both quality-adjusted aid (95.7 million USD) and quality-adjusted charity induced by public policy (1.1 million USD), we arrive at 96.8 million USD for 2009 which amounts to 0.054% of GNI. Translating the percentage onto the standardised CDI scale the Czech Republic… [has] the third least favorable aid policy towards developing countries among DAC countries… [and] Aid allocation is not primarily focused on low-income countries” which is in part explained by the recent shift from aid recipient to aid donor (Syrovatka and Krylova 2012). Even, worse now, donors and international NGOs distribute aid (especially what is left for the poor countries of the world) that is annually allocated, but as the pandemic has led more spending to be domestic, aid to the poorest has decreased among almost all donor countries, bad news as Covid-economic downturns continue and climate change ramps up.
Is there proof there is no more need for aid to places or people? We do not know as we return to evaluate the sustainability after projects close far less than 1% of the time. Mostly this is because we assume that the recipient governments have ‘taken over’, that there is funding from elsewhere, or that the communities and organizations helped have become as resilient as to keep up the good work themselves. Yet the ex-post evaluation data does not bear this out. Few such evaluations are done or done well and many assumptions of positive trajectories are unproven. Donors and INGOs want to help, must leave after money is spent, and assume the best. Local participants implement the projects but they do not design or lead their implementation, which limits continuation after donor support exits.
So who is accountable to the poor whom we help? Peter comments on that from the perspective of the CEO of a local Kenyan NGO targeting 15,000 farmers.
I have over forty years of experience in development; 8 years lead in Community Initiated (Harambee) High schools, 35 years with Catholic Relief Services in Emergency Relief, Sustainable Development and Justice and Peace Programs, and currently 6 years with the (Kenyan) county-devolved sustainable development. I am the Founder and CEO of Decent Living Institute of Organic Farming promoting avocado farming, aquaculture, and apiculture for improved decent livelihoods. My early life as a young boy makes me a living witness of a life in deep poverty, which the New York Times featured.
The question ‘who is accountable for sustained development’ and ‘for how long’ has an assumption that it is possible to attain sustainable development without the continued involvement of those who suffer. I don’t think so. Sustained development occurs as a process to a transformed situation from abject poverty, a condition of want without the capacity to satisfy even the most basic needs, a position of lacking continuously leading to untold suffering and living in dehumanized conditions for the sufferer and the generations to come to the desired decency of fulfilled living. Living as a pauper in my first 30 years of life, having been born in a paupers’ family, I accepted the conditions of poverty and hunger as a way of life. After all, you know nothing better and when you see wealth around you, it is meant for the lucky few, and not for you. The situation limits the poor to survival conditions, eating from hand to mouth and everything is left to luck.
Aid to the poor would make sense if it is used as a catalyst to motivate and enable the poor depart from the circle of poverty (the poor giving birth to more poor) and is able to sustain the conditions of being above the poverty line of US$2 a day. Such aid would enable them to have enough to take care of their daily basic needs and create wealth without falling back below the poverty line repeatedly, for generations to come. The impact measure for aid should therefore be participatory learning from and measuring the extent to which success is sustained documenting representative success stories by participants who have left the circle of poverty sustainably. Such would include ‘in the past I couldn’t to find enough to eat occasionally slept without food and now my family has no idea of how it feels like to be hungry.
Unfortunately, the manner of delivering aid is seen as pure luck by the targeted poor for it comes without involving the poor as to strategically plan long-term impact that they can sustain. The aid donors and implementing agencies will target a given county, while the identification of targeted community cluster location for aid will be influenced by either by powerful persons from the locality or larger numbers in the public participation, so those with greater authority or louder voice will take the day. The decision on who will participate in the project finally will be determined by the same criteria and not the poverty levels. One example is the aid fund for COVID-19 response in Kenya which was distributed to the well connected to persons of authority and not to those who championed the control of the coronavirus. A decision was made at the county governments to disperse one million Kenya shillings to every cluster of villages to pay the youth for engaging in communal work such as community road works, terracing a degraded land, or even constructing an earth dam and paid per piece work completed – termed employment -to cushion the youth who have lost job opportunities due to the COVID -19 effects. A million shillings in a cluster of ten villages would perhaps engage 100 young people for a week earning Kenya shillings 500 (US$5) a day or $30 a week. The rest of the money – 50% of the total or more – does not go for wages as planned but is used to cover the management of the program by the county officials. The youth will spend this money like it is good luck for it is too little to ever think of the future investments.
However, the same amount is what it costs to support a member of the self-help group and collective community-led development in our Decent Living NGO per family of a vulnerable child to grow a vegetable garden, keep six chicks and grow three Hass avocado seedlings. Further, the participants commit to support another poor family with six chicks in a years’ time. From the onset, the poor are involved in ‘planning in advance’ to help others. Their developing vision is guided by the long-term impact they hope for, such as the family economic boost that will cover the full cost of schooling, medical expenses, and family meals, clothing, and shelter for all the children including the most vulnerable. Other long-term indicators will be the percentage of poor families that are above the poverty line meeting the family basic needs sustainably.
I see the role of the aid donor as to holding the aid receiver (local government and recipient communities) to their goals of sustainable development and to account for the funds given by reaching their goals and targets that must be time-bound. The aid receivers are also responsible to account for their aid distribution to their intermediary implementing partners (often local non-profits/ NGOs) to meet their targets, goals and should track expected and measurable long-term outcomes within three years after the closure of the project. This means to deliver not only the aid funds but also through the funding the systems established or improved at the conception of the project should be accounted for during the project implementation period and will be impacting long term results transforming the community to the desired state long after the project activities.
It could be building sustainable infrastructure for long-term support to the poor. The sustained impact would then be numbers of poor that have transformed their poverty and created wealth through the developed infrastructure in an intergenerational, long-lasting way that could be measured in later years. Sadly, most of the aid givers do not see their role beyond the performance short-term outputs such as trainings given or outcomes leading to a change in farming practices, like the deliverables for the specific objectives in an agriculture project. Hence the success of the project is defined by these short-term indicators that measure outputs such as the target number reached with food aid, or even some changes in practices leading to improved yields, but once the project ends, all tracking of results end. The national stakeholders are – or should be- responsible to demonstrate how the results of the project will be assumed by the community’s self-help groups so that the impacts become intergenerational. For it is vital to see that the project does not end with the implementing agencies. It is not only short-sighted by aid donors to believe that it ends, but national stakeholders are absconding their responsibility of accountability to the long-term impacts that are related to relationships and behavior change sustainably when they do not sustain them.
The UN’s ‘Sustainable Development Goals’ are merely a dream for most poor until the individual struggling with conditions of want is able to take steps towards permanent solutions for themselves and their future generations. It takes an oppressed dreamer (the poor with empty stomachs) who believes a change is possible to demand accountability. It also takes a progressive facilitator (donors and national stakeholders) who believes in creating enabling conditions for the oppressed to succeed. Both the oppressed dreamers (project participants, local implementing NGO agencies and the progressive facilitator (donor) are accountable to the transformed conditions. For the ‘sustained development’ to occur it must be intentionally dreamed of by all parties engaged in the process of development.
I dream of “a just world where everyone is fully a participant and celebrates sustainable development for all” wrote Pope Francis in his 2015 Laudato Si encyclical. He calls for all humanity to take care of our Mother Earth and in return, she will provide for all, including addressing issues of global warming. In my world dream, I see a time when a transformation of the sufferer from distressful and oppressive conditions of living is eased by putting future dreams into action, for those who suffer with deprivation today and are thirsty for change. I wish to make reference to a story told in the bible Jesus meeting a blind beggar (Mark 10:51) shouting to Jesus for help. Jesus asked the beggar to identify what type of aid he needed. And the blind beggar’s request for the power to see was heard and his sight was fully restored, emancipating him from the bondage of begging. We are told he transformed from a beggar into a disciple of Jesus. The transformation of conditions to sustainable options starts with the bilateral donor engaging a poor government to undertake a particular development agenda that in return facilitates its citizens to enjoy sustained development. The donor government should hold the recipient government responsible and accountable of delivering sustainable options for its citizen as per the grant agreements with evaluation two to three years after the project closure.
The poor who may be targeted with the aid may seem passive, not having been involved right from the beginning of design, and may have limitations of identifying what to ask for, perhaps those with intermediary implementing INGOs may be aware of how well what is being offered can meet their needs. Setting up the appropriate structures, they may dictate and demand sustained development options for themselves and those who are suppressed in poverty. The major issue is that most often the victims of poverty are never engaged in aid’s design and only implement what is offered. The situation creates room for corrupt national governments, INGOs, and NGOs to make quick money. The donors should hold recipient countries and INGOs accountable for tangible results toward the Sustainable Development Goals indicators for every grant in aid for as long as it takes, not just reporting at the national level.
I see a world where what matters most, is how engaged those who have empty stomachs are in the development aid agenda, and how the aid is administered and accounted for themselves and the neighboring suffering households. That development is all about a sustained transformation for empty stomachs of our project participants, their immediate neighbors, their children, their husbands, and their fathers/in law and mothers /in law, their brothers/ in law and sisters/ in law. It is about a better living standard of their neighbors who lends salt and water, the generosity of their firewood friends, those neighbors who will never turn down an opportunity to offer help no matter what. If these impacts and long term outcomes are not evaluated and accounted for, those who suffer poverty will always consider projects as myths of ‘sustainable development’ and the aid provided by bilateral, multilateral donors or through INGOs/NGOs as beneficial to the lucky few, while recipient governments and participating communities and their future generations have no sustainable impact results.
 MQ Patton in New Directions for Evaluation “Transformation to Global Sustainability: Implications for Evaluation and Evaluators, 2019 (link inaccessible)
Setting a higher bar: Sustained Impacts are about All of us
Global development aid has a problem which may already affect impact investing as well.
It is that we think it’s really all about us (individuals, wealthy donors and INGO implementers) not all of us (you, me, and project participants, their partners and governments). It’s also about us for a short time.
All too often, the measurable results we in global development aid and Corporate Social Responsibility (CSR) funded projects that last 1-5 years track and report data for two reasons:
1) Donors have Compliance for grantees to meet (money spent, not lost, and results met by fixed deadlines of 1-5 years – look at some of the European Commission Contracting rules) and
2) Fund recipients and the participants they serve are accountable to ‘our’ donors and implementers who take what happened through their philanthropic grants as ‘their’ results.
Both can skew how sustainably we get to create impacts. An example of such strictures on sustainability from USAID. As respected CGDev Elliot and Dunning researchers found in 2016 when assessing the ‘US Feed the Future Initiative: A New Approach to Food Security?‘ the $10.15 billion leveraged $20 billion from other funders for disbursement over three years (2013-16). “We are concerned that pressure to demonstrate results in the short term may undermine efforts to ensure any impact is sustainable…. Unfortunately, the pressure to show immediate results can encourage pursuit of agricultural investments unlikely to be sustained. For example, a common response to low productivity is to subsidize or facilitate access to improved inputs… it can deliver a quick payoff… however, if the subsidies become too expensive and are eliminated or reduced, fertilizer use and yields often fall…..
With so much focus on reporting early and often about the progress in implementing the initiative, there is a risk that it increases the pressure to disburse quickly and in ways that may not produce sustainable results. For example, for 2014, Feed the Future reports that nearly 7 million farmers applied “improved technologies or management practices as a result of U.S. Government assistance,” but only 1,300 received “long-term agricultural sector productivity.” Are the millions of others that are using improved inputs or management practices because of subsidies likely to have these practices sustained? And how likely are they to continue using improved practices once the project ends?”
3) Impact investors stick to the same two paths-to-results and add a new objective: market-competitive financial returns. They also need to show short-term results to their investors, albeit with social, environmental and governance results like non-profits (future blog).
4) Altruists create things we want ‘beneficiaries’ (our participants) to have. For instance a plethora of apps for refugees cropped up in recent years, over 5,000 it is estimated, which can be appropriate, nor not so helpful. Much like #2 above, ‘we’re’ helping ‘them’ but again, it seems to be a ‘give a man a fish’… and my fish is cool sort of solution… but do our participants want/ need this?
How often is our work-for-change mostly about us/by us/ for us... when ideally it is mostly about ‘them’ (OK, given human self-interest, shouldn’t changes we want at least be about all of us?).
All too often we want to be the solution but really, our ‘grassroots’ clients who are our true customers need to generate their own solution. Best if we listen and we design for long-term sustainability together?
As the Brilliant Sidekick Manifesto stated in two of its ten steps:
a) “I will step out of the spotlight: Sustainable solutions to poverty come from within are bottom-up, and flow from local leaders who are taking the risks of holding their politicians accountable and challenging the status quo.”
b) “I will read “To Hell with Good Intentions” again and again: Politicians, celebrities and billionaire philanthropists will tell me that I can be a hero. I cannot. The poor are not powerless or waiting to be saved. Illich will check my delusions of grandeur.”
We have examples of where we have stepped away and participants had to fend for themselves. At Valuing Voices, we’ve done post project-exit evaluations 2-15 years afterward. What did participants value so much that they sustained it themselves (all about them, literally)? These Sustained and Emerging Impacts Evaluations (SEIE) also give us indications of Sustained ROI (Sustained Return on Investment (SusROI) is a key missing metric. As respected evaluator Ricardo Wilson-Grau said in an email, “I think calculating cost-effectiveness of an intervention’s outcomes would be a wonderful challenge for a financial officer searching for new challenges — if not a Nobel prize in economics!”)
Most of these evaluations are pretty bad news mixed with some good news about what folks could sustain after we left, couldn’t and why not. (These are the ones folks expect to have great results, otherwise they wouldn’t share them!) While most clients are understandably interested in what of ‘theirs’ was still standing, and it was interesting disentangling where the results were attributable by implementation or design or partnership flaws or something else, what was mesmerizing was what came from ‘them’.
The key is looking beyond ‘unexpected’ results to look at emerging impacts that are about ‘them’ (aka what we didn’t expect that was a direct result of our project, e.g. spare parts were no longer available to fix the water well pump once we left or a drought rehabilitation water project that decreased violence against women), to what emerging results are attributable not to use but only to our participants and partners who took over after our projects closed. One example is a Nepalese project ended yet the credit groups of empowered women spawned groups of support groups for battered women. Another is a child maternal health project changed how it worked as women reverted to birthing at home after NGOs left; community leaders punished both parents with incarceration in the health clinic for a week if they didn’t given birth there (wow did that work to sustain behavior change of both parents!).
Many of us at Valuing Voices are shocked that funders don’t seem that interested in this, as this is where they not only take over (viz picture, sustaining the project themselves), but they are making it theirs, not ours. Imagine assuming the point of development is to BE SUSTAINABLE.
Source: Community Life Competence
Our participants and national stakeholder partners are our true clients, yet… Feedback Labs tell us Americans alone gave $358 billion to charities (equivalent to the 2014 GDP of 20 countries) – in 2014 but how much of this was determined by what ‘beneficiaries’ want? Josh Woodard, a development expert, suggests a vouchers approach where our true clients, our participants, who would “purchase services from those competing organizations… [such an] approach to development would enable us all to see what services people actually value and want. And when we asked ourselves what our clients want, we would really mean the individuals in the communities we are in the business of working with and serving. Otherwise we’d be out of business pretty quickly.”
This opens the door to client feedback – imagine if participants could use social media to rate the sustained impacts on them of the projects they benefited from? A customer support expert wrote in Forbes, “Today, every customer has, or feels she has, a vote in how companies do business and treat customers. This is part of a new set of expectations among customers today that will only grow ... you can’t control product ratings, product discussions or much else in the way of reviews, except by providing the best customer experience possible and by being proactive in responding to negative trends that come to the surface in your reviews and ratings stronger.”
So how well are we working with our participants for ‘development’ to be about them?
What do you think?
Maximizing what we've got… Time is now!
We had a stirring conversation here in D.C. with someone very knowledgeable about sustainability; this person is a strong proponent of local ownership of all development. They also said vehemently, why evaluate the sustainability of projects after closeout; we all know what that will show! What was implied is that our system of international development and aid is so flawed, so broken, that the inevitable result of not focusing on local ownership as the fundamental basis for our work means Nothing. Will. Be. Left…. All. Is. Lost.
We disagree. Our development industry does some good, some bad, and is ever-changing (albeit slowly). Billions of dollars each year are spent trying to improve people's lives and livelihoods around the world, and we've seen great good be done. While. We. Remain.
We know far, far less about what remains after our projects end because less than 1% of the time we return post-project (ex-post) to evaluate anything.
Our problem with time begins with fixed timelines within projects that say we have 1, 3, 5 years to get to success. They work with participants and partners who need to make substantial changes to how they use their resources and beliefs over a relatively short time of a few months to a few years. We expect immediate results from them, changing how they farm (use new seeds, new methods, new ways of interacting with markets) and save money (learn new concepts of profit and interest, repayment and re-lending), and improve their health and that of their families (get prenatal exams, vaccinate your children, exclusively breastfeed without adding water or tea). Everyone in our projects is 'on the clock' from the donor and implementer to partners and participants. This clock ticks down irrevocably as project closeout looms, promised-successes-to-donors at hand or a mirage in the distance. We assume sustained results.
How many of us have ever gone on a diet? How many have learned a new language? How many of us have transferred jobs and had to learn new skills on the job? How quickly have we managed to do all that successfully, all at once?! Probably many. How many of you have had to do this on a fixed timeline? Were you successful when there was a limited, fixed time and you did not set your own pace?
It takes time to implement projects well enough to ensure that most participants ‘got it’, not just the 'early adopters'. It takes time to hand over projects so well that our partners and participants are ready to take over at least some of what we worked so hard to transfer. It took leadership and staff two years in the very successful participatory USAID/ Food For Peace food security project by CRS Niger that was a continuation of similar programming for 15 years. It also takes time to pass for conditions to be ready for our return, to isolate what people could self-sustain from what the project supplied, to learn what was so well designed and implemented during projects that to 'took root' in people's lives, that they have made it their own. We estimate optimal evaluation time is 2-7 years after closeout. Valuing Voices also believes we should not just evaluate the sustainability of outputs and outcomes of what we put in place that we thought they would continue, and the sustained impact of those cumulative investments, but also the emerging, unintended new activities and impacts we never imagined people would innovate from our projects. We are doing just such evaluations in Zimbabwe and Uganda now and hope to do and catalyze much more fieldwork around the world.
And why does it matter? Why shouldn't we write off our time-limited donor-funded projects? Because:
1) It's all we've got. Our current development system is not going anywhere soon, and there is success to learn from.
2) We need to quickly learn from what worked sustainably best and stop wasting time and resources on what we refuse to admit fails because we are too scared to return to see. Go back with the intention to learn what does and focus on doing more of what works.
3) Such analysis – and design of new projects – must have country ownership as a centerpiece throughout the project cycle assumptions, but to throw out decades of good work simply because we are just learning the value of country ownership is foolish.
Finally, here's a lovely example from Brazil of how local, participation (and yes, as my colleague thought, local ownership) works best. And. It. Takes. Time.
"Our results also show that Participatory Budgeting’s influence strengthens over time… Participatory Budgeting’s increasing impact indicates that governments, citizens, and civil society organizations are building new institutions… cities incorporate citizens at multiple moments of the policy process, allowing community leaders and public officials to exchange better information." How often do we return to do what are called longitudinal reviews of our work abroad, using the same rigorous standards we evaluate our domestic projects? Not often. Shouldn't that change?
Only by working together, honoring the value of our participants, that they deserve the same chance at change that we take for granted will things change. We must value both the voices of our participants and our own expertise for development to improve for true aid effectiveness…. Let us begin anew!
Whose responsibility is it to sustain project activities?
Billions of dollars are pumped into development activities in developing countries all over the world. Communities getting involved in these projects have a clear objective, which is to have their lives improved in the sectors that the projects target. As to whether this is the main objective of the development partners is not clear. What is clear is that the development partners focus more on numbers than on getting people to participate.
We note that majority of these projects are designed to last between 2-5 years. Delays occasioned poor planning or other unforeseen factors eat into the implementation time to an extent that in some projects it takes about 1-2 years to get a program running. This means that the planned implementation time is reduced. Baselines, midlines and end lines studies are conducted to inform changes that may have occurred within the program life, and in most cases they happen shortly after the program has started or just before it ends. In fact, some baselines are conducted after programs have started.
Considering the reduced implementation time and the fact that it takes a much longer time to get concrete behavior change related results, questions emerge whether indeed the reported changes are solid enough during implementation to be sustained. There is also a difference between measuring what can be referred to artificial changes (activities that community members adopt as a way of short-term trial in their excitement, but don't find useful afterward) and long lasting changes that community members adopt because they are useful part and parcel of their lives.
Almost all projects have logical frameworks (logframes) that show how project activities will be implemented and to some extent there are also exit strategies for closing out the project. This can be an illusion long-term. In most cases donors and implementers assume that communities will adopt activities that are being implemented within a specified period of time, and so projects close down at the end of the specified period of implementation assuming things will continue, but have no proof. Valuing Voices has done projected sustainability work in Ethiopia which points to possible differences between what donors expect to the sustained versus what communities are able to sustain.
The big questions remain: "whose responsibility is it to ensure that whatever has been adopted is continued? Whose responsibility is it to sustain project activities post project implementation?" It is silently assumed that communities can take up this responsibility and a key question is "what guarantees are there that this is possible and is happening?" Project sustainability should not be seen as a community-alone responsibility but rather a responsibility for all those who are involved in program activities. Sustainability studies should be planned for and executed in the same breath that the baselines, midlines, end-lines and in the rare cases impact assessments in real time should be planned for. We must do sustainability studies as they provide an additional realistic opportunity to inform us on actual community development post project implementation. Communities should not be left alone with it.
It's not just Me, it's We
Many of us want to be of service. That's why we go into international development, government, and many other fields. We hope our words and deeds help make others' lives better.
For 25 years I've written proposals, designed and evaluated projects, knowing that while I could not live in-country due to my family constraints, I could get resources there and help us learn how well they are used. I became a consultant so I could raise my kids without being on the road 60% of the time, one who promotes national consultants so that African, Asian, Latin American and European experts evaluate their own projects. I put myself into the shoes of our participants and realized any local person my age wants to leave behind a better, more sustainably viable livelihood for her family, so I looked to see what was most sustained and how we knew it. I took my love of participatory approaches of listening to and learning from the end-users and founded Valuing Voices to promote learning from projects whose activities were most self-sustained.
Yet this is not enough. I am one person with only my views (however great I think they are :), many of us have great views and knowledge about how to best promote sustainable development. For the state of things today seem to me that too often our donors have limited funds for limited time with goals that they limit because they can only assure success by holding the outcome and funding reins so tightly that none of us are fostering self-sustainable development which takes time, faith in one's participants. I have found that the lack of post-project evaluation (see ValuingVoices.com/blogs such as this one on causes and conditions being ripe for sustainability) is a symptom but doing them also provides a huge opportunity to design projects well learning from what communities were able to sustain themselves, based on why/how it worked and how can we do this well again? For instance, from my fieldwork I have realized that questions such as ‘sustainable by whom for how long’ are ones I never asked and don’t think others have ways to go about it well (yet)… unless you have ideas!
How can we foster aid effectiveness, effective philanthropy, community-driven-development, community-driven and NGO-led impact , and effective policy? It takes many of us – giraffes, ostriches, wliderbeast, gazelles, each with our own expertise.
This takes Time to Listen, respect for local capacities (Doing Development Differently) and an openness to step out of the limelight of 'we saved you' to asking "how can we best work together for a sustainable world?". This takes you, me, WE. One way is to join together in a LinkedIn Group: Sustainable Solutions for Excellent Impact where we can discuss how can we best design, implement, evaluate, fund, promote (etc!) projects well that are programmatically, financially, institutionally and environmentally sustainable. Please join us!
Longing to do an Ex-post Sustainability Evaluation? How to support this work…
Just back from Niger where Catholic Relief Services, Rutere Kagendo and I are doing a fascinating post-project evaluation. Fresh on my mind is the commitment we all had to make to get this quite ground-breaking research going. Here is the full report, but there are three kinds of conditions we found were integral for success: client-ValuingVoices match; project and site selection; and resources.
Client – Valuing Voices match:
The study needs to be appreciated as innovative, adding to the program quality and learning of the organization so funding is provided and there is in-house interest in the findings;
The local office needs to allocate staff and technical time to support the study technically and logistically (see below);
Shared clarity is needed among all involved that such a study looks for self-sustained activities and outcomes. While there are lessons that emerge about the quality of implementation, its focus is what participants and their country-partners could continue themselves after project close-out and withdrawal of resources. It also can include lessons about what the local non-profit and the national stakeholders are doing to support community success (or not) and unexpected outcomes. Our clients need an openness to honestly seeing what was not sustained and exploring why;
While Valuing Voices provides expertise given review of the handful of post-project and exit evaluations that exist, client is interested in sharing findings and advocating to donors to fund more of these studies;
Disseminating findings internally where a possibility of learning from this evaluation to support similar current implementation; research could help similar project learning, lessons for country nationals such as Ministries;
Prioritizing local capacity – Valuing Voices believes in using regional M&E capacity to do the work; where possible we partner with regional evaluators while also building capacity within our client's staff to carry out such work;
Sharing and discussing findings locally: Valuing Voices believes knowledge learned needs to be shared in immediate feedback loops. We present: a) to each village after each site’s research; b) to local key partners and representatives from each village at the end of the qualitative Rapid Rural Appraisal findings; c) to the non-profit in-country at the end of qualitative research and d) internationally to headquarters at the end of the combined analysis of the qualitative and quantitative research with findings in the final report.
Project and Site Selection:
Non-profit programming projects has been closed out for at least two years and no more than seven years (for recall);
No other NGO has done very similar work in the region in the intervening years;
The region selected is representative of the project as a whole (e.g. agro-ecological zones, economic/ livelihood/ health, educational or other sectoral criteria);
Research areas are secure and safe (e.g. from civil unrest, severe drought/ floods, epidemics, to the degree possible);
Timing does not interfere with urgent priorities of those involved with the study (e.g. livelihoods are not jeapordized in communities, holidays are kept, other technical work is not disrupted).
Resources (Time, Material and Project Expertise):
Time: the research is qualitative followed by quantitative, coming to 80-90 days of qualitative and quantitative research overall (roughly 5 weeks of fieldwork in teams of 4-10, analysis, report-writing and presentation;
Project and evaluation documents are available to inform and contextualize approach including activities, outcomes and projected impacts;
Data: key to the fieldwork are village and participant lists from pre-closeout days so participants can be interviewed both during a Rapid Rural Appraisal and a follow-on household survey;
Internal/external sectoral staff and at least one past project staff are part of the team to inform and ‘ground truth’ research;
Logistics support is provided by the client; from vehicle/driver and lodging support in the field to materials such as mobile phones, flipcharts, photocopying and advances;
A consultant or staff prepares the sites before the research teams come, e.g. to confirm communities are willing to be visited (each visit will be 2-5 days) and to identify participants and partners still there;
Partners familiar with the closed project can be identified so they can be interviewed by the research team;
Local language expertise is needed, e.g. translator to local language, as well as data entry personnel afterwards.
While Valuing Voices provides the technical lead experts and statistical back office analysis, including sampling and rigorous analysis, senior non-profit staff are needed in-country for contextualization and input on preliminary findings, as well as senior technical staff to review the final product;
A home for findings and on the road for dissemination: good knowledge management is needed for data retention, for the findings to have a sustainable ‘home’– be that info-graphics and print copies distributed to villages and partners or online repositories created that are language-accessible both nationally and by foreign donors; webinars, conference presentations etc are needed to optimize the learning via sustainability dissemination campaigns.
Much of this is needed for the research to be the best quality and yield the highest results. It is exciting learning is to be had from not only what communities and supporters could sustain but what they exceeded or dropped! Consider doing one to see how post-project sustainability research can improve your current implementation, future design, and long-term self-sustainability!