On June 24th under GLocal’s UNConference, “Co-creating our future stories of hope and action”, Jindra Cekan, Holta Trandafili, and Isabella Jean presented their work on sustainability evaluations and exit strategies via local voices. We chaired a 2-hour discussion session on the following topics:
Sustainability of global development projects and exit from them,
The importance of valuing local partners’ and participants’ voices,
How to embed ex-post evaluation of sustainability into the project cycle,
How expectations, benchmarking, and early joint planning in exit strategies, as well as considering long-term ownership & relevance will support projects to be sustained locally, including questioning who will maintain results,
Considering power dynamics between donors and ultimate ‘beneficiaries’ and
the value of the impact of the project from a variety of perspectives, and more
Jindra Cekan:
· Fear of learning about failure in our global development industry – INGOs are “waiting for a successful enough project” to commission ex-post project sustainability evaluation
· It needs to be a culture of learning, not a culture of success.
· Lack of transparency in sharing program evaluation results with communities and local government is widespread, and even more rarely do we come back after many years and share learnings
· Participatory approaches are vital, listening to participants about sustained impacts is key
· It is never attribution, always contribution. To isolate impacts, we need to look for project sites that haven’t had multiple other organizations overlapping through all phases.
· Building sustainability planning throughout the project cycle is key – but often doesn’t happen
Holta Trandafili:
· Sustainability needs to be planned to be researched, including evaluating why or why not were project elements sustained, and why? What has the project done to enable communities to sustain improvements?
· Expectations of sustainability need to be more modest (as most results are mixed good/bad)
· We need to ask: How are you defining and measuring sustainability – for how long should the results last? Among how many participants? Have you set benchmarks for success?
· We should expand your toolbox on methodology to investigate sustainability. Stories of success are one of a myriad of methods used, including mixed-methods, cost-benefit, etc.
· Start with the need for learning not [just] accountability
Isabella Jean:
· Sustainability investigation/evaluation/learning should be mindful that this is NOT about projects. It is about people.
· We have a system that focuses on gaps and needs to be filled vs. existing capacities’ structures to be reinforced. How can our work on measuring sustainability bring this to light and call it out, so that we change the norm?
· Planning for sustainability requires the insight to integrate resources and experiences of outsiders with the assets and capacities of insiders to develop context-appropriate strategies for change
If you would like to discuss this with any of us, please send us comments and we’re happy to respond. Thanks again to the G-Local UnConference team!
Below please find our bios:
Jindra Cekan/ova has worked in global development for 33 years focused on participatory design and M&E for global non-profits. She founded Valuing Voices 7 years ago. For details, see: Valuing Voices Founder
Holta Trandafili is the Research, Learning, and Analytics Manager with World Vision US and has been leading field research, monitoring, and evaluation since 2007. She has led sustainability measurement studies for World Vision programs in Uganda, Kenya, Sri Lanka, India, Burma, and Bolivia. Her areas of expertise and interest include program and community groups’ sustainability measurements; agency-level measurements; empowerment approaches to development; integrated programming; local capacities for peace; gender analysis; and outcome monitoring. Currently, Mrs. Trandafili serves as an Advisory Committee member for InterAction’s Effectiveness and Program Evaluation Working Group and chairs one of the sub-working groups under The Movement for Community-led Development.
Isabella Jean supports international and local organizations and funders to document promising practices, facilitate learning and strengthen capacities for conflict sensitivity, peacebuilding and humanitarian effectiveness. She has facilitated action research, collaborative learning and advisory engagements in over 25 countries, and serves as an advisor to policymakers, senior leadership and program teams. Isabella co-authored the book, Time to Listen: Hearing People on the Receiving End of Aid and developed practical guidance to support accountability to communities, listening and feedback loops, and responsible INGO exits. She teaches graduate-level courses on aid effectiveness, program strategies and M&E of peacebuilding at Brandeis University’s Heller School for Social Policy and Management. Previously, Isabella directed training at a community organizing network and conducted policy research for the Institute for Responsive Education, UNDP, and Coexistence International.
Setting a higher bar: Sustained Impacts are about All of us
Global development aid has a problem which may already affect impact investing as well.
It is that we think it’s really all about us (individuals, wealthy donors and INGO implementers) not all of us (you, me, and project participants, their partners and governments). It’s also about us for a short time.
All too often, the measurable results we in global development aid and Corporate Social Responsibility (CSR) funded projects that last 1-5 years track and report data for two reasons:
1) Donors have Compliance for grantees to meet (money spent, not lost, and results met by fixed deadlines of 1-5 years – look at some of the European Commission Contracting rules) and
2) Fund recipients and the participants they serve are accountable to ‘our’ donors and implementers who take what happened through their philanthropic grants as ‘their’ results.
Both can skew how sustainably we get to create impacts. An example of such strictures on sustainability from USAID. As respected CGDev Elliot and Dunning researchers found in 2016 when assessing the ‘US Feed the Future Initiative: A New Approach to Food Security?‘ the $10.15 billion leveraged $20 billion from other funders for disbursement over three years (2013-16). “We are concerned that pressure to demonstrate results in the short termmay undermine efforts to ensure any impact is sustainable…. Unfortunately, the pressure to show immediate results can encourage pursuit of agricultural investments unlikely to be sustained. For example, a common response to low productivity is to subsidize or facilitate access to improved inputs… it can deliver a quick payoff… however, if the subsidies become too expensive and are eliminated or reduced, fertilizer use and yields often fall…..
With so much focus on reporting early and often about the progress in implementing the initiative, there is a risk that it increases the pressure to disburse quickly and in ways that may not produce sustainable results. For example, for 2014, Feed the Future reports that nearly 7 million farmers applied “improved technologies or management practices as a result of U.S. Government assistance,” but only 1,300 received “long-term agricultural sector productivity.” Are the millions of others that are using improved inputs or management practices because of subsidies likely to have these practices sustained? And how likely are they to continue using improved practices once the project ends?”
3) Impact investors stick to the same two paths-to-results and add a new objective: market-competitive financial returns. They also need to show short-term results to their investors, albeit with social, environmental and governance results like non-profits (future blog).
4) Altruists create things we want ‘beneficiaries’ (our participants) to have. For instance a plethora of apps for refugees cropped up in recent years, over 5,000 it is estimated, which can be appropriate, nor not so helpful. Much like #2 above, ‘we’re’ helping ‘them’ but again, it seems to be a ‘give a man a fish’… and my fish is cool sort of solution… but do our participants want/ need this?
How often is our work-for-change mostly about us/by us/ for us... when ideally it is mostly about ‘them’ (OK, given human self-interest, shouldn’t changes we want at least be about all of us?).
All too often we want to be the solution but really, our ‘grassroots’ clients who are our true customers need to generate their own solution. Best if we listen and we design for long-term sustainability together?
As the Brilliant Sidekick Manifesto stated in two of its ten steps: a) “I will step out of the spotlight: Sustainable solutions to poverty come from within are bottom-up, and flow from local leaders who are taking the risks of holding their politicians accountable and challenging the status quo.”
b) “I will read “To Hell with Good Intentions” again and again: Politicians, celebrities and billionaire philanthropists will tell me that I can be a hero. I cannot. The poor are not powerless or waiting to be saved. Illich will check my delusions of grandeur.”
We have examples of where we have stepped away and participants had to fend for themselves. At Valuing Voices, we’ve done post project-exit evaluations 2-15 years afterward. What did participants value so much that they sustained it themselves (all about them, literally)? These Sustained and Emerging Impacts Evaluations (SEIE)also give us indications of Sustained ROI (Sustained Return on Investment (SusROI) is a key missing metric. As respected evaluator Ricardo Wilson-Grau said in an email, “I think calculating cost-effectiveness of an intervention’s outcomes would be a wonderful challenge for a financial officer searching for new challenges — if not a Nobel prize in economics!”)
Most of these evaluations are pretty bad news mixed with some good news about what folks could sustain after we left, couldn’t and why not. (These are the ones folks expect to have great results, otherwise they wouldn’t share them!) While most clients are understandably interested in what of ‘theirs’ was still standing, and it was interesting disentangling where the results were attributable by implementation or design or partnership flaws or something else, what was mesmerizing was what came from ‘them’.
The key is looking beyond ‘unexpected’ results to look at emerging impacts that are about ‘them’ (aka what we didn’t expect that was a direct result of our project, e.g. spare parts were no longer available to fix the water well pump once we left or a drought rehabilitation water project that decreased violence against women), to what emerging results are attributable not to use but only to our participants and partners who took over after our projects closed.One example is a Nepalese project ended yet the credit groups of empowered women spawned groups of support groups for battered women. Another is a child maternal health project changed how it worked as women reverted to birthing at home after NGOs left; community leaders punished both parents with incarceration in the health clinic for a week if they didn’t given birth there (wow did that work to sustain behavior change of both parents!).
Many of us at Valuing Voices are shocked that funders don’t seem that interested in this, as this is where they not only take over (viz picture, sustaining the project themselves), but they are making it theirs, not ours. Imagine assuming the point of development is to BE SUSTAINABLE.
Source: Community Life Competence
Our participants and national stakeholder partners are our true clients, yet… Feedback Labs tell us Americans alone gave $358 billion to charities(equivalent to the 2014 GDP of 20 countries) – in 2014 but how much of this was determined by what ‘beneficiaries’ want? Josh Woodard, a development expert, suggests a vouchers approach where our true clients, our participants, who would “purchase services from those competing organizations… [such an] approach to development would enable us all to see what services people actually value and want. And when we asked ourselves what our clients want, we would really mean the individuals in the communities we are in the business of working with and serving. Otherwise we’d be out of business pretty quickly.”
This opens the door to client feedback – imagine if participants could use social media to rate the sustained impacts on them of the projects they benefited from? A customer support expert wrote in Forbes, “Today, every customer has, or feels she has, a vote in how companies do business and treat customers. This is part of a new set of expectations among customers today that will only grow ... you can’t control product ratings, product discussions or much else in the way of reviews, except by providing the best customer experience possible and by being proactive in responding to negative trends that come to the surface in your reviews and ratings stronger.”
So how well are we working with our participants for ‘development’ to be about them?
Investing in Youth for Project Effectiveness and Sustainability
One out of every six people on earth is between the ages of 15-24, says the UN. That is 1.2 billion youth. As one young leader says, “if the world’s problems are to be solved, it’s not going to happen without us.” Yet in 2015, the International Labor Organization said 73.3 million youth between 15 and 24 were unemployed. Not only do and an estimated 169 million young workers lived on less than $2 a day, 75 percent of youth workers are only informally employed. In Africa alone, the UN estimates 200 million are such youth; not only does Africa have the youngest population in the world, this figure will double by 2045, but the largest numbers remain in Asia (IMF 2015). The World Bank has striking African and Asian demographics:
How often do we fund projects that are designed and run by youth? How engaged are youth in sustaining the projects we have funded, designed, implemented, monitored and evaluated? What have we in global development, including corporate social responsibility and investing spheres done to ensure that youth are both engaged in our projects, but are in the leadership to direct how they are done now, and sustaining them beyond donor departure? Further, how well are we collaboratively developing technology with them for them to use to thrive in this sped-up, high-tech world?
The exodus of youth diminished during and after the project [by] using the same land to train 100 new vegetable farmers and trainers. Youth seasonal outmigration decreased due to increased food production, especially due to vegetable gardening even during the dry season, and increased knowledge of practices such as rainfed agricultural [practices] which kept youth locally employed.“
Youth, too, having learned [agricultural water-conservation] techniques, and generated income [even] while seasonally working outside the village.
Versus not engaging youth:
Although most committees are still functioning, there are no processes in place to engage and train youth and new inhabitants of the villages [in project activities after close-out]… … there are serious questions about how well they will be engaged and train youth and new members of the communities and how much will be transferred cross-generationally. This is pivotal given that 50% of Nigeriens are under the age of 15;
The [sustainability] problem was that [youth] were not elected or chosen for the [management] committees. This is another issue to flag in other projects interested in sustainability: the implications of selecting a limited number of elders to staff multiple committees than a broad array of young committee members that could grow into leadership positions. Given the youth’s overall dissatisfaction with group leadership, other projects need to be aware of “elite capture” and its potential threat to sustainability
Investing in youth is a terrific investment in sustainability. How often do we consider it?
A post-project example from Mercy Corps/ PCI’s early ex-post evaluation in Central Asia showed that such investments are not easy but can pay off after the project closed. “72% of youth report that they continue to use at least one skill they learned during the [infrastructure] program”… including teamwork and communication, sewing, construction, roofing, journalism and cooking.” This may have been in part due to the project’s youth summer camps, organized each year to promote youth leadership and participation in community decision-making, which were supported by [some of] the adult population. While the project “encouraged communities to elect young people as representatives…within the cultural context this was not met enthusiastically by the communities because young people were not felt to be ‘qualified’ as leaders.” Yet inter-generational collaboration was fostered by the project by establishing mentoring programs where older people with technical skills mentored [some] young people during the infrastructural construction activities.”
Raj Kumar, Founder of Devex and chair for the World Economic Forum said this about what to do post-Davos: “With a dozen years to go before the finish line of the Sustainable Development Goals, we need to get the underlying plumbing right in order to have a chance to reach those goals. That plumbing includes everything from having the country-level data to track progress against the goals to having the project-level data to know what’s working and what’s not…. Most importantly, it’s about the development leaders of today building out the best systems so the development leaders of tomorrow can focus on delivery.”
What we at Valuing Voices are most encouraged by is the prospect of overtly considering sustained impacts to inform the funding, designing, implementing, monitoring and evaluating of projects today for the adult millennials of tomorrow. Regarding youth, we will need a mix of focused initiatives such as longstanding work by the International Youth Foundation, and new investments funds such as the Global Youth Empowerment Fundand integrating youth into projects at all stages of the projects and beyond, as we showed regarding CRS and Mercy Corps/PCI, above.
That is one way to get SusROI (Sustainable Return on Investment).
As mother who has worked in 26 countries, I feel the great urge to harness youths’ yearning to succeed through their love of technology. The growth of mobile money in Africa is one example of technology use in daily life. Who is supporting youth employment in technology? Mercy Corps’ 2017 Social Ventures Fund that supports “positive trends offered by technology. Trends in micro-work, micro-manufacturing, digital livelihoods and mobile-enabled agent networks are paving the way for the acceleration of a distributed and digitally enabled workforce and the reinvention of manufacturing, sales and distribution. Their investments related to youth employment are:
NewLight Africa – network of rural sales and customer service agents
Wobe – anyone with an Android phone in Indonesia can be a micro-entrepreneur
Lynk – job-matching platform
Sokowatch – network of urban sales and customer service agents
I dream of youth crowd-sourcing post-project sustained impact results. Feedback Labs has a lot of really interesting tools for… feedback from people on the ground in-country. I searched high and low and found only this crowdsourcing data collection overview of three aid research tools for ICT4D (information and communications technology for development), the best of which appears to be Findyr. (FYI here is feedback on the limitations of crowdsourcing in emergencies.)
Finding technologies and funding to hear youth’s voices and feedback on what they could sustain or could not after our projects closed, and why is unbelievably valuable to inform funding, design, implementation, M&E, and of course foster youth empowerment.Good listening to participants comes first. As an impact evaluation in Uganda found that “when villagers and teachers, instead of school officials, are allowed to set their own priorities for improving schools and directly monitor performance, the results can be priceless. In Uganda, World Vision knew that community-based monitoring of school performance could help sustain improvements in education that building schools, supplying textbooks, and training teachers alone could not. They tried two approaches: the use of a standard scorecard with performance questions identified by education officials and development partners, and a participatory scorecard, where community members defined the issues they would monitor. A randomized controlled trial [RCT] revealed that the participatory scorecard delivered more than the standard scorecards. The participatory approach prompted higher efforts by teachers, as expected. But it also prompted higher efforts from villagers— local politicians learned more about their country’s education policies and what they could advocate for on behalf of their constituents, parents increased their support of schools by contributing to midday meals, and children found a forum to report teacher absenteeism and other factors that hurt their education. In the end, while the standard scorecard made little difference in school performance, the participatory approach improved attendance by teachers and students and helped raise student test scores.’”
By accessing mobile technology, ground-truthing project sustainability, given youth’s familiarity with technology and network-interconnected habits, I believe together we can cost-effectively democratize evaluations and help ‘development’ be ‘sustainable’. Collaborate with us!
The Name of the Game is “Sustainability” but Does the Last Player Count?
by John Lowrie, Reblog from https://www.scribd.com/document/11621086/The-Name-of-the-Game-is-Sustainability-but-Does-the-Last-Player-Count-by-John-Lowrie
Today, it is obligatory to answer the question “how will your proposed activities be sustainable after the project?” Most of us dutifully play the game and repeat various sentences that describe the measures that “should” bring sustainability about. Then it is usually left at that. We all move on to the next project and donor. Seldom does anyone look back over the passing of many years to check to see if promise has turned in to reality. At best there may be an end-project evaluation which will say that planned outcomes are achieved and “likely” to be sustained. It would be a very brave evaluator to be more committed beyond that.
My question is why do we play this game? (I should add: apart from the obvious answer we need the donors’ money!) Sustainability is much more than a ruse in a game. It goes to the very essence of what kind of organisation we belong to; plus all the others with us whether staff, supporters or beneficiaries; how we originated and what are our real long- term plans?
Many NGOs are artificially created groupings of people, unlike the first ones last century that emerged as people on a mission to address specific issues that they felt strongly about. Latterly, NGOs are now lumped in to “Civil Society” which is another recent fashionable label in development jargon. This is despite many officials and even NGO workers in Cambodia having no idea that it is supposed to encompass more than NGOs, to embrace other groups such as the press, trade unions, etc.
Many NGOs have actually been started by or for charismatic individuals, tapping in to somebody else’s cause or source of funding, rather than part of a collective if not mass movement in its own right towards a common end. Sometimes, international departing NGOs promote them, as part of “exit-strategies” to demonstrate that they leave something behind to show for their good work and all the money spent. They may have even included a plan for it in their sustainability proposal write-up. In Cambodia the “gravy train” that accompanied
UNTAC, the massive international effort to bring lasting peace and democracy in the early 90s, spawned many. Some of these NGOs have been good and stood the test of time. Many have fallen by the wayside. Others have been co-opted by political interests. Few have proper accountable self-governance structures.
◄“It’s the money that matters”, but will these NGO beneficiaries, the final players, get what they need to be sustainable?
So how can they be really sustainable? What is driving them – apart that is from the obvious – money? It is true that many do good work. They often do work that local authorities should be doing. But, I ask, towards what end? For example “poverty alleviation” is not an end in itself; for it to be sustained it needs much more than the usual 1-3 year time-frames that donors favour for their projects. Yet the usual pattern is –identify your target groups, go there, pass on whatever to them, then move on (to the next ones). There are exceptions. Lutheran World Federation (LWF) expect to work for 10 or more years in their target communities before they “graduate” and begin a systematic staged withdrawal. Mostly, however, the projects end on time or soon after, the files are closed, and that is that, at least until next time.
Meanwhile NGOs try to source repeat funding or new funding, and the game rules dictate where and how they proceed. It is only by luck rather than design that there will be a true match between what they want to do and what the donor is willing to give the money for. Usually there is accommodation, most likely on the part of the NGO; indeed some big donors explicitly rule out comments, queries, and changes to their guidelines which become grails of holiness. The most desperate NGOs have to re-invent themselves; depart from their original mission, suddenly acquire new skills in often far removed fields to stay in the game. I have seen an election-monitoring organisation become an agency to consult displaced families affected by a road-widening. I know of one local rural development NGO in one province become a pro-citizen governance campaigner in another. Adaptability and learning to diversify are good qualities, but when they cause such radical changes within an NGO, it cannot be re-assuring for sustainability.
The nature of what an NGO does and its underlying philosophy is therefore key to sustainability. Those NGOs created to work in the immediate post-conflict or disaster emergency relief periods are prone to short-term visions, and sometimes they leave legacies which handicap development such as “dependency” and even “easy-come-easy- go” attitudes when foreign money seems plentiful. Those NGOs born in the next phase, i.e., after emergency relief, the start of infrastructure reconstruction and restoring public services and the economy, can also be equally short-term in their vision. In fact they have an inherent flaw. If they succeed, they do themselves out of a job and few want that as it would mean no future jobs and income! Even the Cambodian Government, despite an addiction to foreign aid, now maintains that some have overstayed their welcome.
These NGOs tend to be “welfare” or “service-provision” providers. They are confident in their abilities “we know best” but are they committed to passing their best skills and knowledge on and to the right people to take progress forward without them?
Cambodia is not alone in that politics plays a big part. The ruling party now has a 73% majority in the National Assembly, holds 98% of the 1,629 Commune Council Chiefdoms and 70% of Councillors. As the national and commune members form the constituency for the Senate, Provincial, and District Authorities, the party is guaranteed monopoly control. The situation is not helped by the absence of a neutral civil service or public service – in fact it is mystifying that UNTAC and every international donor has not tried to cultivate one. So we are left with what we have; we are where we are. It means that sustainability will only follow if we accept that reality. We have to engage constructively with the powers-that-be; we have to find the best people we can to work with, and through them, reach out to others. If not, eventual opposition or just plan lack of good will and support will affect the final outcome. Authorities always outlast NGOs, at least in Cambodia they do!
Cambodia did not have a good start when NGOs first came on the scene. Many of the first NGOs were human rights activists needed at that time and to a certain extent now, to expose calamitous treatment meted out to victims. Unfortunately this profile, still high in perceptions, has made many in power believe that this is only or mainly what NGOs do and stand for1. The legalistic approach to human rights where abuses are reported, perpetrators identified, and “name, blame shame” attached, cannot be a development tool in the sustainability tool-kit. It must be separated out and equally important alternative human rights approaches such as “rights-based development” which arouse less hostility co-exist with similar enthusiasm and means. NGOs and civil society will never have sustainable activities while their undoubted overall positive contribution gets little or no recognition by the people that count most when it comes to change.
NGOs are agents of change, which if mishandled does lead to suspicion, so to achieve change, NGOs must be clear in their message to persuade all (or a majority) that their change is worthwhile. If a long-term, wide cross-section commitment towards that change is not implanted, it is not sustainable. Too many development initiatives in Cambodia have resulted in temporary change. The “status-quo” reverts soon afterwards. In some cases the change is only tacitly accepted “take their money”, “go along with them” but “once they have gone, we go back to how we were, OK?” This is the opposite of sustainability.
There have to be contextually appropriate solutions, which can only be country-by- country, culture-by-culture. We should not have to operate on the basis of big international donors with their “one-size-fits-all” development policies and calls for proposals that allow just one format (theirs) that automatically favours the big international NGOs. They have their professional fund-raisers, who can knock out proposals that score high marks in the assessment/evaluation boxes, without many of the authors and assessors concerned ever going near the intended target beneficiaries! Yet be under no doubt, the words they pen and peruse answer beautifully on how such beneficiaries were involved, but who ever checks?
I may not be a good team-player in the present development game, so what am I suggesting as an alternative? First of all I would like to see fewer NGOs, ones which are smaller, more self-contained, and manageable to operate country-by country, sector-by- sector, region-by-region. They need to have good links to others elsewhere for best practice to be shared, but their core mission needs to be focussed to bring about certain defined changes; with the right people and the resources they need, and in the fullness of time. Therefore 5 years is an absolute minimum and asking for “core costs” to be provided should not be regarded as a mortal sin, as it tends to be with most donors. It has to be allowed, to be seen as value for money, and though indirect, still an essential element in bringing about that change. If not, how can sustainability be served?
Ironically, it is only with core support or independent means, that NGOs can play the game, so again favouring the big players. How else does an NGO cover its costs in preparing bids to donors? For example after UK-DfiD released their worldwide Global Transparency Fund in 2008, 450 organisations applied, and just 38 succeeded. Even a recent in-country release by UNDP for environmental projects attracted no less than 67 applications, of which just 13 won out. This means that for 412 in the DfiD case and 54 in the UNDP case who had devoted considerable effort, on top of their normal work, the process ultimately proved to be a waste of time; to be disappointing and especially for local NGOs to be discouraging.
I have two (NGO) organisations in mind while writing this article. Neither conforms to the pattern I criticize. They are different and I am not the only one to think that they are better. Both have poor disabled people as their target beneficiaries. One is in sports. The other is in poverty alleviation/new livelihoods or careers and self-advocacy. Both are now purely local NGOs, with none of the trappings or spending power of the big international players that dominate the disability sector. In fact neither has anything like a long-term future, because of funding gaps, and so a lot of their work, despite excellent results so far, cannot be said to be sustainable. If they go out of business too soon, how will their beneficiaries stay involved, if sustainability is to be realised eventually?
Local NGO “Cambodia National Volleyball League Disabled2” is the sports one. Most interestingly, it is taking a route party by choice, partly by necessity, towards the private sector or “corporate social responsibility” funds in its hope to realise sustainability. CNVLD has earned a worldwide reputation for transforming the self-esteem of disabled athletes. They actually enjoy high standing on the world stage unlike their “able-bodied” compatriots. They play volley-ball, pursue wheel-chair racing and some athletes may well qualify for the 2012 London Paralympics, even if there may be no money to support them to go and compete there.
◄Disabled Sports Athletes of NGO CNVLD playing the game at its best – but do they have a future?
Now CNVLD’s quest is to raise such money and cover their modest (compared to INGO) costs. Yet this laudable aim is treated with derision by some in the sector. Pursuing private funding is viewed as an anathema to many who see it as contradictory to the “not- for-profit” concept. Does this make sense? Does such obstruction make for sustainability? Is it a sin not to want to depend solely on the usual institutional sources of funding? One critic is interesting. It has in its mission that all its services to beneficiaries must be provided free. No charges whatsoever must be levied even for those who can afford to pay! In Cambodia, everybody pays, even when services are supposed to be free3. Even if a direct charge is not levied, recipients expect and are expected to show their gratitude4, and do so especially the poorest who see it as an inescapable obligation. Surely sustainability means (a) people who can afford to pay should do so, and (b) NGOs who can attract funding and not depend on taxpayers’ money should be welcomed5?
The second organisation, New Horizons Society6, is not as lucky as CNVLD, as they cannot go down the route of sponsorship which is an accepted feature of sport. This is an NGO that did localise from an INGO but on their own terms. They voted against joining another national body created by and favoured by the big international disability NGOs. Over 200 of their Focus Persons (Group Leaders) voted in a secret ballot to form their own NGO in order to stay true to their close-knit grassroots upwards origins and growth. Now they have 3,175 members in 135 self-help groups federated up to provincial level and going on to the national stage. They have remarkable accomplishments in creating new livelihoods for their [once] ultra-poor members and have accumulated over $130,000 in revolving funds. Individual lives have been transformed. One boy has gone from beggar in the market to national singing celebrity. One young man went from lonely at home; never seen a computer; no English, to being one of today’s high-flying geeks. His class-mate, from a similar start, went on to become the Publicity Officer for CNLVD and to lead her own troupe of dancers in the NHS Child Advocacy Group performing at international conferences.
◄NGO Child Advocates demonstrate “We can do” but will donors let them?
The 135 groups went from fear of talking to officials to successful advocacy. They started with the right to education and health-care for disabled youngsters, and once they were confident went on to persuade ministers to take action to stop their meagre pension rights being denied to them. These people chose not to ask for pity, or welfare, or for service provisions to be given to them. Instead they ask simply for the chance to show that “they can do” and that they can be self-sufficient when given the opportunities and means.
Their problem is that they cannot do this for all members yet, let alone go on to include others in the same fate as they once were. Yet despite their accomplishments, right now they can only win project activity funds. Donors refuse to pay more than 20% for running costs. Some specify as little as 5%. That does not even cover the running costs of their multi-purpose centre where meetings, training, sports, dancing, computer classes etc., go on. It may stretch to pay modest salaries to 2 or 3 staff, but they have to depend on consultant/advisors like me to help them voluntarily. Their entire organisation is radically different from the familiar set-up to be seen in international disability, donor and development organisations. There is not an air-conditioner, land-cruiser, or voucher paying university fees of expatriate’s children, etc., in sight. Yet despite its low cost and high yield, it is not yet sustainable within the present rules of most donors. If it can get over its current shortfalls and continue to build the revolving funds in to a sizeable trust or investment fund, with many members whose incomes have grown sufficiently able to subscribe fees to run their organisation, then they may have an independent viable future. But is there a donor who can adopt such a long-term vision; who will give enough to meet the real needs they identify, and who will then stay the course even when inevitable setbacks happen on the way?
So finally sustainability should be expressed in one simple notion and by the last player. It should be a measure of the change made in individual lives and over life-times of beneficiaries and their families. They are the last players in the game – who else but them can make that assessment? The present game means that it is the other players that have most say. They arrive on the scene much earlier; play in their own compact time-frames, and to their own rules. Then as external entrants, they depart the scene as soon as they can. Sustainability?
_____________________________________________
Footnotes:
1 The perception is not helped by the limitations of the Khmer language. When the word “advocacy” was first introduced “tasumateh” was used, literally “to struggle for” associated with confrontation, not partnership, and so it was viewed by many as an alien Western concept.
3 Understanding pro-poor political change:the policy process Cambodia by Caroline Hughes and Tim Conway, Overseas Development Institute, 2003 – DfiD Publication.
4 It is this “tradition” that is at the heart of a current dispute within the UN-backed Khmer Rouge
Trials, where Cambodian staff are alleged to have paid a proportion of their salary to officials with a role in their appointment.
5 Subject of course to disclosure and transparency.
Local Accountability and Transparency… During and Post Project?
Local development partners? Check. Long-term transparent and accountable investments through them as “local solutions” partners? Not so much. While President Obama and former United States Agency for International Development (USAID) Administrator Raj Shah promised up to 30% of all contracts would go to ‘local solutions’ that “promote sustainable development through high-impact partnerships and local solutions”, nowhere near that percentage became true then, much less now. While there seem to be good examples such as Haiti [1], Afghanistan is a poorer example. While most international non-profits implement projects through local sub-contractors, certainly building their capacity to manage and account for foreign taxpayer dollars spent, like this MSI in Lebanon example [2], if we extend the measure of ‘success’ beyond our project implementation, policies and programming needs to change tosustain capacity and implementation post-exit (see INTRAC report [3]).
How local partners are presented can appear as somewhat of a shell game. For while Haiti and Afghanistan have been featured by USAID, I have never seen a full inventory of partners for even a handful of the 60+ countries and regional missions that USAID works in. We hear about ‘local solutions’ and undoubtedly USAID’s ‘implementing partners’ do much good using local subcontractors. Yet are the locals winning the contracts these days? USAID posts contracts lists, for instance those who ‘won’ contracts amounting to $4.68 billion in 2016 [4]. The for-profits of Chemonics ‘won’ over $1 billion, then Tetra Tech and DAI got $800 million of contracts each [4]. These three contractors comprised 39% of all USAID obligated contract funding that year, whereas (U.S.) non-profits garnered 13% of the contracts and small and woman-owned businesses 12% and 7% [4]. Only Kenya Medical Supplies Authority, a state corporation, was listed in the top 20, winning a five-year $122 million contract for Kenya [4]. There are no equivalent sub-contractor lists, much less amounts allocated to national NGOs which would prove we are building ‘development’ ground-up.
While I am focusing on USAID, I believe this is true of most bilateral and multilateral donors. For USAID, caveats abound regarding their ability to accomplish local and sustained ‘development’. A 2015 Congressional Research Paper about their Background, Operations and Issues, cites “multiple challenges in the course of fulfilling its mission” [5], including:
Local Solutions. Providing assistance to local entities incurs the risk of loss of taxpayer dollars. Efforts to mitigate risk generally require more personnel and consequent funding to monitor local entities and build their capacities [5]
Sustainability. ‘Country ownership’ and domestic resource mobilization efforts are two ways the agency has sought to address sustainability, but a clear path to sustainability remains a work in progress [5] …
[Yet] the agency argues that investments are best sustained in the long-term if development is locally owned, locally led, and locally resourced.”
While I am scratching the surface, at least one private sector Corporate Social Responsibility company seem to have more transparent systems. This balance sheet from Abengoa, a sustainable energy technology company, could be updated with such a breakdown.
[7]
Their website also talks about its 25-year CSR investments (which is an enviable timespan, for most donors have 1-5 year projects) [7]. “Abengoa believes that the good relationship it has with local communities, as well as respect and development in the areas where it operates, reaps benefits, referring to this method as “social licence to operate” [7]. Abengoa’s social engagement aims to further the social and cultural development of the communities where they operate. From 2014 to 2016, the company reported its social performance in line with the criteria proposed by the London Benchmarking Group (LBG) methodology. This model defines a method to measure, manage, assess and disclose contributions, achievements and impacts of the company’s social engagement with the community” [7].
Their website also describes current events. “A flagship initiative of the company is the PE&C (People, Education and Communities. Committed to Development) programme.… is now present in nine countries (Argentina, Peru, Brazil, India, Mexico, Chile, Spain, Sri Lanka and Morocco), [but] currently, the complex situation that the company is undergoing and the severe limitation of financial resources in recent months has meant the gradual and temporary reduction of the contributions made to social projects in the different regions. As part of the restructuring plan agreed with creditors and in order to limit the social engagement items based on the resources available in the different business units, each company has assessed its capacity to fund social development projects, maintaining, in some cases, their commitment to certain local social projects. In an effort to avoid the negative impact on these communities and disadvantaged groups, the company has worked hard to find partners and collaborators who could provide continuity to these projects until Abengoa can recover a solid economic position that allows it to continue working and giving support to them” [7] (which their Press Room tells us they have in 2018).
Sharing “achievements and impacts of the company’s social engagement with the community”, “find[ing] partners and collaborators who could provide continuity to these projects” is not often done in bureaucratically time-fixed global development [7]. Fortoorarelydothefixedtimelinesandbudgets, inflexiblemetricsanddemandingdeliverablesenabletruepartnerships. Save the Children’s 2008 brief on aid modalities for country ownership includes a vital point, which is willingness. “The United States lags behind other donors in its willingness to use all the aid mechanisms that would buildcapacity, suchaschannelingaidthroughhostcountries’ systems”, including where only “1 percent [of aid] was passed through projects directly implemented by the host government” [8]. At least US funds were aligned with national government priorities in Bangladesh, Ethiopia and Malawi and in Liberia the US partnered with UK’s DFID for community infrastructure. Nonetheless, even aid to governments must be as locally transparent as possible for true accountability. “LocalNGOsareakeyactorinholdinghostgovernmentsaccountableforthedeliveringmeaningfulresults, andshouldincreasinglybeanimportantlinkbetweengovernmentandcommunitythroughcommunicationsandprovisionofservices” [8] … So why aren’t we funding more? Wedon’tknow, don’tmuchdata how well we are, butunlesswestartwithaccountabilitytothecountrynationalsweareostensibly ‘developing’, sustainedsuccesswillnotensue.
The scant number of post project sustainability evaluations have shown how rarely our international donor funded partners return to partners and participant communities to see what they could self-sustain after our projects ended. So much for accountability to our true clients! Public and private sector needs to turn away from beingdataextractorsaimingatshort–termresults, and ratherturn tobeingledbysustainedpartners’ prioritiesandmyriadvoices. Private sector companies may have lessons to teach, forwouldtheystayafloatifitsinvestorsdidnotlearnhowwelltheirproductworked by notreturningtoaskafterthesaleassessclient satisfaction?
We don’t have a moment to waste.
Thoughts? Questions? Look forward to your comments.
PS – There are surely 500 sources I didn’t find in time to include, including this blog regarding Cambodia and aid, huge numbers of organizations focused on capacity building and also thanks to Abu Ala Mahmudul Hasan for a Pelican online discussion that spurred this. We hope to create a podcast this spring, so stay tuned…
[3] Hayman, R., & Lewis, S. (2017). INTRAC’s Experience of Working with International NGOs on Aid Withdrawal and Exit Strategies from 2011 to 2016. VOLUNTAS: International Journal of Voluntary and Nonprofit Organizations, 29, 361-372. Retrieved from https://link.springer.com/article/10.1007/s11266-017-9901-x
Poor villagers like Edith, Aminata, Rituu, and Juan don’t appear much on the nightly news. You might never know they exist unless you stop and read your mail from some charity asking you to help them. On the brochures, they can look scared or sad; maybe surrounded by their thin children, with a parched land or dying animals behind them. Our foreign aid programs should be helping them, but are they?
I have met these people while they are working in their fields, growing corn and peas, millet and coffee, raising their chickens and goats. I have talked to them outside of health centers where they have brought their babies to be vaccinated or their parents for medical care. I got to know them when I interviewed them under the big tree in the middle of their village, or in empty school rooms, asking them what they need from us, and how we can design foreign aid projects to better help them.
I have worked in international development as a technical expert in project design and monitoring and evaluation for international non-profits such as Catholic Relief Services, Save the Children, the Red Cross, and many others including the Bill and Melinda Gates Foundation and the US Agency for International Development (USAID). I estimate that I have designed and evaluated over 200 projects in 28 countries over the last 29 years. I have felt lucky to do this work, and foreign aid does achieve some very good work while we are there: helping farmers to farm better, or helping men and women to care for their family’s health, wealth, and future with new knowledge, tools, and items they need for daily living. My colleagues do wonderful work as well, in hard conditions, within countries with few resources, and for donors with unrealistic expectations of how much can be done well in short timeframes. In 2010, USAID stated that they would aim for 30 percent of funding to be spent by national partners under USAID Forward.1 This is an excellent step toward the country-led development that the Paris Accords promised, yet as of 2016, there is no list of local partners, other than a handful of examples.2 The only ‘country partners’ list posted to the USAID website includes 80 organizations doing programming via USAID in Afghanistan alone, 55 of which are American firms, four US agencies, nine Afghan government-affiliated organizations, six foreign governments, six UN agencies, and two MENA firms.3 Not quite the national civil-society-NGO partners we envisioned in 2010. Under the new U.S. administration, these are likely to shrink even more as the 0.5 percent of our GNP we allocate to foreign aid is redirected inward—that much more reason to make it as sustained as possible. European aid as well as other rising world nations need this approach just as much.
Large parts of international aid system remain broken. We design too many projects outside of the countries themselves. We have fixed funding and leave in pre-set times rather than when participants are actually ready to take over. We ‘handover’ without partnering throughout the whole project so that partners can determine what they are able to sustain. Even worse, we leave and do not look back to learn from our Ediths and Juans after our projects have closed. Sometimes, we disparage their knowledge, and at other times we don’t make enough time to ask but wish we did. Mostly, our aid industry is designed around measuring success while we intervene, and then abruptly leave because funding ended. Yet development is, as international evaluator Ian Davies says, “A process, not a result.”
Our policies say we are doing “sustainable development”, that we are helping our ‘beneficiaries’ (really our partners and participants) feed themselves over the long-term, that our projects are almost all successful, and that all we need to do is to scale up the great projects out there. But the numbers prove we are not, in fact, achieving sustainable development. Nine times out of ten, we rarely go back to talk to our participants and partners after our project end, and we move on.
The numbers are staggering:
Of the US$5 trillion dollars of international foreign aid spent since 1945, we have evaluated the long-term sustainability far less than one percent of the time.
Since 2000, for example, USAID and the Millennium Challenge Corporation have only done three such evaluations apiece, yet they spent well over US$300 billion.
The EU evaluated only a few dozen of its projects and programs, in spite of spending US$1.5 trillion in the last 15 years. The United Nations Development Program may do up to six a year, and the World Bank more, but how often do any of them talk to project participants and design anew based on what we learned that succeeded and failed?
The Japan International Cooperation Agency, and to some degree EU bilateral countries (through the Organization for Economic Cooperation and Development), have evaluated the sustainability of over 300 projects.
Despite this, tens of thousands of new projects are launched every year.
Satellites atop homes in a slum in Tigray, Ethopia.
This is why I founded Valuing Voices—to analyze what little we know to make development better.4 Not to destroy international development, but to change how we fund, design, and implement it. We need to design for sustainability of the activities by the country nationals themselves, rather than designing for results we can show to get more funding. We also must jointly implement, monitor, and evaluate our projects so countries can continue after we leave. Smaller organizations can do even simple activities, designing projects based on what the participants feel they can self-sustain, and partnering with those who will take over while they are still there.
Having spread the word for the last three years, to mostly little response, I now turn to you, readers. Our analysis – and a wonderful 2012 book, Time to Listen by Mary Anderson and Dayna Brown, shows that sometimes when our projects partner with country nationals, their people become – and stay – better off.5They want to be engaged, yet our very structure of delivering aid prevents this. Often we are not there long enough to make a lasting difference, or we invest scarce time on untested innovations that work in some places but don’t in others. Even worse, sometimes we design activities so badly that villages are left with irrelevant technology and trainings, wasted funding, and lost hopes. At other times, there are successes as well, but not returning robs us of the chance to replicate those. We do ‘impact evaluations,’ but only on successes during project implementation, and not on what people can self-sustain after we leave. Our vision is so limited. Our well-meaning self-interest blinds us.
Across the board, our development projects make one massive and incorrect assumption that once we ‘handover’ the project, the local government, community, and households have the means to sustain our multi-million dollar investments.6 We assume that technical knowledge will still be locally available to the villagers, that inputs like seeds and tools, data and vaccines will be accessible both physically and financially, that the government staff have the means to get to villages or that new NGOs and donors will appear to fill the gaps. The Huffington Post has stated, “as long-term projects and action-plans are established, more investment must go into financing locally designed solutions and projects that ensures ownership is placed back into local communities.”7 While more project have begun using feedback loops of listening to participants during implementation, virtually all good work stops when project funding stops.
Don’t we want development to be sustained after resources leave, and the opinions of these aid recipients to be heard? Don’t we want the next project to address the needs better? Don’t we, as taxpayers, want to demand that agencies using our tax dollars learn what is really sustainable and what is not? And shouldn’t we demand that all projects costing more than US$1 million over the past 10 years be examined now for lessons learned by sector (agriculture, health, credit, education, etc) and region? Shouldn’t post-project sustainability evaluations be included in all new projects? Don’t our participants and partners deserve the dignified futures they hope for, our creating channels for their voices that enable them to evaluate us and teach us how we can help them to be successful?
In fact, a radical Foreign Aid Transparency Act was just passed in the U.S. in June 2016.8 The bill calls for the President, within 18 months of enactment, to “set forth guidelines…for the establishment of measurable goals, performance metrics, and monitoring and evaluation plans that can be applied with reasonable consistency to covered United States foreign assistance.” These include ‘ex-post’ (sustainability) evaluations, and “can have enormous value when it comes to making programming and budgeting decisions.” Yet while there is a call for guidance to be developed, no funding came along with this bill. Without the funds to make this happen, this may be more ‘window dressing’ for sustainable development than excellent policy.
A small enterprise in Cap Haitien, Haiti.
There is some hope coming from the corporate sector. While impact investors are often more focused on return from emerging economies than fostering sustainable development, corporate social responsibility is building bridges in lovely ways. Tsikululu Social Investment of South Africa has thought about what advice to give to the companies they advise on such investments, as well as exiting from them.9
We argue that our budgeting needs a basic business metric: Return on Investment. In a time of huge demands on our resources worldwide from refugee flows, terrorism and climate change, we currently do little or no analysis of:
How much actual investment: What percent of allocated funds went to the activities that benefitted the partners and participants themselves, rather than being used as overhead for operations?
How much return: What is the value of what remains used 3-10 years after we leave? What was the value-added that communities and other funders (including the national governments themselves) who followed catalyzed based on our earlier investments? What were unexpected new results that emerged?
We envision a beautiful future, one where Edith, Aminata, Rituu, and Juan and their local partners are at the center of development. We imagine a world in which we listen to what people in need can sustain for themselves. Through these approaches data is shared widely on what has worked best and why; aid projects invest in country-systems and staff that boost their ability to self-sustain; and only sustainable projects are designed and funded that foster country-led development. The global adoption of the Sustainable Development Goals finally puts the focus on all we can do to foster sustainability of our work.
By valuing voices and focusing on sustainable solutions for excellent impacts, this will promote truly sustainable development from our aid organizations, government and non-governmental alike. We have much to learn, and there is not a moment to waste before we start Valuing Voices of those we serve and partner with on country-led development.
Jindra Cekan, Ph.D. has used participatory methods for 30 years to connect with participants, ranging from villagers in Africa, Central/ Latin America and the Balkans to policy makers and Ministers around the world for her international clients. Their voices have informed the new Sustained and Emerging Impacts Evaluation, other M&E, stakeholder analysis, strategic planning, knowledge management and organizational learning.
If you don’t find what you are looking for via the search, categories, or posts above, you can go to the Blog page, scroll to the bottom, and click “previous posts” to go through all of the posts (newest–>oldest).
Many NGOs have actually been started by or for charismatic individuals, tapping in to somebody else’s cause or source of funding, rather than part of a collective if not mass movement in its own right towards a common end. Sometimes, international departing NGOs promote them, as part of “exit-strategies” to demonstrate that they leave something behind to show for their good work and all the money spent. They may have even included a plan for it in their sustainability proposal write-up. In Cambodia the “gravy train” that accompanied
Local NGO “Cambodia National Volleyball League Disabled2” is the sports one. Most interestingly, it is taking a route party by choice, partly by necessity, towards the private sector or “corporate social responsibility” funds in its hope to realise sustainability. CNVLD has earned a worldwide reputation for transforming the self-esteem of disabled athletes. They actually enjoy high standing on the world stage unlike their “able-bodied” compatriots. They play volley-ball, pursue wheel-chair racing and some athletes may well qualify for the 2012 London Paralympics, even if there may be no money to support them to go and compete there.
◄NGO Child Advocates demonstrate “We can do” but will donors let them?
