by Jindra Cekan | May 16, 2018 | Accountability, Aid effectiveness, CGDev, Evaluation, ex-post evaluation, Feed the Future, Feedback Labs, foreign aid, impact investing, Impact Investors, International aid, international development, Local Participants, Participants, Participation, post project evaluation, post-project evaluation, Return on Investment (ROI), SALT, Sidekick Manifesto, SUSRoi, Sustainability, Sustainable development, Sustained and Emerging Impacts Evaluation, Sustained and Emerging Impacts Evaluations (SEIE), USAID, Valuing Voices
Setting a higher bar: Sustained Impacts are about All of us
Global development aid has a problem which may already affect impact investing as well.
It is that we think it’s really all about us (individuals, wealthy donors and INGO implementers) not all of us (you, me, and project participants, their partners and governments). It’s also about us for a short time.
All too often, the measurable results we in global development aid and Corporate Social Responsibility (CSR) funded projects that last 1-5 years track and report data for two reasons:
1) Donors have Compliance for grantees to meet (money spent, not lost, and results met by fixed deadlines of 1-5 years – look at some of the European Commission Contracting rules) and
2) Fund recipients and the participants they serve are accountable to ‘our’ donors and implementers who take what happened through their philanthropic grants as ‘their’ results.
Both can skew how sustainably we get to create impacts. An example of such strictures on sustainability from USAID. As respected CGDev Elliot and Dunning researchers found in 2016 when assessing the ‘US Feed the Future Initiative: A New Approach to Food Security?‘ the $10.15 billion leveraged $20 billion from other funders for disbursement over three years (2013-16). “We are concerned that pressure to demonstrate results in the short term may undermine efforts to ensure any impact is sustainable…. Unfortunately, the pressure to show immediate results can encourage pursuit of agricultural investments unlikely to be sustained. For example, a common response to low productivity is to subsidize or facilitate access to improved inputs… it can deliver a quick payoff… however, if the subsidies become too expensive and are eliminated or reduced, fertilizer use and yields often fall…..
With so much focus on reporting early and often about the progress in implementing the initiative, there is a risk that it increases the pressure to disburse quickly and in ways that may not produce sustainable results. For example, for 2014, Feed the Future reports that nearly 7 million farmers applied “improved technologies or management practices as a result of U.S. Government assistance,” but only 1,300 received “long-term agricultural sector productivity.” Are the millions of others that are using improved inputs or management practices because of subsidies likely to have these practices sustained? And how likely are they to continue using improved practices once the project ends?”
3) Impact investors stick to the same two paths-to-results and add a new objective: market-competitive financial returns. They also need to show short-term results to their investors, albeit with social, environmental and governance results like non-profits (future blog).
4) Altruists create things we want ‘beneficiaries’ (our participants) to have. For instance a plethora of apps for refugees cropped up in recent years, over 5,000 it is estimated, which can be appropriate, nor not so helpful. Much like #2 above, ‘we’re’ helping ‘them’ but again, it seems to be a ‘give a man a fish’… and my fish is cool sort of solution… but do our participants want/ need this?
How often is our work-for-change mostly about us/by us/ for us... when ideally it is mostly about ‘them’ (OK, given human self-interest, shouldn’t changes we want at least be about all of us?).
All too often we want to be the solution but really, our ‘grassroots’ clients who are our true customers need to generate their own solution. Best if we listen and we design for long-term sustainability together?
As the Brilliant Sidekick Manifesto stated in two of its ten steps:
a) “I will step out of the spotlight: Sustainable solutions to poverty come from within are bottom-up, and flow from local leaders who are taking the risks of holding their politicians accountable and challenging the status quo.”
b) “I will read “To Hell with Good Intentions” again and again: Politicians, celebrities and billionaire philanthropists will tell me that I can be a hero. I cannot. The poor are not powerless or waiting to be saved. Illich will check my delusions of grandeur.”
We have examples of where we have stepped away and participants had to fend for themselves. At Valuing Voices, we’ve done post project-exit evaluations 2-15 years afterward. What did participants value so much that they sustained it themselves (all about them, literally)? These Sustained and Emerging Impacts Evaluations (SEIE) also give us indications of Sustained ROI (Sustained Return on Investment (SusROI) is a key missing metric. As respected evaluator Ricardo Wilson-Grau said in an email, “I think calculating cost-effectiveness of an intervention’s outcomes would be a wonderful challenge for a financial officer searching for new challenges — if not a Nobel prize in economics!”)
Most of these evaluations are pretty bad news mixed with some good news about what folks could sustain after we left, couldn’t and why not. (These are the ones folks expect to have great results, otherwise they wouldn’t share them!) While most clients are understandably interested in what of ‘theirs’ was still standing, and it was interesting disentangling where the results were attributable by implementation or design or partnership flaws or something else, what was mesmerizing was what came from ‘them’.
The key is looking beyond ‘unexpected’ results to look at emerging impacts that are about ‘them’ (aka what we didn’t expect that was a direct result of our project, e.g. spare parts were no longer available to fix the water well pump once we left or a drought rehabilitation water project that decreased violence against women), to what emerging results are attributable not to use but only to our participants and partners who took over after our projects closed. One example is a Nepalese project ended yet the credit groups of empowered women spawned groups of support groups for battered women. Another is a child maternal health project changed how it worked as women reverted to birthing at home after NGOs left; community leaders punished both parents with incarceration in the health clinic for a week if they didn’t given birth there (wow did that work to sustain behavior change of both parents!).
Many of us at Valuing Voices are shocked that funders don’t seem that interested in this, as this is where they not only take over (viz picture, sustaining the project themselves), but they are making it theirs, not ours. Imagine assuming the point of development is to BE SUSTAINABLE.
Source: Community Life Competence
Our participants and national stakeholder partners are our true clients, yet… Feedback Labs tell us Americans alone gave $358 billion to charities (equivalent to the 2014 GDP of 20 countries) – in 2014 but how much of this was determined by what ‘beneficiaries’ want? Josh Woodard, a development expert, suggests a vouchers approach where our true clients, our participants, who would “purchase services from those competing organizations… [such an] approach to development would enable us all to see what services people actually value and want. And when we asked ourselves what our clients want, we would really mean the individuals in the communities we are in the business of working with and serving. Otherwise we’d be out of business pretty quickly.”
This opens the door to client feedback – imagine if participants could use social media to rate the sustained impacts on them of the projects they benefited from? A customer support expert wrote in Forbes, “Today, every customer has, or feels she has, a vote in how companies do business and treat customers. This is part of a new set of expectations among customers today that will only grow ... you can’t control product ratings, product discussions or much else in the way of reviews, except by providing the best customer experience possible and by being proactive in responding to negative trends that come to the surface in your reviews and ratings stronger.”
So how well are we working with our participants for ‘development’ to be about them?
What do you think?
by Jindra Cekan | Apr 12, 2018 | Accountability, Aid effectiveness, Catholic Relief Services (CRS), Demographics, ex-post evaluation, Feedback Labs, Mercy Corps, PCI, post-project evaluation, SDGs, Sustainability, Sustainable development, Youth, youth employment
Investing in Youth for Project Effectiveness and Sustainability
One out of every six people on earth is between the ages of 15-24, says the UN. That is 1.2 billion youth. As one young leader says, “if the world’s problems are to be solved, it’s not going to happen without us.” Yet in 2015, the International Labor Organization said 73.3 million youth between 15 and 24 were unemployed. Not only do and an estimated 169 million young workers lived on less than $2 a day, 75 percent of youth workers are only informally employed. In Africa alone, the UN estimates 200 million are such youth; not only does Africa have the youngest population in the world, this figure will double by 2045, but the largest numbers remain in Asia (IMF 2015). The World Bank has striking African and Asian demographics:
How often do we fund projects that are designed and run by youth? How engaged are youth in sustaining the projects we have funded, designed, implemented, monitored and evaluated? What have we in global development, including corporate social responsibility and investing spheres done to ensure that youth are both engaged in our projects, but are in the leadership to direct how they are done now, and sustaining them beyond donor departure? Further, how well are we collaboratively developing technology with them for them to use to thrive in this sped-up, high-tech world?
Valuing Voices youth blogs covered the barriers to youth success in the ‘developing world’ which included a lack of access to sufficient numbers of jobs, compounded by a lack of job-appropriate skills, access to capital, decision-making etc. We heartily agree with the IMF that “youth have a huge stake in bringing about a political and economic system that heeds their aspirations, addresses their need for a decent standard of living, and offers them hope for the future…. [Also] that communities, cities, provinces, and countries can set up forums for the purpose of listening to the concerns and ideas of adolescents and young adults and stimulating change. Young people could be offered a voice in decision-making bodies…Inclusion can benefit all.”
Why should we make this happen? Taking inclusive steps fosters sustained impacts long after we grow old.
As CRS Niger’s otherwise very successful Sustained and Emerging Impacts Evaluation of the food security project shows us, there is much room to grow in inclusion of ‘youth’ (up to age 35 in Niger), both by including youth:
- The exodus of youth diminished during and after the project [by] using the same land to train 100 new vegetable farmers and trainers. Youth seasonal outmigration decreased due to increased food production, especially due to vegetable gardening even during the dry season, and increased knowledge of practices such as rainfed agricultural [practices] which kept youth locally employed.“
- Youth, too, having learned [agricultural water-conservation] techniques, and generated income [even] while seasonally working outside the village.
Versus not engaging youth:
- Although most committees are still functioning, there are no processes in place to engage and train youth and new inhabitants of the villages [in project activities after close-out]… … there are serious questions about how well they will be engaged and train youth and new members of the communities and how much will be transferred cross-generationally. This is pivotal given that 50% of Nigeriens are under the age of 15;
- The [sustainability] problem was that [youth] were not elected or chosen for the [management] committees. This is another issue to flag in other projects interested in sustainability: the implications of selecting a limited number of elders to staff multiple committees than a broad array of young committee members that could grow into leadership positions. Given the youth’s overall dissatisfaction with group leadership, other projects need to be aware of “elite capture” and its potential threat to sustainability
Investing in youth is a terrific investment in sustainability. How often do we consider it?
A post-project example from Mercy Corps/ PCI’s early ex-post evaluation in Central Asia showed that such investments are not easy but can pay off after the project closed. “72% of youth report that they continue to use at least one skill they learned during the [infrastructure] program”… including teamwork and communication, sewing, construction, roofing, journalism and cooking.” This may have been in part due to the project’s youth summer camps, organized each year to promote youth leadership and participation in community decision-making, which were supported by [some of] the adult population. While the project “encouraged communities to elect young people as representatives…within the cultural context this was not met enthusiastically by the communities because young people were not felt to be ‘qualified’ as leaders.” Yet inter-generational collaboration was fostered by the project by establishing mentoring programs where older people with technical skills mentored [some] young people during the infrastructural construction activities.”
Raj Kumar, Founder of Devex and chair for the World Economic Forum said this about what to do post-Davos: “With a dozen years to go before the finish line of the Sustainable Development Goals, we need to get the underlying plumbing right in order to have a chance to reach those goals. That plumbing includes everything from having the country-level data to track progress against the goals to having the project-level data to know what’s working and what’s not…. Most importantly, it’s about the development leaders of today building out the best systems so the development leaders of tomorrow can focus on delivery.”
What we at Valuing Voices are most encouraged by is the prospect of overtly considering sustained impacts to inform the funding, designing, implementing, monitoring and evaluating of projects today for the adult millennials of tomorrow. Regarding youth, we will need a mix of focused initiatives such as longstanding work by the International Youth Foundation, and new investments funds such as the Global Youth Empowerment Fund and integrating youth into projects at all stages of the projects and beyond, as we showed regarding CRS and Mercy Corps/PCI, above.
That is one way to get SusROI (Sustainable Return on Investment).
As mother who has worked in 26 countries, I feel the great urge to harness youths’ yearning to succeed through their love of technology. The growth of mobile money in Africa is one example of technology use in daily life. Who is supporting youth employment in technology? Mercy Corps’ 2017 Social Ventures Fund that supports “positive trends offered by technology. Trends in micro-work, micro-manufacturing, digital livelihoods and mobile-enabled agent networks are paving the way for the acceleration of a distributed and digitally enabled workforce and the reinvention of manufacturing, sales and distribution. Their investments related to youth employment are:
- NewLight Africa – network of rural sales and customer service agents
- Wobe – anyone with an Android phone in Indonesia can be a micro-entrepreneur
- Lynk – job-matching platform
- Sokowatch – network of urban sales and customer service agents
I dream of youth crowd-sourcing post-project sustained impact results. Feedback Labs has a lot of really interesting tools for… feedback from people on the ground in-country. I searched high and low and found only this crowdsourcing data collection overview of three aid research tools for ICT4D (information and communications technology for development), the best of which appears to be Findyr. (FYI here is feedback on the limitations of crowdsourcing in emergencies.)
Also, the ‘impact tracking’ platform by Makerble looked good but who among you have a wider perspective to advise us what’s best?
Finding technologies and funding to hear youth’s voices and feedback on what they could sustain or could not after our projects closed, and why is unbelievably valuable to inform funding, design, implementation, M&E, and of course foster youth empowerment. Good listening to participants comes first. As an impact evaluation in Uganda found that “when villagers and teachers, instead of school officials, are allowed to set their own priorities for improving schools and directly monitor performance, the results can be priceless. In Uganda, World Vision knew that community-based monitoring of school performance could help sustain improvements in education that building schools, supplying textbooks, and training teachers alone could not. They tried two approaches: the use of a standard scorecard with performance questions identified by education officials and development partners, and a participatory scorecard, where community members defined the issues they would monitor. A randomized controlled trial [RCT] revealed that the participatory scorecard delivered more than the standard scorecards. The participatory approach prompted higher efforts by teachers, as expected. But it also prompted higher efforts from villagers— local politicians learned more about their country’s education policies and what they could advocate for on behalf of their constituents, parents increased their support of schools by contributing to midday meals, and children found a forum to report teacher absenteeism and other factors that hurt their education. In the end, while the standard scorecard made little difference in school performance, the participatory approach improved attendance by teachers and students and helped raise student test scores.’”
By accessing mobile technology, ground-truthing project sustainability, given youth’s familiarity with technology and network-interconnected habits, I believe together we can cost-effectively democratize evaluations and help ‘development’ be ‘sustainable’. Collaborate with us!
by Jindra Cekan | Oct 29, 2015 | Accountability, Feedback Labs, Feedback loops, Fund for Shared Insight, Giving Evidence, Hewlett Foundation, Innovations for Poverty Action, International aid, Keystone, Participation, VOTO Mobile
The Disruptive Potential of Feedback
by MARC GUNTHER, OCTOBER 18, 2015 ( reblogged http://nonprofitchronicles.com/2015/10/18/the-disruptive-potential-of-feedback/)
Few institutions in the US are as undemocratic as endowed foundations. The executives in charge of foundations answer to, er, no one. They give money away, so people tend to laugh at their jokes, tell them they look well, nod in agreement at their banal remarks. What’s not to like?
As for nonprofits, they pay heed to foundations and donors, but they need not listen to their “beneficiaries,” unless they feel a moral obligation to do so. What if, goodness knows, the people they are trying to serve turn out to be unhappy with the service? Talk about inconvenient truths.
Last week in Washington, a group of about 70 people — the generals and foot soldiers of a growing movement to devolve power to mostly poor recipients of aid in the US and abroad — came together to talk about how to turn that power dynamic of philanthropy upside down. They believe that feedback from constituents “has the potential to unleash massive, timely and necessary changes in the way social change and development are pursued,” in the words of Feedback Labs, a DC-based NGO that convened the firstFeedback Summit.
As Dennis Whittle, the executive director of Feedback Labs, has written:
Will aid and philanthropy democratize themselves? Will aid agencies and foundations cede power and sovereignty to the people they are trying to serve?
It’s too soon to say but there were signs during the two-day confab that a half-dozen or so forward-thinking foundations, along with a growing number of nonprofits, are starting to figure how to create tight feedback loops that will enable them to solicit feedback from citizens, listen, analyze and, most important, change their practices as a result of what they learn.
“It’s the right thing to do, morally and ethically, philosophically. It’s the smart thing to do,” Whittle said. Now the goal is to make it “the feasible thing to do, financially and operationally.”
How do feedback loops differ from conventional monitoring and evaluation (M&E)? One attendee told me that feedback loops are the equivalent of diagnosing and treating a disease; a conventional evaluation is more like an autopsy, and thus of limited value to the patient.
Here are three signs that the feedback movement is gathering momentum:
A group called the Fund for Shared Insight, a collaboration of foundations that makes grants to improve philanthropy, has launched an initiative called Listen for Good that intends to fund 50 nonprofits to seek feedback from the people they are trying to help. They will use the now-famous Net Promoter System methodology developed for business by Bain & Co., which is working with the fund to make sure that the simple, elegant and yet rigorous system is deployed effectively. “It needs to be a high velocity loop of feedback, learning and action,” said Vikki Tam, a Bain partner. To the extent possible, the feedback results will be made public, enabling nonprofits to compare their net promoter scores with peers. “It’s so important for foundations to be open about what they do, and what they’re learning,” said Lindsay Austin Louie, a program officer at William and Flora Hewlett Foundation who works closely with the fund. Supporters of the fund include the David and Lucile Packard Foundation, the Ford Foundation, the Gordon and Betty Moore Foundation, the JPB Foundation, Liquidnet (which I wrote about here), the Rita Allen Foundation and the W.K. Kellogg Foundation.
Efforts to build “good enough” feedback loops are underway, aimed at helping small or midsize NGOs measure their impact without having to undertake expensive, long-term randomized control trials. Thoai Ngo, a senior director of research at Innovations for Poverty Action, talked about an effort called The Goldilocks Project that aims to build “right-fit” evaluation systems, focusing on collecting credible, actionable data in a timely way–that is, feedback to help an NGO change course if needed. Ken Berger, the former chief executive at Charity Navigator, described his work at a firm called Algorhythm which offers impact measurement to small NGOs for as little as $750. “These are organizations that never before had an opportunity to measure what matters most,” Berger said.
Technology is making it much easier to gather feedback, and make sense of it. Louis Dorval is the co-founder of VOTO Mobile, a Ghana-based tech startup that aims to “amplify the voice of the under-heard” by using voice and text messages on mobile devices to survey citizens, as well as send one-way messages. Less than three years old, Voto Mobile has already worked with about 250 organizations, including Unicef and Innovations for Poverty Action. David Bonbright of Keystone Accountability, who is a pioneer in the feedback arena, talked about Feedback Commons, an online platform designed to allow organizations to “share and compare” the feedback they collect from their constituents.
SUPPLY AND DEMAND
All these initiatives are designed to improve the development of feedback loops. Before long, NGOs should be able to show how collecting feedback generates better outcomes for their clients. That’s all to the good. Think of that as the supply side of the feedback “business.”
But what about the demand side? Who’s going to fund feedback loops? I’m still a newcomer to the development world but my impression (from reporting on water and sanitation projects, and on cookstoves) is that most foundations and NGOs are not as rigorous as they could and should be about measuring their impact.
This brings us back to the fundamental power dynamic of philanthropy, as Caroline Fiennes of Giving Evidence explained during the Feedback Summit.
“Why don’t foundations ask for feedback?” Fiennes asked. “Because they don’t have to.”
“It’s difficult and it’s a bit painful,” she went on. “If you have made a chunk of money and you want to give it away, in general you will feel good about that, and everybody will love you. Once you start asking questions” — questions designed to find out if the work is making as much of a difference as it can–“you might not like the answers.”
In an essay called What do they want? at Aeon, Claire Melamed elaborates:
While most individual aid workers do care, very much, about the people they work with and for, the actual structure of the aid business offers few reasons for anyone to worry about what aid recipients think or want. Staff in aid agencies need to think about what their funders want to pay for. For their own performance reviews, they need to think about how to demonstrate that what they are doing is achieving the best possible results with the smallest amount of money. So the incentives for spending money on expensive surveys to find out what representative samples of poor people think of their operations are just not there.
And besides, the information might be a threat. What if it turned out that people feel patronised by aid workers? Or that they would rather their food didn’t arrive with logos announcing their indebtedness to foreign governments? Or that they resent being given a T-shirt when really they would sooner just have the money? What if people don’t really want another agricultural programme, and they’d rather have a bus ticket to the nearest town and somewhere to stay when they get there? These kinds of discoveries could be quite discomfiting for the agencies themselves – though in the long run, they would presumably do a better job.”
This is why those funders (like the foundations behind the Fund for Shared Insight) who push for feedback loops and rigorous evaluation deserve a lot of credit. Let’s hope their numbers grow.
by valuingvoicesjin | Jul 25, 2014 | Aid effectiveness, Ethiopia, Evaluation, Feedback Labs, Feedback loops, Food security, LIvelihoods, Participants, Red Cross, Sustainability
Reposted from Feedback Labs: http://feedbacklabs.org/see-how-it-turned-out-feedback-loops-for-implementation-and-sustainability/
When I met Aminata in a central Malian village, she asked me whether I was with the people with the yellow trucks or the white trucks. That was her way of differentiating between development projects. I explained to her I was doing (PhD) research, with neither. She asked me whether they (the trucks) would come back. I couldn’t answer, and therein lies the problem. “Country-led development” begins with communities being involved at every stage of a project. Ongoing community input during project design, implementation, and monitoring is needed for impact, local community ownership and sustainability. Developing “feedback loops” that facilitate two-way communication are key for building cultures of collaboration, adaptation and learning into international development programs. Valuing Voices sees data as another resource to deliver development, beyond serving the needs of donors and international non-profits.
The distance between our intentions and our reality
Too often, data is extracted from communities by development organizations, in order to evaluate how well they fulfilled the project, rather than communitiesevaluating how well international development projects supported community needs. The best projects co-design interventions and monitoring with communities. Too often communities have no mechanism to learn how their feedback during implementation or during evaluations led to real changes. Such feedback leads to community buy-in, as there is proof that their voices matter and that they are co-driving development. Outside of submitting a format report in a PDF to a donor, the development field has consistently inadequately retained the data from projects once they close, much less leave it in-country, and rarely return afterwards for follow-up evaluations. This further reinforces the idea that these communities are being exploited for resources or development experimentation, with no thought to long-term capacity, learning or sustainability.
Ever farther from this vision of collaborative partnership in project conceptualization, design, and monitoring is what happens once these short and long term projects are closed out. Due to budget restrictions and bureaucratic habits, too often the task of sustaining these projects is handed over to local partners, without funding, staff or data continuity. Cursed with “pilot-itis”, development initiatives too often lose sight of achieving scalability and sustainability. ‘Sustainability’ is often incorrectly defined only as whether the specific project got follow-on funding, rather than by asking communities what activities they were able to continue long after the project ends. Communities provide feedback only 1% of the time, missing great learning opportunities to be had from returning to assess these same projects 1-10 years later to see what was expected, unexpected and what could be sustained. We therefore never truly learn what had the greatest prospect for replicability and scaling elsewhere.
How do we really show that we Value Voices?
At Valuing Voices, we believe that community members are our true clients. We have identified two kinds of feedback loops that are needed to make international development far more effective.
-
Country-Led Project Implementation Feedback Loops: Valuing Voices wants to create standardized methodologies and data collection processes that can be integrated into most international development projects to create feedback loops that continue working long after projects close. Valuing Voices believes key elements to developing collaborative feedback loops are:
-
Identification of different feedback loop methodologies. Based on what is appropriate for the population, Valuing Voices identifies different methodologies to create these loops through a mix of traditional and digital tools. This means explicitly targeting the building, capturing and sharing of feedback during a specific project, to test and document different methodologies and create standard processes and infrastructure. This includes participatory methodologies and studies such as Empowerment Evaluation and “Who Counts? The Power of Participatory Statistics, as well as ways of evaluating qualitativeStorytelling. We must of course protect our respondents as well.
-
Use of a franchise approach to replicate and scale. The identified methodologies can be taught to local and in-country evaluators and development experts who can then “sell” those services to governments, local NGOs, international donors, and the private sector. Valuing Voices is a catalyst for this country-based franchise approach which strengthens national evaluation, capture and learning. It also allows for feedback loops to exist within a program as local evaluators provide feedback on impact and lessons for improvement. This leads to local empowerment, sustainability and aid transparency.
-
Use of digital tools- capture once, share forever. Following evaluations on the local level, Valuing Voices then rolls up national evaluations by sector, analyzes them for what’s most sustained, and shares that learning around the world, influencing project design, implementation, funding and empowerment. In addition to generating feedback loops, this valuable feedback is entered into a curated database of findings for comparison across projects such as agriculture, livelihoods, credit, natural resources, health and education. By using structured data analysis, we can compare feedback data and actual behavior. We will look at whether we are capturing the right associated metadata (date, location, project) to contextualize feedback and bring forth lessons our partners can learn from, analyzing similarities and learning from them to improve program quality, organizational learning (of all partners) and sustainability prospects.
-
Post-Project evaluation Feedback Loops: The eagerly awaited Local Systems: A Framework for Supporting Sustainable Development was published by USAID in May:
“More ‘ex-post’ evaluations — which are meant to determine the impact of a project after it is completed, sometimes years later… are needed to design and implement projects.”
This is great news, as in the last 24 years, USAID has not published a single post-project evaluation, and the World Bank has only published one. This could indicate that none were written, or none were seen as ‘publishable’.
Post-project (ex-post) evaluations should look at:
-
The resilience of expected impacts of the project two, five, and ten years after close-out;
-
The communities’ and NGOs’ ability to self-sustain those activities;
-
Positive and negative unintended impacts of the project, both immediately in in the long term;
-
Which activities the community and NGOs felt were successes which could not be maintained without further funding;
-
Lessons for other projects on what interventions were most resilient which communities valued enough to continue themselves or for which NGOs valued enough to obtain additional funding, as well as what was not resilient.
These evaluations are rarer than snow in Sri Lanka. Feedback loops post-implementation are key to understanding the sustainability of projects and to improve transparency, efficacy and learning. Valuing Voices has a handful of examples of ex-post (post-project) evaluation done by organizations such as Mercy Corps and Plan International as well as bilateral JICA, but shockingly 99.9% of all development projects just don’t go back and check.
Voices to be valued
These are just a few examples of areas where we can value the voices of our communities, to be more effective and impactful with our efforts, and show our respect and value of communities’ input.
The time is ripe for growth, for these voices to be heard more loudly, telling us what they want development to be. Join us in advocating for funding for such feedback, join us as partners in our field sites, as partners in ICT database creation, as voices for communities.
Communities want their voices heard. Sustained development depends on it.
Jindra Cekan, PhD is the founder of Valuing Voices at Cekan Consulting LLC. She has worked in international development for 25 years in participatory design, monitoring & evaluation and knowledge management in over 20 countries. Her PhD was in Mali, “Listening to One’s Clients” and she consults to non-profit, public (USAID), foundation and private sectors.
Siobhan Green, MA is the founder of Sonjara, Inc, and a member of ValuingVoices. She has worked in international development since 1992, and specializes in ICTs for development, knowledge management, and technology for monitoring and evaluation. Her master’s thesis was on “The Internet in Africa: Policy Perspectives and Approaches” in 1997, and she works with USAID and other USG clients, non-profits, and for-profit partners.
