What should projects accomplish… and for whom?

by | Dec 11, 2014 | Africa, Aid effectiveness, Big data, Effective Philanthropy, Evaluation, Feedback loops, foreign aid, IATI, International aid, International non-profits, Local Participants, Milenium Development Goals, Open Data, Participants, self-sustainability, Sustainability, water

 

What should projects accomplish… and for whom?

 

An unnamed international non-profit client contacted me to evaluate their resilience project mid-stream, to gauge prospects for sustainable handover. EUREKA, I thought! After email discussions with them I drafted an evaluation process that included learning from a variety of stakeholders, ranging from Ministries, local government and the national University who were to take over the programming work about what they thought would be most sustainable once the project ended and how in the next two years the project could best foster self-sustainability by country-nationals. I projected several weeks for in-depth participatory discussions with local youth groups and sentinel communities directly affected by the food security/ climate change onslaught and who benefited from resilience activities to learn what had worked, what didn’t and who would take what self-responsibility locally going forward.

Pleased with myself, I sent off a detailed proposal. The non-profit soon answered that I hadn’t fully understood my task.  In their view the main task at hand was to determine what the country needed the non-profit to keep doing, so the donor could be convinced to extend their (U.S.-based) funding.  The question at hand became how could I change my evaluation to feed them back this key information for the next proposal design?

Maybe it was me, maybe it was the autumn winds, maybe it was my inability to sufficiently subsume long-term sustainability questions under shorter-term non-profit financing interests that led me to drop this.  Maybe the elephant in the living room that is often unspoken is the need for some non-profits to prioritize their own organizational sustainability to ‘do good’ via donor funding rather than working for community self-sustainability.

Maybe donor/funders should share this blame, needing to push funding out, proving success at any cost to get more funding and so the cycle goes on. As a Feedback Lab feature on a Effective Philanthropy report recently stated: “Only rarely do funders ask, ‘What do the people you are trying to help actually think about what you are doing?’ Participants in the CEP study say that funders rarely provide the resources to find the answer. Nor do funders seem to care whether or not grantees are changing behavior and programs in response to how the ultimate beneficiaries respond” [1].

And how much responsibility do communities themselves hold for not balking?  Why are they so often ‘price-takers’ (in economic terms) rather than ‘price-makers’? As wise Judi Aubel asked in a recent evaluation list-serve discussion “When will communities rise up to demand that the “development” resources designed to support/strengthen them be spent on programs/strategies which correspond to their concerns/priorities??” 

 

We can help them do just that by creating good conditions for them to be heard.  We can push advocates to work to ensure the incoming Sustainable Development Goals (post-MDGs) listen to what recipient nations feel are sustainable, more than funders. We can help their voices be heard via systems that enable donor/ implementers to learn from citizen feedback, such as Keystone has via their Constituent Voice practice (in January 2015 it is launching an online feedback data sharing platform called the Feedback Commons) or GlobalGiving’s new Effectiveness Dashboard (see Feedback Labs).

We can do it locally in our work in the field, shifting the focus from our expertise to theirs, from our powerfulness to theirs. In field evaluations can use Empowerment Evaluation. We can fund feedback loops pre-RFP (requests for proposals), during project design, implementation and beyond, with the right incentives tools for learning from community and local and national-level input so that country-led development begins to be actual not just a nice platitude.  We can fund ValuingVoices’ self-sustainability research on what lasts after projects end. We can conserve project content and data in Open Data formats for long-term learning from country-nationals.

 

West.Mali_.TFSI_.Water_.Women_.Well_.Africare.1

 

Most of all, we can honour our participants as experts, which is what I strive to do in my work. I’ll leave you with a story from Mali. in 1991 I was doing famine-prevention research in Koulikoro Mali where average rainfall is 100mm a year (4 inches). I accompanied women I was interviewing to a deep well which was 100m deep (300 feet). They used plastic pliable buckets and the first five drew up 90% of the bucket full. When I asked to try, they seriously gave me a bucket. I laughed, as did they when we saw that only 20% of my bucket was full. I had splashed the other 80% out on the way up. Who’s the expert?

How are we helping them get more of what they need, rather than what we are willing to give? How are we prioritizing their needs over our organizational income? How are we #ValuingVoices?

 

Sources:

[1] The Center for Effective Philanthropy. (2014, October 27). Closing the Citizen Feedback Loop. Retrieved December 2014, from https://web.archive.org/web/20141031130101/https://feedbacklabs.org/closing-the-citizen-feedback-loop/

[2] Better Evaluation. (n.d.). Empowerment Evaluation. Retrieved December 2014, from https://www.betterevaluation.org/plan/approach/empowerment_evaluation

[3] Sonjara. (2016). Content and Data: Intangible Assets Part V. Retrieved from http://www.sonjara.com/blog?article_id=135

 

See how it turned out! Feedback loops for implementation and sustainability

by | Jul 25, 2014 | Aid effectiveness, Ethiopia, Evaluation, Feedback Labs, Feedback loops, Food security, LIvelihoods, Participants, Red Cross, Sustainability

Reposted from Feedback Labs: http://feedbacklabs.org/see-how-it-turned-out-feedback-loops-for-implementation-and-sustainability/

See how it turned out! Feedback loops for implementation and sustainability

VVblog2

When I met Aminata in a central Malian village, she asked me whether I was with the people with the yellow trucks or the white trucks. That was her way of differentiating between development projects. I explained to her I was doing (PhD) research, with neither. She asked me whether they (the trucks) would come back. I couldn’t answer, and therein lies the problem. “Country-led development” begins with communities being involved at every stage of a project. Ongoing community input during project design, implementation, and monitoring is needed for impact, local community ownership and sustainability. Developing “feedback loops” that facilitate two-way communication are key for building cultures of collaboration, adaptation and learning into international development programs. Valuing Voices sees data as another resource to deliver development, beyond serving the needs of donors and international non-profits.

The distance between our intentions and our reality

Too often, data is extracted from communities by development organizations, in order to evaluate how well they fulfilled the project, rather than communitiesevaluating how well international development projects supported community needs. The best projects co-design interventions and monitoring with communities. Too often communities have no mechanism to learn how their feedback during implementation or during evaluations led to real changes. Such feedback leads to community buy-in, as there is proof that their voices matter and that they are co-driving development. Outside of submitting a format report in a PDF to a donor, the development field has consistently inadequately retained the data from projects once they close, much less leave it in-country, and rarely return afterwards for follow-up evaluations. This further reinforces the idea that these communities are being exploited for resources or development experimentation, with no thought to long-term capacity, learning or sustainability.

Ever farther from this vision of collaborative partnership in project conceptualization, design, and monitoring is what happens once these short and long term projects are closed out. Due to budget restrictions and bureaucratic habits, too often the task of sustaining these projects is handed over to local partners, without funding, staff or data continuity. Cursed with “pilot-itis”, development initiatives too often lose sight of achieving scalability and sustainability. ‘Sustainability’ is often incorrectly defined only as whether the specific project got follow-on funding, rather than by asking communities what activities they were able to continue long after the project ends. Communities provide feedback only 1% of the time, missing great learning opportunities to be had from returning to assess these same projects 1-10 years later to see what was expected, unexpected and what could be sustained. We therefore never truly learn what had the greatest prospect for replicability and scaling elsewhere.

How do we really show that we Value Voices?

At Valuing Voices, we believe that community members are our true clients. We have identified two kinds of feedback loops that are needed to make international development far more effective.

  1. Country-Led Project Implementation Feedback Loops: Valuing Voices wants to create standardized methodologies and data collection processes that can be integrated into most international development projects to create feedback loops that continue working long after projects close. Valuing Voices believes key elements to developing collaborative feedback loops are:

     

    1. Identification of different feedback loop methodologies. Based on what is appropriate for the population, Valuing Voices identifies different methodologies to create these loops through a mix of traditional and digital tools. This means explicitly targeting the building, capturing and sharing of feedback during a specific project, to test and document different methodologies and create standard processes and infrastructure.  This includes participatory methodologies and studies such as Empowerment Evaluation and “Who Counts?  The Power of Participatory Statistics, as well as ways of evaluating qualitativeStorytelling. We must of course protect our respondents as well.
    2. Use of a franchise approach to replicate and scale. The identified methodologies can be taught to local and in-country evaluators and development experts who can then “sell” those services to governments, local NGOs, international donors, and the private sector. Valuing Voices is a catalyst for this country-based franchise approach which strengthens national evaluation, capture and learning. It also allows for feedback loops to exist within a program as local evaluators provide feedback on impact and lessons for improvement.  This leads to local empowerment, sustainability and aid transparency.
    3. Use of digital tools- capture once, share forever. Following evaluations on the local level, Valuing Voices then rolls up national evaluations by sector, analyzes them for what’s most sustained, and shares that learning around the world, influencing project design, implementation, funding and empowerment. In addition to generating feedback loops, this valuable feedback is entered into a curated database of findings for comparison across projects such as agriculture, livelihoods, credit, natural resources, health and education. By using structured data analysis, we can compare feedback data and actual behavior. We will look at whether we are capturing the right associated metadata (date, location, project) to contextualize feedback and bring forth lessons our partners can learn from, analyzing similarities and learning from them to improve program quality, organizational learning (of all partners) and sustainability prospects.
  2.  Post-Project evaluation Feedback Loops: The eagerly awaited Local Systems: A Framework for Supporting Sustainable Development was published by USAID in May:

“More ‘ex-post’ evaluations — which are meant to determine the impact of a project after it is completed, sometimes years later… are needed to design and implement projects.”

This is great news, as in the last 24 years, USAID has not published a single post-project evaluation, and the World Bank has only published one. This could indicate that none were written, or none were seen as ‘publishable’.

Post-project (ex-post) evaluations should look at:

  1. The resilience of expected impacts of the project two, five, and ten years after close-out;
  2. The communities’ and NGOs’ ability to self-sustain those activities;
  3. Positive and negative unintended impacts of the project, both immediately in in the long term;
  4. Which activities the community and NGOs felt were successes which could not be maintained without further funding;
  5. Lessons for other projects on what interventions were most resilient which communities valued enough to continue themselves or for which NGOs valued enough to obtain additional funding, as well as what was not resilient.

These evaluations are rarer than snow in Sri Lanka. Feedback loops post-implementation are key to understanding the sustainability of projects and to improve transparency, efficacy and learning. Valuing Voices has a handful of examples of ex-post (post-project) evaluation done by organizations such as Mercy Corps and Plan International as well as bilateral JICA, but shockingly 99.9% of all development projects just don’t go back and check.

Voices to be valued

These are just a few examples of areas where we can value the voices of our communities, to be more effective and impactful with our efforts, and show our respect and value of communities’ input.

The time is ripe for growth, for these voices to be heard more loudly, telling us what they want development to be. Join us in advocating for funding for such feedback, join us as partners in our field sites, as partners in ICT database creation, as voices for communities.

Communities want their voices heard. Sustained development depends on it.

 

 

About the Authors:

 

Jindra Cekan, PhD is the founder of Valuing Voices  at Cekan Consulting LLC. She has worked in international development for 25 years in participatory design, monitoring & evaluation and knowledge management in over 20 countries. Her PhD was in Mali, “Listening to One’s Clients” and she consults to non-profit, public (USAID), foundation and private sectors.
Siobhan Green, MA is the founder of Sonjara, Inc, and a member of ValuingVoices. She has worked in international development since 1992, and specializes in ICTs for development, knowledge management, and technology for monitoring and evaluation.  Her master’s thesis was on “The Internet in Africa: Policy Perspectives and Approaches” in 1997, and she works with USAID and other USG clients, non-profits, and for-profit partners.