Learning from the Past… for Future Sustainability

by | Feb 16, 2015 | Bolivia, Catholic Relief Services (CRS), Ethiopia, ex-post evaluation, Exit strategies, Food for Peace (FFP), Food security, Honduras, India, International aid, Kenya, post-project evaluation, Project design, self-sustainability, Sustainable development

Learning from the Past… for Future Sustainability

Heading up Food Security for Catholic Relief Services (CRS) was my first international development job in 1995-1999 and I have watched this organization grow in its commitment to program quality and learning/ knowledge management ever since.  At the time I oversaw 17 of of CRS' USAID/ Food for Peace (FFP) programs.  So I was delighted that not only has CRS done an ex-post evaluation and used the findings for programming (e.g. the effectiveness of investing in a particular sector—for example, the importance of supporting girls’ education within a food security program) and also for advocacy (e.g. evaluation lessons from Rwandan peace-building projects seven years after the genocide informed CRS’ evolving approach to peace and justice strategies), but I get to celebrate FFP learning too.  In addition to having consulted to USAID/PPL (Policy, Planning and Learning) and the FANTA project, all featured below, I went to Tufts University’s Fletcher School. Super to see great organizations learning about sustainability!

CRS’ 2007 project package guidance for implementation and guidance (ProPackII), described ex-post evaluation/sustainable impact evaluation’s aim “to determine which project interventions have been continued by project participants on their own [which] may contribute to future program design…. it is fair to say that NGOs rarely evaluate what remains following the withdrawal of project funding [which] is unfortunate [as] important lessons can be generated regarding factors that help to ensure project sustainability.”

A 2004 Catholic Relief Services excellent ex-post evaluation in Ethiopia was featured: “Looking at the past for better programming: dap I Ex-Post Assessment Report”. It assessed sustainability of Agriculture Natural Resource Management, and Food-Assisted Child Survival/Community Based Health Care programming, done as an internal evaluation by CRS staff and partners with document review, partner, government and community interviews. Results were mixed.

  • Some activities generated enough food and income that households could eat throughout the year and have some savings, making them more resilient against drought
  • Almost 100% of cropland bunding and irrigation practices for improved crop production were still being applied and buffered them during a subsequent drought
  • Health practices had also continued (e.g. trained traditional birth attendants had continued to provide services with high levels of enthusiasm and commitment, and increased levels of health care-seeking behaviors existed).

However, many other benefits and services had severely deteriorated:

  • Nearly all water committees had dissolved, fee collection was irregular or had been discontinued
  • Many water schemes were not operational
  • The centrally managed [tree] nurseries had been abandoned (given the existing management capacity of communities and government).

CRS/Ethiopia and its partners came to see that:

  • “The potable water strategy had over-focused on the technical aspects (“hardware”) while not paying enough attention to the community organizing dimensions and support by existing government services (“software”).
  • Even limited post-project follow-up by partners and government staff might have gone a long way towards mitigating the deterioration of project benefits and services.

What was terrific was that they “went on to use these findings and lessons learned from this ex-post evaluation to inform the design of similar projects in Ethiopia… while also raising awareness of these issues among partner staff”. The ex-post recommended increased planning for sustainability, setting up village management for post-project and incentive maintenance.  Great learning, yet we have found few ex-posts at CRS or elsewhere.  Our industry needs to explore issues such as those the evaluators posed: Was the lack of sustainability due to technical, institutional or financial faults in the programming? In other words, was the lack of self-sustainability due to the design/ aim of the activity itself or how it was implemented?

In 2013, USAID’s Food For Peace commissioned fascinating research on Exit Strategies.  Tufts University went to Bolivia, Honduras, India and Kenya which were phasing out of Title II food aid to look at how to “ensure that the benefits of interventions are sustained after they end, [as] there is little rigorous evidence on the effectiveness of different types of exit strategies used in USAID Office of Food for Peace Title II development food aid prog

rams.” The research is to “assess the extent to which the programs’ impacts were maintained or improved and to help understand factors of success or failure in the specific exit strategies that were used.” They have made the important discernment that the effectiveness of Title II programs depends on both short-term impact and long-term sustainability.

FANTA_Exit_Strategies_Bolivia.pdf

The FANTA project (contractor) made the following preliminary results available:

  1. Impact assessment at exit does not consistently predict sustained impact two years later…. It can be misleading.
  2. Many activities, practices, and impacts across sectors declined over the two years after exit. These declines are related to inadequate design and implementation of sustainability strategies and exit processes.
  3. There are specific ways to increase the likelihood of sustainability: Sustaining service provision and beneficiary utilization of services and practices depends on three critical factors: Resources, Technical and Management Capacity, Motivation
  4. Withdrawal of free food rations or any other free input (as incentive) jeopardizes sustainability without consideration of substitute incentives. For instance,

     

     

    • Withdrawal of food was a disincentive for participation in and provision of [child] growth monitoring…. Resources and health system linkages are needed to sustain health activities
    • Motivation, capacity and resources are all needed to maintain water systems
    • Agriculture and Natural Resource Management suffered greatly when resource incentives disappeared

Their main recommendations are that sustainability should be built into the design from the beginning, program cycles are longer and exit is gradual.

CRS found the same issue of incentives as a barrier, as they did technical and (institutional) capacity/ motivation/ management issues.  We have much to learn… at least we’ve started Valuing Voices and asking… and eventually designing for sustainability!

Ideas we’re not ready for are the ones we need most

by | Dec 16, 2014 | Evaluation, India, Industrial Revolution, International aid, Shrujan, South Africa, Sustainable development, Umzi Wethu, Valuing Voices

Ideas we’re not ready for are the ones we need most

Guest Post: Barbara Geary Truan

 

Some ideas precede and shape reality, the ones that lie behind everything we create. Other ideas lag behind an often hidden reality: they seek then find it. Galileo Galilei observed the skies for years before coming up with the notion that the earth was revolving around the sun. Both kinds of ideas have something essential to offer development professionals today.  

Galileo

The first tells us that if we can envision a world without poverty, we can create it. But it also begs the question of why, in spite of the last 50 years of development work, more than a billion people today live on less than $1.25, putting them below the internationally accepted norm of extreme poverty. This, I think, is where the second kind of idea comes in.

 

If we gaze at the development skies, we might begin to sense a centre around which both poverty and wealth rotate; we might come to recognise these opposites as distinct faces of the industrial revolution and the mechanistic thinking that underpins it.

 

With machines (technology, automatic process) as the ground of our systems, the reigning values are: mass production, identical forms, standard measurements. And the requirement is: unlimited consumption, both of raw materials and finished products. An unintentional result is that the system tends to overlook and undervalue a whole set of powerful variables and realities – those which are distinctly human.

 

What would happen if we allowed development thinking to cut itself loose from the tentacles of a mechanised view of the world? What if, for example, we introduced a quality like humility – the kind of humility Galileo experienced when gazing at the skies – into our thinking? What if we concentrated as much on what we don’t know as on what we do know? What if we threw out the molds and conveyor belts of development initiatives and decided to let it be an exclusively human endeavour?

 

I think if we had the courage required for such an undertaking, three exciting things would happen:

  1. We would shift the order of the “five Ws”: what, where and when would play second fiddle to WHO and WHY. To stop assuming that these last two questions always have the same generic answers (“poor people” and “we want to help”) can be powerful. These assumptions become labels and labels become reductionist. Making them more specific opens wide two windows of possibility we often forget are there. A specific and thus enriched “who” automatically introduces collaboration, partnership and diversity into our work. A deeply specific, clearly articulated “why” has the potential to keep it honest.

2. We would shift our thinking about time: when time becomes human, we notice two things. First, we get to be in charge of it. Paying attention to detail, allowing time for deep thinking while planning, or shifting time frames to accommodate reality during a project, all give us sovereignty over time. Secondly, we would notice that creating wealth and health where poverty and illness have installed themselves takes time. Put simply, development projects that take a long view are the only ones that really work because, as in any relationship, commitment is a game changer. 

3. We would see a different picture: when we allow development to be human, we begin to see and understand the human beings who have the most at stake in the process. And when we meet and know them, we look beyond the mere context of their lives to see how they live their lives. When we do this we see vast – seemingly unlimited – amounts of creativity, ingenuity, intelligence, courage, fortitude, to name just a few notable human qualities for which there are no standard measures.

 

These ideas aren’t theoretical. A few of the finest development initiatives I know of embody the human approach. In northwestern India, in a geographically isolated region with limited resources, an organisation called Shrujan has been quietly creating wealth and helping artisans sustain themselves while nurturing an integral part of their culture for almost fifty years. In South Africa, a project known as Umzi Wethu has come up with an original, dynamic and transferrable model for completely turning around the lives of vulnerable youth.

 

There are other examples like these. Valuing Voices exists to celebrate them and to encourage development professionals to look at them carefully and imitate what makes them so successful. Such work whispers of an unmoving sun, nudging us to imagine that there just may be another centre somewhere else in the sky: maybe even right in the heart of the enormous potential of what we label “poor”. 

 

When Jindra invited me to write this blog, she mentioned that at a recent conference on development monitoring and evaluation, someone told her that Valuing Voices was an idea for which people were not yet ready. Hurray, I thought. That means she must really be on to something great.

 

 

Barbara Geary Truan holds a master’s degree in international relations from the Fletcher School. She worked for six years at the International Federation of Red Cross and Red Crescent Societies, and for over a decade as an evaluator for a philanthropical initiative that supports original working projects designed to better the world.