Leading in Challenging Times: Sustained and Emerging Impacts Evaluation (SEIEs)- reposted from Medium.com

 

Leading in Challenging Times:
Sustained and Emerging Impacts Evaluation (SEIEs)

 

Some American organizations are retrenching, focusing more attention on domestic rather than international programming. Some are pulling back from critique of international development to informing legislators of its benefits; the Center for Global Development’s changed ‘Rethinking US Development Policy’ blog to only “US Development Policy“. UN’s Refugee Agency questions whether to challenge Washington’s tough line on refugees from countries such as Syria, or should it stay quiet in the hopes of protecting its funding [1]?”

Reticence is understandable in this ‘climate’, so to speak, but fear does not change the world, leadership does. Envisioning and creating the world we want gets us there.

There may be no better time to build the evidence base on what works in sustainable development as these are low cost investments if we use national staff and focus research well. We have seen this in the fewer than 1% of all projects that have been evaluated post-closeout for sustainability [2]. At the very least, we can learn what we should do differently in the next design, to fully foster sustainability, once more funding emerges. Many are interested in great results. Hundreds of ‘impact evaluations’ are happening on aid effectiveness; our industry wants to learn what works and what we could do better.

Our SEIE work goes beyond current understanding of ‘impact’ to see what projects our partners and participants can self-sustain ex-post for years to come which is an excellent investment in proving cost-effectiveness. While some governments’ investments can diminish in the short term, national governments, and other funders such as a range of international bilateral and multilateral donors, foundations corporate social responsibility and impact investors do want to invest in provably “sustainable” development [3].

 

Why should we invest in SEIEs?

  • Hundreds of thousands of projects are still being implemented.
  • Millions of participants are still hoping what we are doing together will be sustainable.
  • Billions of dollars, euros, kwacha, pesos, rupees are being spent on new projects that need to be designed and implemented for future sustainability.

 

Implementing organizations could be fearful to see what remains once funding and technical assistance are withdrawn, but such a view not only robs our industry of exciting lessons on what did change and was so valued that it was sustained, but also what to not do again. Not returning post-project also short-changes our participants. In our SEIEs, we have found participants and partners creating new ways to carry on, innovating beyond what we could imagine during our assistance.

We also need to start now to design and implement for sustainability. doing SEIEs, we can start to understand the ‘drivers’ behind the Sustainable Development Goals (SDG) results with countries tracking some 120 indicators across 17 goals. Currently countries are tracking up to 230 indicators across the 17 Goals [4]. But while such monitoring shows ‘GDP has increased or ‘under-nourishment has decreased’, there is little or no information on what has caused it. Yet doing and SEIE on a large donor-funded programme, we can explore what elements made projects sustainable and how to do more (or less) there and elsewhere. Such sentinel site support for learning about sustained and emerging impacts is key to understand some of the why, for example, did income or health improve.

 

 

Dare to lead, especially in these challenging times. We know of organizations that are doing these evaluations internally, others are publishing them on their sites. Leadership happens at all levels, from internal, technical to managerial and administrative work to external evaluators and consultants as well as public pressure.

 

How can you foster sustained impact?

  • You can advocate for such evaluations
  • You can share the SEIE guidance, below, and start to design and implement, monitor and evaluate sustainably in all projects/ proposals you are designing now.
  • You can see if your organization has done any post-project sustainability evaluations and we can post them on Valuing Voices’ repository, celebrating your organization.

 

We can help you learn how to do these. Our partner, Better Evaluation, just published our Sustained and Emerging Impacts Evaluation as a ‘new’ evaluation ‘theme.’

Guidance there shows you [5]:

1. What is SEIE?
2. Why do SEIE?
3. When to do SEIE?
4. Who should be engaged in the evaluation process?
5. What definitions and methods can be used to do an SEIE?
Resources
References

SEIEs will grow as will examples, discussions, and joy as embracing sustainability sprouts, and sends us progressing in yet-unforeseen ways! We are excited to be in the final stages of receiving a research grant to further guide SEIEs. We will share that news in our next blog.

 

We want to learn from you:

  • What do you think needs to be in place for funders to move beyond the funding cycle and do an SEIE?
  • What would help to make this type of evaluation more widely undertaken?
  • If you have done a post-project evaluation, how did you do it? What were some of the barriers you faced and resources you were able to draw on to overcome them?

 

How can we lead together to Value the Voices of those we serve!?

(Reposted from https://medium.com/@WhatWeValue/leading-in-challenging-times-sustained-and-emerging-impacts-evaluation-seies-617b33bf4d27#.ec7fcg4ty)

 

 

Sources:

[1] Foulkes, I. (2017, February 27). Is there a US diplomacy vacuum at the UN in Geneva? Retrieved from https://www.bbc.com/news/world-europe-39080204

[2] Cekan, J. (2015, March 13). When Funders Move On. Retrieved from https://ssir.org/articles/entry/when_funders_move_on

[3] UN DESA. (2011, March 2). Lasting impact of sustainable development. Retrieved from https://www.un.org/en/development/desa/news/sustainable/sustainable-development.html

[4] UN Statistics Division. SDG Indicators: Global indicator framework for the Sustainable Development Goals and targets of the 2030 Agenda for Sustainable Development. Retrieved March, 2017, from https://unstats.un.org/sdgs/indicators/indicators-list/

[5] Cekan, J., Zivetz, L., & P, R. (2016). Sustained and Emerging Impacts Evaluation (SEIE). Retrieved from https://www.betterevaluation.org/en/themes/SEIE

 

Embedding Sustainability Everywhere – All Five Slices Now

 

Embedding Sustainability Everywhere – All Five Slices Now

 

It has been a tumultuous year, and next year does not look like we will have much stability as a respite. As domestic concerns grow larger in two huge economies, US and UK, the question of the place foreign aid will play abound in conversations around the world.

However 2017’s transitions transform our work, for now I do have some good news,

1) Sustainability can be cheap. Far cheaper, in fact to design for sustainability, create feedback loops checking on sustainability through the eyes of our partners and participants, monitor for sustainability than to assume it’ll happen and far cheaper than finding out funds could have had far greater impact if we had valued their voices in the first place.

In our work to help our clients and partners fund, design, implement and monitor/ evaluate with sustainability in mind, we created what we hope is a helpful tool (guidance forthcoming).

a) By Designing for Sustainability with those who will sustain them, their financial buy-in and commitment are far higher (see CRS/ Niger [1]), as is advocacy and community buy-in (see new post-project OXFAM/ DRC [2]) and there are indicators the costs of start-up later are more cost-effective.

b) By clarifying Sustainability Indicators we check assumptions about who will do so, how much of a priority our activities are before we scale them up (Federation/ Ethiopian Red Cross). Retrospective post-project sustainability evaluations also enable us to learning from past successes and do better.

c) Sustainability Monitoring and Adaptation involve those pesky but pivotal feedback loops which are vital to understanding if we have gone off the rails or not, especially in terms of unexpected shocks derailing logical frameworks of designed projects.  USAID’s nice recent CLA (Collaborating, Learning, Adapting) process includes donor funding for adaptation mid-stream which fosters effectiveness and sustainability [3]. Even lovelier are the Doing Development Differently examples that are often very low-cost and high-effect.

d) Informed Exit, Stakeholder Sustainability Consultation should be done throughout the cycle, at least a year earlier than most projects can begin this (note; not the last few months please). Transitioning for success leverages sunk costs for ongoing results. It includes heavy knowledge management on how the implementers managed information and resources, tracked data, sustained outputs and outcomes which now local partners will need to do, etc (FFP Exit Strategies study, a Czech study, and a new UK Results study shows range of items to consider during and post-transition) [4] [5].

e) Post-Project Sustainability: Our Valuing Voices/ Better Evaluation/ Tufts joint presentation at AEA on SEIE has more on how learning from post-project evaluation lessons can change sustainability of future projects for the better [6]! Further, what capacities and systems have been built in-country to sustain the results after our funding and expertise leaves? What can we do differently?

Can it be done? Demand is rising. I recently presented at a conference about embedding sustainability in programming now, and an enterprising NGO took this idea to heart. They proposed joint design with communities, which is the very bedrock of designing for sustained impact. Kudos!

 

2) Secondly, donors are willing to pay for sustainability.  Sustainability is ensconced in USAID/ Food For Peace’s (FFP) 2016 Strategy and CLA (above) [7]. In the section called New learning and Implications for FFP Programming, they present findings such as “actions that drive big results during the life of the project may actually undermine sustainability in the long run. It raises the question as to whether FFP is willing to accept more modest results in the near term if they can be delivered in a way that will yield more sustainable gains over time….”

They also point us in the direction of country-led development: “Sustained capacity, resources, motivation, and linkages all require a focus on catalysts for change beyond FFP.  Facilitative approaches that rely on and strengthen local actors help ensure that resource and knowledge transfers, and the incentives and linkages that support them, will be self-perpetuating beyond project end”.  Notably, while UK’s DFID focuses on maximizing impact through Value-for-Money, it is a shorter-term economy, efficiency and effectiveness rather than sustained impact for the end-users and DFID’s exit strategies have recently been critiqued [5].

There is an issue of disincentives for the new administration to heed (if the agricultural lobby for US food exports does not prevail): “In [USAID’s exit] study evaluating how sustainable the results of Title II development programs are 2–3 years after project closure, FFP found thatproviding free resources can threaten sustainability, unless replacement of those resources both as project inputs and as incentives has been addressed” [8]. As the Natural Resources section notes, “Whether entirely in the hands of the community or linked to a formal institution, the incentives and resources necessary to maintain a community asset are part of the system that will sustain it. The lack of such systems is visible in rusted irrigation pumps, failed mangrove plantations, abandoned bore wells, eroded dikes, and silted-in fish ponds around the world” [7]. Yet the private and public sectors are important: “Sustainable, broad-based change is more likely to be achieved by supporting and strengthening existing community, private sector, and public sector mechanisms for product and service delivery, and by supporting the capacity, quality, and accountability of government institutions” [7].

FFP’s new strategy calls for taking a systems approach to change that emphasizes sustainable long-term gains over unsustainable short-term wins. Even more delightful, in a small meeting at MFAN with Dina Esposito, Director of Food For Peace in November, she announced that they were looking to do pilot funding of an additional three years to typical five-year DFAP development projects. One year would involve collaborative participatory design between partners, communities and FFP, the second additional years being evaluation post-project of sustained and emerging impact!

This is a sea shift that can hopefully withstand political winds. After all, US foreign aid accounts for less than 1% of our federal budget, even though many Americans believe it is over 15% (hence easy to cut)…. but fingers crossed the aid effectiveness value of our work is… Valued.

 

 

Sources:

[1] Cekan, J., PhD, Kagendo, R., & Towns, A. (2016). Participation by All: The Keys to Sustainability of a CRS Food Security Project in Niger. Retrieved from https://www.crs.org/our-work-overseas/research-publications/participation-all

[2] Lindley-Jones, H. (2016, November 16). ‘If we don’t do it, who will?’ A study into the sustainability of Community Protection Structures supported by Oxfam in the Democratic Republic of Congo (DRC). Retrieved from https://policy-practice.oxfam.org.uk/publications/if-we-dont-do-it-who-will-a-study-into-the-sustainability-of-community-protecti-620149

[3] USAID Learning Lab. (n.d.). Collaborating, Learning, and Adapting (CLA)? Retrieved from https://usaidlearninglab.org/faq/collaborating%2C-learning%2C-and-adapting-cla

[4] Food and Nutrition Technical Assistance (FANTA). (n.d.). Effective Sustainability and Exit Strategies for USAID FFP Development Food Assistance Projects. Retrieved from https://www.fantaproject.org/research/exit-strategies-ffp

[5] Del Mese, F. (2016, November 16). When aid relationships change: DFID’s approach to managing exit and transition in its development partnerships. Retrieved from https://icai.independent.gov.uk/report/transition/

[6] Cekan, J., Rogers, B. L., Rogers, P., & Zivetz, L. (2016, October 26). Barking Up a Better Tree: Lessons about SEIE (Sustained and Emerging Impact Evaluation). Retrieved from https://valuingvoices.com/wp-content/uploads/2016/11/Barking-up-a-Better-Tree-AEA-Oct-26-FINAL.pdf

[7] USAID. (2016, October 6). 2016–2025 Food Assistance and Food Security Strategy. Retrieved from https://www.usaid.gov/ffpstrategy#:~:text=FFP’s%20new%20strategy%2C%20the%202016,USG)%20food%20assistance%20as%20a

[8] Rogers, B. L., & Coates, J. (2015, December). Sustaining Development: A Synthesis of Results from a Four-Country Study of Sustainability and Exit Strategies among Development Food Assistance Projects. Retrieved from https://www.fantaproject.org/research/exit-strategies-ffp

 

Presenting Lessons on (post-project) Sustained and Emerging Impact Evaluations from the U.S. AEA Conference

 

Presenting Lessons on (post-project) Sustained and Emerging Impact Evaluations from the U.S. AEA Conference

 

Dear readers, attached please find the Barking up a Better Tree: Lessons about SEIE Sustained and Emerging Impact Evaluation presentation we did last week at the American Evaluation Association (AEA) conference in Atlanta GA [1]. I had the pleasure of co-presenting with Beatrice Lorge Rogers PhD, Professor, Friedman Nutrition School, Tufts University (aka the famous Food for Peace/ Tufts Exit Strategy study [2]), Patricia Rogers PhD, Director, BetterEvaluation, Professor, Australia and New Zealand School of Government (where we recently published guidance on SEIE [3]) and Laurie Zivetz PhD, International Development Consultant and Valuing Voices evaluator.

 

We integrated our presentations from Africa, Asia and Latin America into this fascinating overview:

1.Sustained and Emerging Impact Evaluation: global context

2.SEIE: definitions and methods

3.Case studies: findings from post-project evaluations

4.Designing an SEIE: Considerations

5.Q&A — which fostered super comments, but since you couldn’t come, please tell us what you think and what questions you have…

 

There are amazing lessons to learn about design, implementation, M&E from doing post-project evaluation.  We have also grown in appreciating that sustainability can be tracked throughout the project cycle, not just during post-project SEIE evaluation.

We’ll be building this into a white paper or a … (toolkit? webinar series? training? something else?). What’s your vote ___? (I know in this US election season, so… :)).

 

What would you like to get to support your learning about Sustained and Emerging Impact Evaluations? Look forward to hearing from you- Jindra@ValuingVoices.com

Enjoy!

 

 

 

 

The full presentation is available here:

https://valuingvoices.com/wp-content/uploads/2016/11/Barking-up-a-Better-Tree-AEA-Oct-26-FINAL.pdf

 

Sources:

[1] Cekan, J., Rogers, B. L., Rogers, P., & Zivetz, L. (2016, October 26). Barking Up a Better Tree: Lessons about SEIE (Sustained and Emerging Impact Evaluation). Retrieved from https://valuingvoices.com/wp-content/uploads/2016/11/Barking-up-a-Better-Tree-AEA-Oct-26-FINAL.pdf

[2] Food and Nutrition Technical Assistance (FANTA). (n.d.). Effective Sustainability and Exit Strategies for USAID FFP Development Food Assistance Projects. Retrieved from https://www.fantaproject.org/research/exit-strategies-ffp

[3] Zivetz, L., Cekan, J., & Robbins, K. (2017, May). Building the Evidence Base for Post-Project Evaluation: Case Study Review and Evaluability Checklists. Retrieved from https://valuingvoices.com/wp-content/uploads/2013/11/The-case-for-post-project-evaluation-Valuing-Voices-Final-2017.pdf

 

Sustaining projects during and after Implementation: Does gender count?

 

Sustaining projects during and after Implementation: Does gender count?

Rutere Kagendo from Ronto Research and Valuing Voices

Gender as a concept is one of the many kinds of indicators of change measured in development projects. However, like many others, its reporting in many cases is limited to the number of women and men participating in the project. Whereas participation by gender is an important variable in measuring community engagement and inclusiveness in a project, this measure is more focused at the registration level and the mere fact that women’s or men’s names appear in the registers. However, numbers only are not a clear indicator of participation and the extent to which it happens.

Gender reporting, as shown above, can be too simplified. There is also ongoing misunderstanding surrounding gender, which only separates genders and omits a gendered analysis that would help understand the relationships between men and women, boys and girls within the dynamics of a project. This can include differentiated decisions about how resources are allocated within household expenditures (e.g. women overseeing food consumption, health and children’s expenses versus men overseeing large investments into livelihoods) and division of labor (e.g. men ploughing and planting fields and women weeding and conserving crops leading to a family harvest). Gender negotiations are richer than numbers of men and women participating in projects.

As we consider the various stages of a project, starting from design, implementation to evaluation and sustainability, it is critical to assess the role of gender in each of these stages. It is obvious that women and men play important roles in projects but what is not clear is how these roles build, complement and sustain the project. The questions that come into mind after interacting with communities during project evaluations (mid, endline, impact and sustainability) are:

    • How do the various roles played by both men and women, and their interaction with each other shape the outcome of a project?
    • Does gender contribute to project sustainability?

These are important questions that need to be answered through more research and writing but there are early insights I have learned:

 

1. Resilience in roles during the project life cycle

Studies highlighting the roles played by both women and men in projects shed light on how gender shapes project outcomes. I once attended a dairy project launch exercise in Kenya, my home country, and was surprised by what I saw. All the men sat at the front and the women took a back seat with their young children. The women remained silent the entire time the meeting was held, and efforts to involve them were unsuccessful.  I later learned that in that community, women do not share their views in the presence of men. However, I left with one question in my mind, how will this project succeed if only one party is going to be talking and making decisions? Was there no mechanism for women to share their views as well? Is it not going to become a project that benefits men only? Years later, I got an opportunity to evaluate the project. I held both men and women focused group discussions separately.   I found that firstly, men may be the gatekeepers but may not necessarily have the resilience needed to push through the whole project cycle. For example in this particular project, men did not continuously stay with the project, as noted by one female group participant:

“When this project started, we were both men and women. However, men decided to pull out because they thought milk was a women’s commodity and there was no money [to be made] in it. The women got organized in groups and started selling the milk in bulk. We made money and within no time, the men came back and took over the milk business. If we women had not stayed in the project, the milk business would not have been realized although we lost it to the men”.  In another dairy-related project elsewhere, the men were the cattle and milk owners so they sold all the milk. But during a season when the market was flooded with milk (milk glut), the men abandoned the milk business.  The women took it up and tried to sell the milk whenever possible until the milk glut’ season passed, only for the men to get back to the milk business again.

Whether it is a good idea or not for men to have taken the milk business away from women in both of these cases is subject for discussion, but the main issue here is the role played by the women to sustain the project long enough to grow it from an idea and push it into a viable business that attracted the men back into the project. Indeed one wonders if the project would have grown if the women had not remained in the project.  Valuing Voices at Cekan Consulting found the same in an all-women and highly successful sesame seed production Catholic Relief Services project in the Gambia. Women had grown the communal agribusiness from microenterprise size to being so successful that they became of interest to local banks. At that point, men in the communities began to take the project, and savings, over. Valuing Voices has documented savings and loans sustained post-project, and even scaled up by women in several countries.

Also thinking about the initial community entry meetings and seeing how unengaged and distant the women appeared, one would have dismissed their role in the project. As it turned out, women became the engine of this project.  Surface impressions may not hold true.

GenderAfricaKagendo

2. Saving the project for whose benefit?

In another project, community members were encouraged and supported to start savings and lending groups to run their small businesses. The qualitative discussions in the final evaluation revealed that at the end of the project, it was perceived as more of a women’s project in spite of being targeted to men.  This is because almost all the men had left the project after taking loans and failing to repay. The women, especially those whose spouses had taken the loans took over the burden of repaying the loans which saved the project.

Borrowing from these examples one is tempted to ask, what happens in those projects that target men only or where women get demoralized and leave? Do they fail at a higher rate altogether than those that include women? Arguing along these lines one would ask, what are the driving forces for men and women to remain or quit from project activities and how has this been addressed so far in project designs?

In most of the evaluation projects Ronto Research has conducted around Africa, men may take a lead in project implementation depending on their expected quick/ shorter-term gains/ expectations which determine whether they remain active or not. On the other hand, women seem to get into projects with a determined resolve to see their problems solved, no matter how long it takes. As a result, they hold onto the project activities despite the challenges that come along.  It is observable though, that as they progress and begin to take hold and show signs of success, the men get attracted back.

In most of the project evaluations I have participated in, there are always few men who are loyal and remain part and parcel of the project. Pivotally, we also need to understand the role of these few men who have remained in these projects with the women the whole time. This can be either because they hold positions or just because they believed in the projects and they support the women with ideas or just their mere presence in the project activities. These men seem to play a critical role in encouraging the women to push on. Their role cannot be ignored and there is need to understand them better in terms of their opinions. Why do they remain with the women even when there seems to be few benefits forthcoming to them, whereas their fellow men pull out? In an interview with women in Tanzania on an agricultural project, some women felt that the few men who had been left in their group were very instrumental in their achievements by giving the project an identity in as far as it being a community and not a ‘women-only’ project was concerned.

 

3. Gender- Informing evaluation

Thirdly, while conducting evaluation interviews, gender counts. Typically, women and men are asked about results in gender-specific groups. At the household level, the preferred interviewee is the household head. In most cases, they would be the direct ‘beneficiaries’ (participants) of the project information on behalf of their households. Due to the cultural set-ups in much of Africa, in married families, men are automatically the household head, which would mean they would be the target interviewees. However, our experience during interviews either through Focus Group Discussions or on a one-on-one situation is that consulting with women as well as considering differentiated gender roles and gender relations are key missing ingredients. For often during interviews, the men end up involving their spouses for details about projects, in part because men might not consistently follow up the project activities and therefore might not have all the details.

There is also a possibility that although it is the men who are registered as participants, in actual sense it is the women who participate and therefore have the detailed information about some issues being discussed, so it is important to check who is the actual ‘information bank’ (and day-to-day participant) of the household for the project.  This issue needs to be explored in-depth both in regular project evaluation and more needs to be learned about how this can be used to inform sustainability studies. Asking only men or only women may limit evaluative learning.

 

4. Sustaining results – which gender is best?

What can we learn from the projects that have been completed so far years after they close out?  When projects come to an end, communities are left to experience and foster the project impacts on their own. They learn a lot on their own post-project and this could be an important source of information on sustainability, but what mechanisms can be put in place for these ‘information banks’ to share the information long after projects end?  They may be the only key informants that remain (as partners may have long gone onto other projects far away); before the knowledge they hold that can be eroded with time. It would be interesting to compare the information held by men and women about outcomes and impacts being sustained, stopped or new ones emerging. The question on information especially related to project sustainability is going to be very critical considering that most of this information has to be sought from community members post-project period.

 

Does gender play a role in holding a project together/implementation?
What role does gender play in project sustainability?

 

A few resources:

UN and Gender for Sustainable Development resources:
Calder, M., & Fien, J. (n.d.). Women & Sustainable Development. Retrieved 2016, from https://web.archive.org/web/20160505120029/http://www.unesco.org/education/tlsf/mods/theme_c/mod12.html

UN on Gender Mainstreaming:
Pan American Health Organization. (2015, August). Virtual Course on Gender and Health within a Framework of Diversity and Human Rights: Module 4. Retrieved from https://drive.google.com/file/d/1f_Ufarl2S2YZ-9Dz0itZGVBEJAFyVPV_/view?usp=sharing

Gender and Evaluation:
Gender and Evaluation. (n.d.). Retrieved 2016, from https://gendereval.ning.com/?xg_source=msg_mes_network
By fellow Valuing Voices’ team member Rituu

ICRW:
International Center for Research on Women. (n.d.). Men And Gender Equality Policy Project. Retrieved 2016, from https://www.icrw.org/research-programs/men-and-gender-equality-policy-project/

Finally, an ActionAid:
ActionAid USA. (2014, November 20). Public Finance for Agriculture Project. Retrieved from https://www.youtube.com/watch?v=osscXjT17sI

 

PARTICIPATION BY ALL: The Key To Sustainability of CRS/ Niger’s Food Security Project

 

PARTICIPATION BY ALL:
The Key To Sustainability of CRS/ Niger’s Food Security Project

 

Valuing Voices is delighted to share our sustainability evaluation of Catholic Relief Service Niger’s PROSAN project [1]. This project that ran from 2006-2012 in Niger and was implemented by three NGOs: CRS, Cooperative for Assistance and Relief Everywhere (CARE), and Helen Keller International (HKI) under the direction of United States Agency for International Development (USAID) Office of Food for Peace (FFP) as a multi-year assistance program (MYAP) to support food security activities in the Dosso, Tahoua, and Zinder regions. PROSAN focused on increasing agricultural production and agro-enterprise, improving household health and nutrition status, reinforcing the capacities of health agents, and enhancing community resiliency.

Here are the highlights from the report which itself is an excerpt from a longer analysis we did. Also please note one Annex highlights the similarlties/ differences we found to USAID/ FFP’s 4 elements of sustainability:

 

AIM, METHODS, AND RESEARCH QUESTIONS

The aim of this sustainability evaluation was to explore perceptions of sustainability from Nigeriens involved in PROSAN, former CRS staff and donors. It focused on evaluating participants’ adherence to project outcomes and their creation of new innovations. It also evaluated partners’ involvement in sustaining project outcomes.

This evaluation used qualitative and quantitative methods including community mapping, focus group discussions, beneficiary interviews, and key stakeholder interviews. The evaluation was carried out in six communities in the Dosso region, with more than 500 interviewees, focusing on the following research questions:

  1. Sustainability of activities and groups: Are the communities sustaining the activities three to five years after the end of the project? What can we learn from the communities and their post-project implementation partners?
  2. Spread and unexpected outcomes: If the project was considered a success in the eyes of the community, how well did it spread?
  3. Fostering Sustainability: What are the long-term prospects for continued sustainability?

 

FINDINGS

Three years after PROSAN’s conclusion, the project was considered a success by community members, national partners, the implementer (CRS), and donor (USAID) staff. The main findings include:

1. SUSTAINABILITY OF ACTIVITIES AND GROUPS

Eighty percent (80%)[*] of all activities were reported to have become self-sustained and community innovations have emerged:

  • On average, households reported moving from being food secure for 3-6 months per year during PROSAN to 8-12 months at the time of this evaluation, which is a remarkable impact [1].
CRS_Niger_PROSAN_Sustainability_Evaluation_pdf

[1]

  • Women reported greater income through the increase in sales of food that was produced and processed due to the grain mills [1].
  • Respondents also reported improved household health, hygiene, and nutrition, with 91% of survey respondents indicating that their health and sense of well being had improved, especially through the efforts of the health posts and clinics that CRS helped build and the government of Niger’s efforts in sustaining them with resources and staff [1].

Community groups/committees have continued and are well-supported by NGO partners:

  • 81% of the committees set up by PROSAN were functioning at the time of this evaluation, with many participants discussing ways to sustain best practices within their communities, and members still receiving regular trainings or updates [1].
  • Several new and refresher trainings come through national partners, NGOs, and new channels such as radio programs [1].
  • Some new NGOs and international organizations have built upon PROSAN’s success, for instance, by using land previously managed by PROSAN for a new vegetable gardening training program, building hygiene programs on past health awareness efforts, or extending agricultural credit for further inputs [1].

Twenty percent (20%) of implemented activities were not sustained or have stagnated:

  • While hygiene practices were sustained by households and there was widespread latrine construction, sanitation was poor in the villages, and most latrines had fallen into disrepair [1].
  • Fewer than 50% of women reported practicing exclusive breastfeeding for children less than six months of age [1].
  • While almost half of all health committees no longer exist, new health clinics staff have replaced some of the work of the committees with health and agricultural promotion messages now being sent via radio, television, and cell phones [1].
  • Literacy training and theater groups have completely ceased [1].
  • With the exception of the Système Communautaire d’Alerte Précoce-Réponses aux Urgences’ (SCAP-RU) SCAP-RU early warning system which has expanded, other resilience activities such as roadwork and caring for the environment are a lesser priority due in part to the lack of food and cash-incentives to continue doing them [1].

 

2. SPREAD AND UNEXPECTED OUTCOMES

New innovations and ceased activities reflected the project’s legacy:

  • Community innovations have emerged such as collective funds paying for cleaners of the new health center, community-imposed sanctions for births occurring outside of the health centers, and the monitoring of savings from well water sales.
  • National partners have praised the project, with many lamenting its withdrawal. One non-PROSAN village told an Agriculture Ministry staff and potential NGO partner that “No one should bring a program here unless it is like PROSAN.”
  • PROSAN-trained masons, well repair technicians, and village youth have learned land recuperation techniques (zai holes, bunds and demi-lunes) that helped generate income beyond project communities.
  • Project activities that received free inputs have largely stopped being implemented once the incentives were withdrawn such as Food for Training (FFT), Food for Work (FFW), or Cash for Work (CFW) (e.g. literacy, seedlings, latrines, theater etc.); nonetheless the inputs were highly valued and have continued to support agriculture and health (carts, bicycles).

 

3. FOSTERING SUSTAINABILITY

The following areas were identified as potential barriers to sustainability that could be systematically explored in other projects:

  • Although most committees are still functioning, there are no processes in place to engage and train youth and new inhabitants of the villages.
  • While village communities have been maintained, there is an increasing lack of ministry resources (e.g., staff, transportation, and communications) to take the place of NGOs like CRS after a program ends.
  • There is little management of knowledge around project data, which is further exacerbated by staff changes in NGOs, government ministries, and donors. Project data (proposal content, monitoring data, evaluation results, participant lists, partner names, and exit agreements) must be managed ethically, locally and be held online, accessible for future projects to use and for villages to conduct self-evaluations.

 

The full report is available here:

https://www.crs.org/sites/default/files/tools-research/participation-by-all.pdf

 

Sources:

[1] Cekan, J., PhD, Kagendo, R., & Towns, A. (2016). Participation by All: The Keys to Sustainability of a CRS Food Security Project in Niger. Retrieved from https://www.crs.org/our-work-overseas/research-publications/participation-all

 


[*] Percentages were calculated as a combination of the number of activities that had been continued and the percentage of participants which continued them.