by Rutere Kagendo | Sep 7, 2016 | Gender, Implementation, international development, Kenya, Project cycle, Sustainability, Sustainable development, Women and gender
Sustaining projects during and after Implementation: Does gender count?
Rutere Kagendo from Ronto Research and Valuing Voices
Gender as a concept is one of the many kinds of indicators of change measured in development projects. However, like many others, its reporting in many cases is limited to the number of women and men participating in the project. Whereas participation by gender is an important variable in measuring community engagement and inclusiveness in a project, this measure is more focused at the registration level and the mere fact that women’s or men’s names appear in the registers. However, numbers only are not a clear indicator of participation and the extent to which it happens.
Gender reporting, as shown above, can be too simplified. There is also ongoing misunderstanding surrounding gender, which only separates genders and omits a gendered analysis that would help understand the relationships between men and women, boys and girls within the dynamics of a project. This can include differentiated decisions about how resources are allocated within household expenditures (e.g. women overseeing food consumption, health and children’s expenses versus men overseeing large investments into livelihoods) and division of labor (e.g. men ploughing and planting fields and women weeding and conserving crops leading to a family harvest). Gender negotiations are richer than numbers of men and women participating in projects.
As we consider the various stages of a project, starting from design, implementation to evaluation and sustainability, it is critical to assess the role of gender in each of these stages. It is obvious that women and men play important roles in projects but what is not clear is how these roles build, complement and sustain the project. The questions that come into mind after interacting with communities during project evaluations (mid, endline, impact and sustainability) are:
-
- How do the various roles played by both men and women, and their interaction with each other shape the outcome of a project?
- Does gender contribute to project sustainability?
These are important questions that need to be answered through more research and writing but there are early insights I have learned:
1. Resilience in roles during the project life cycle
Studies highlighting the roles played by both women and men in projects shed light on how gender shapes project outcomes. I once attended a dairy project launch exercise in Kenya, my home country, and was surprised by what I saw. All the men sat at the front and the women took a back seat with their young children. The women remained silent the entire time the meeting was held, and efforts to involve them were unsuccessful. I later learned that in that community, women do not share their views in the presence of men. However, I left with one question in my mind, how will this project succeed if only one party is going to be talking and making decisions? Was there no mechanism for women to share their views as well? Is it not going to become a project that benefits men only? Years later, I got an opportunity to evaluate the project. I held both men and women focused group discussions separately. I found that firstly, men may be the gatekeepers but may not necessarily have the resilience needed to push through the whole project cycle. For example in this particular project, men did not continuously stay with the project, as noted by one female group participant:
“When this project started, we were both men and women. However, men decided to pull out because they thought milk was a women’s commodity and there was no money [to be made] in it. The women got organized in groups and started selling the milk in bulk. We made money and within no time, the men came back and took over the milk business. If we women had not stayed in the project, the milk business would not have been realized although we lost it to the men”. In another dairy-related project elsewhere, the men were the cattle and milk owners so they sold all the milk. But during a season when the market was flooded with milk (milk glut), the men abandoned the milk business. The women took it up and tried to sell the milk whenever possible until the milk glut’ season passed, only for the men to get back to the milk business again.
Whether it is a good idea or not for men to have taken the milk business away from women in both of these cases is subject for discussion, but the main issue here is the role played by the women to sustain the project long enough to grow it from an idea and push it into a viable business that attracted the men back into the project. Indeed one wonders if the project would have grown if the women had not remained in the project. Valuing Voices at Cekan Consulting found the same in an all-women and highly successful sesame seed production Catholic Relief Services project in the Gambia. Women had grown the communal agribusiness from microenterprise size to being so successful that they became of interest to local banks. At that point, men in the communities began to take the project, and savings, over. Valuing Voices has documented savings and loans sustained post-project, and even scaled up by women in several countries.
Also thinking about the initial community entry meetings and seeing how unengaged and distant the women appeared, one would have dismissed their role in the project. As it turned out, women became the engine of this project. Surface impressions may not hold true.
2. Saving the project for whose benefit?
In another project, community members were encouraged and supported to start savings and lending groups to run their small businesses. The qualitative discussions in the final evaluation revealed that at the end of the project, it was perceived as more of a women’s project in spite of being targeted to men. This is because almost all the men had left the project after taking loans and failing to repay. The women, especially those whose spouses had taken the loans took over the burden of repaying the loans which saved the project.
Borrowing from these examples one is tempted to ask, what happens in those projects that target men only or where women get demoralized and leave? Do they fail at a higher rate altogether than those that include women? Arguing along these lines one would ask, what are the driving forces for men and women to remain or quit from project activities and how has this been addressed so far in project designs?
In most of the evaluation projects Ronto Research has conducted around Africa, men may take a lead in project implementation depending on their expected quick/ shorter-term gains/ expectations which determine whether they remain active or not. On the other hand, women seem to get into projects with a determined resolve to see their problems solved, no matter how long it takes. As a result, they hold onto the project activities despite the challenges that come along. It is observable though, that as they progress and begin to take hold and show signs of success, the men get attracted back.
In most of the project evaluations I have participated in, there are always few men who are loyal and remain part and parcel of the project. Pivotally, we also need to understand the role of these few men who have remained in these projects with the women the whole time. This can be either because they hold positions or just because they believed in the projects and they support the women with ideas or just their mere presence in the project activities. These men seem to play a critical role in encouraging the women to push on. Their role cannot be ignored and there is need to understand them better in terms of their opinions. Why do they remain with the women even when there seems to be few benefits forthcoming to them, whereas their fellow men pull out? In an interview with women in Tanzania on an agricultural project, some women felt that the few men who had been left in their group were very instrumental in their achievements by giving the project an identity in as far as it being a community and not a ‘women-only’ project was concerned.
3. Gender- Informing evaluation
Thirdly, while conducting evaluation interviews, gender counts. Typically, women and men are asked about results in gender-specific groups. At the household level, the preferred interviewee is the household head. In most cases, they would be the direct ‘beneficiaries’ (participants) of the project information on behalf of their households. Due to the cultural set-ups in much of Africa, in married families, men are automatically the household head, which would mean they would be the target interviewees. However, our experience during interviews either through Focus Group Discussions or on a one-on-one situation is that consulting with women as well as considering differentiated gender roles and gender relations are key missing ingredients. For often during interviews, the men end up involving their spouses for details about projects, in part because men might not consistently follow up the project activities and therefore might not have all the details.
There is also a possibility that although it is the men who are registered as participants, in actual sense it is the women who participate and therefore have the detailed information about some issues being discussed, so it is important to check who is the actual ‘information bank’ (and day-to-day participant) of the household for the project. This issue needs to be explored in-depth both in regular project evaluation and more needs to be learned about how this can be used to inform sustainability studies. Asking only men or only women may limit evaluative learning.
4. Sustaining results – which gender is best?
What can we learn from the projects that have been completed so far years after they close out? When projects come to an end, communities are left to experience and foster the project impacts on their own. They learn a lot on their own post-project and this could be an important source of information on sustainability, but what mechanisms can be put in place for these ‘information banks’ to share the information long after projects end? They may be the only key informants that remain (as partners may have long gone onto other projects far away); before the knowledge they hold that can be eroded with time. It would be interesting to compare the information held by men and women about outcomes and impacts being sustained, stopped or new ones emerging. The question on information especially related to project sustainability is going to be very critical considering that most of this information has to be sought from community members post-project period.
Does gender play a role in holding a project together/implementation?
What role does gender play in project sustainability?
A few resources:
UN and Gender for Sustainable Development resources:
Calder, M., & Fien, J. (n.d.). Women & Sustainable Development. Retrieved 2016, from https://web.archive.org/web/20160505120029/http://www.unesco.org/education/tlsf/mods/theme_c/mod12.html
UN on Gender Mainstreaming:
Pan American Health Organization. (2015, August). Virtual Course on Gender and Health within a Framework of Diversity and Human Rights: Module 4. Retrieved from https://drive.google.com/file/d/1f_Ufarl2S2YZ-9Dz0itZGVBEJAFyVPV_/view?usp=sharing
Gender and Evaluation:
Gender and Evaluation. (n.d.). Retrieved 2016, from https://gendereval.ning.com/?xg_source=msg_mes_network
By fellow Valuing Voices’ team member Rituu
ICRW:
International Center for Research on Women. (n.d.). Men And Gender Equality Policy Project. Retrieved 2016, from https://www.icrw.org/research-programs/men-and-gender-equality-policy-project/
Finally, an ActionAid:
ActionAid USA. (2014, November 20). Public Finance for Agriculture Project. Retrieved from https://www.youtube.com/watch?v=osscXjT17sI
by valuingvoicesjin | May 10, 2016 | Sustainable development
This blog post by Jindra Cekan and Laurie Zivetz of Valuing Voices discusses the need for post-project impact evaluations. An area that needs more attention, BetterEvaluation will be working with Valuing Voices over the next couple of months to expand the available information and resources for this type of evaluation.
The question of what is impact is one that has been coming up again and again recently. Simon Hearn’s recent blog on BetterEvaluation (and his accompanying Methods Lab paper) began unraveling the many ways we use the term. Simon notes that in addition to the range of definitions of impact, our industry has a growing appetite to demonstrate the impact of our international development projects with evidence. In a blog post last year, Developing a Research Agenda for Impact Evaluation, Patricia Rogers and Greet Peersman note that current impact evaluations tend to look “only at relatively short-term, intended direct effects” and attention to unintended or unexpected (emerging) impacts of our projects remains undeveloped. While organisations such as 3ie and J-Pal have risen to prominence for their work on impact evaluation, the different views surrounding what constitutes a sustained impact in these organisations are important (read the Valuing Voices blog post on the differences between our definition and theirs, and the implications of this).
The preoccupation with short-term impacts represents a serious gap in evaluation practice, theory and design. Returning 2-10 years post-project offers an opportunity to assess whether improvements- such as those in organisational efficiencies, community infrastructure, knowledge, behaviour change, livelihoods- that may have been planned and shown at the end of the project cycle, actually endured. It also provides a chance to understand whether other unintended impacts emerged over time as a result of a project or program or participants’ efforts in the intervening years after the project ended. How can we claim we are doing “sustainable development” if we do not return to assess sustainability that we either envisioned or that emerged from people’s own efforts?
This gap presents serious challenges to our ability to be genuinely accountable or to learn and improve – also fundamental to effectively meeting the Sustainable Development Goals. We believe that funding, doing and learning from Sustained and Emerging Impact Evaluations (SEIE) needs to become a standard part of the program cycle. This is key to doing sustainable development. Building an evidence base and generating lessons across projects and sectors, and from design to implementation, monitoring to evaluation promise benefits in terms of better design and implementation of projects, more robust, accountable partnerships, and a path to foster true country ownership.
The numbers are staggering. The OECD tells us that over $5 trillion in foreign aid has been given since 1970 (excluding private donations). Over $1.5 trillion of that has been given by the EU and USAID since 2000 alone and even last year $137 billion was spent worldwide. At Valuing Voices, we have reviewed hundreds of documents tagged as ‘post-project’ and ‘ex-post’ evaluation from nearly 30 international development and implementer organisations. Most of these documents only suggest that such evaluations be done; a few are desk studies. Only a handful (some 1% of all projects actually evaluated) have actually returned to the field post-project, and systematically asked participants and partners what they had sustained themselves 2-10 years after the project ended. Even fewer of these have delved into why some outcomes were sustained, and asked questions about what other impacts or innovations emerged. It seems that much of the ‘sustainability’ focus is on the continuation of funding for projects, rather than assessing impact, and including the efforts and hopes of our stakeholders. Curiosity about what might be done better seems to stop at the end of the resourced project cycle.
Valuing Voices Full-Cycle Development
The 19 organisations that we have found that have done one or more such evaluations deserve to be strongly celebrated as catalytic learning organisations. They are at the vanguard of redefining ‘success’ through our participants’ and partners‘ eyes and improving our programming accordingly. We have piloted this approach in a number of post-project sustained impact evaluations that found important lessons for design with follow-on funding as well as for monitoring and implementation. We have documented the tip of the sustained impact iceberg, including an excellent study by USAID that highlighted four essential elements for successful exit (resources, capacity, linkages and motivation). We discuss some of the assumptions many of us make about barriers to doing more such evaluations, Including needed improvements to our M&E systems, how to forge stronger partnerships and country ownership.
We ask you to join Valuing Voices and BetterEvaluation in advocating creating new success measure in international development: sustained and emerging impact evaluations. We ask you to fund and do these SEIEs and share them for inclusion in our database. Such learning should not only be standard in our industry, but the lessons from them to be a globally available resource. We envision creating regional centres of analysis of learning and country-ownership, with data available long after any project ends. We ask you to help us put participant feedback at the centre of all of our work so we can better learn about sustained and emerging impact, foster design for success, implement for national ownership of projects long after we are gone. That is why we do what we do, isn’t it?
Have you been involved in a post-project evaluation of sustained and emerging impacts? Or do you know of any good examples or have some advice on conducting one? Let us know in the comments below or send us a message! We’ll be developing a new BetterEvaluation option for this over the coming months so your comments and examples will help contribute to this.
Author
President, Valuing Voices at Cekan Consulting.
Washington, United States of America.
Author
Independent Consultant, Valuing Voices at Cekan Consulting.
Washington, United States of America.
by Jindra Cekan | Apr 13, 2016 | Catholic Relief Services (CRS), Exit strategies, Food for Peace (FFP), international development, LIvelihoods, Niger, Participation, Sustainability, Sustainable development, USAID
PARTICIPATION BY ALL:
The Key To Sustainability of CRS/ Niger’s Food Security Project
Valuing Voices is delighted to share our sustainability evaluation of Catholic Relief Service Niger’s PROSAN project [1]. This project that ran from 2006-2012 in Niger and was implemented by three NGOs: CRS, Cooperative for Assistance and Relief Everywhere (CARE), and Helen Keller International (HKI) under the direction of United States Agency for International Development (USAID) Office of Food for Peace (FFP) as a multi-year assistance program (MYAP) to support food security activities in the Dosso, Tahoua, and Zinder regions. PROSAN focused on increasing agricultural production and agro-enterprise, improving household health and nutrition status, reinforcing the capacities of health agents, and enhancing community resiliency.
Here are the highlights from the report which itself is an excerpt from a longer analysis we did. Also please note one Annex highlights the similarlties/ differences we found to USAID/ FFP’s 4 elements of sustainability:
AIM, METHODS, AND RESEARCH QUESTIONS
The aim of this sustainability evaluation was to explore perceptions of sustainability from Nigeriens involved in PROSAN, former CRS staff and donors. It focused on evaluating participants’ adherence to project outcomes and their creation of new innovations. It also evaluated partners’ involvement in sustaining project outcomes.
This evaluation used qualitative and quantitative methods including community mapping, focus group discussions, beneficiary interviews, and key stakeholder interviews. The evaluation was carried out in six communities in the Dosso region, with more than 500 interviewees, focusing on the following research questions:
- Sustainability of activities and groups: Are the communities sustaining the activities three to five years after the end of the project? What can we learn from the communities and their post-project implementation partners?
- Spread and unexpected outcomes: If the project was considered a success in the eyes of the community, how well did it spread?
- Fostering Sustainability: What are the long-term prospects for continued sustainability?
FINDINGS
Three years after PROSAN’s conclusion, the project was considered a success by community members, national partners, the implementer (CRS), and donor (USAID) staff. The main findings include:
1. SUSTAINABILITY OF ACTIVITIES AND GROUPS
Eighty percent (80%)[*] of all activities were reported to have become self-sustained and community innovations have emerged:
- On average, households reported moving from being food secure for 3-6 months per year during PROSAN to 8-12 months at the time of this evaluation, which is a remarkable impact [1].
[1]
- Women reported greater income through the increase in sales of food that was produced and processed due to the grain mills [1].
- Respondents also reported improved household health, hygiene, and nutrition, with 91% of survey respondents indicating that their health and sense of well being had improved, especially through the efforts of the health posts and clinics that CRS helped build and the government of Niger’s efforts in sustaining them with resources and staff [1].
Community groups/committees have continued and are well-supported by NGO partners:
- 81% of the committees set up by PROSAN were functioning at the time of this evaluation, with many participants discussing ways to sustain best practices within their communities, and members still receiving regular trainings or updates [1].
- Several new and refresher trainings come through national partners, NGOs, and new channels such as radio programs [1].
- Some new NGOs and international organizations have built upon PROSAN’s success, for instance, by using land previously managed by PROSAN for a new vegetable gardening training program, building hygiene programs on past health awareness efforts, or extending agricultural credit for further inputs [1].
Twenty percent (20%) of implemented activities were not sustained or have stagnated:
- While hygiene practices were sustained by households and there was widespread latrine construction, sanitation was poor in the villages, and most latrines had fallen into disrepair [1].
- Fewer than 50% of women reported practicing exclusive breastfeeding for children less than six months of age [1].
- While almost half of all health committees no longer exist, new health clinics staff have replaced some of the work of the committees with health and agricultural promotion messages now being sent via radio, television, and cell phones [1].
- Literacy training and theater groups have completely ceased [1].
- With the exception of the Système Communautaire d’Alerte Précoce-Réponses aux Urgences’ (SCAP-RU) SCAP-RU early warning system which has expanded, other resilience activities such as roadwork and caring for the environment are a lesser priority due in part to the lack of food and cash-incentives to continue doing them [1].
2. SPREAD OF EMERGING AND UNEXPECTED OUTCOMES
New emerging innovations and ceased activities reflected the project’s legacy:
- Community innovations have emerged such as collective funds paying for cleaners of the new health center, community-imposed sanctions for births occurring outside of the health centers, and the monitoring of savings from well water sales.
- National partners have praised the project, with many lamenting its withdrawal. One non-PROSAN village told an Agriculture Ministry staff and potential NGO partner that “No one should bring a program here unless it is like PROSAN.”
- PROSAN-trained masons, well repair technicians, and village youth have learned land recuperation techniques (zai holes, bunds and demi-lunes) that helped generate income beyond project communities.
- Project activities that received free inputs have largely stopped being implemented once the incentives were withdrawn such as Food for Training (FFT), Food for Work (FFW), or Cash for Work (CFW) (e.g. literacy, seedlings, latrines, theater etc.); nonetheless the inputs were highly valued and have continued to support agriculture and health (carts, bicycles).
3. FOSTERING SUSTAINABILITY
The following areas were identified as potential barriers to sustainability that could be systematically explored in other projects:
- Although most committees are still functioning, there are no processes in place to engage and train youth and new inhabitants of the villages.
- While village communities have been maintained, there is an increasing lack of ministry resources (e.g., staff, transportation, and communications) to take the place of NGOs like CRS after a program ends.
- There is little management of knowledge around project data, which is further exacerbated by staff changes in NGOs, government ministries, and donors. Project data (proposal content, monitoring data, evaluation results, participant lists, partner names, and exit agreements) must be managed ethically, locally and be held online, accessible for future projects to use and for villages to conduct self-evaluations.
The full report is available here:
https://www.crs.org/sites/default/files/tools-research/participation-by-all.pdf
Sources:
[1] Cekan, J., PhD, Kagendo, R., & Towns, A. (2016). Participation by All: The Keys to Sustainability of a CRS Food Security Project in Niger. Retrieved from https://www.crs.org/our-work-overseas/research-publications/participation-all
[*] Percentages were calculated as a combination of the number of activities that had been continued and the percentage of participants which continued them.
by Jindra Cekan | Feb 29, 2016 | 3ie, Accountability, Appreciative Inquiry, Assumptions, CARE, Fears, Food for Peace (FFP), Funding, Learning, microcredit, OECD, OXFAM, PACT, Sustainable development, USAID, Village Banks
What happens after the project ends?
Lessons about Funding, Assumptions and Fears (Part 3)
In part 1 and part 2 of this blog, we showcased 11 of the 18 organizations that have done post-project evaluations. While this was scratching the surface of all that is to be learned, we shared a few insights on How we do it Matters, Expect Unexpected Results and Country-national Ownership. We gained some champions in this process of sharing our findings, including Professor Zenda Ofir of South Africa, who said “we cannot claim to have had success in development interventions if the outcomes and/or impacts are not durable, or at least have a chance to sustain or endure.”
In this third blog of Lessons Learned from What Happens After the Project Ends, we turn to some of the curious factors that hold us back from undertaking more post project evaluations: Funding, Assumptions, and Fears.
Funding
- Why haven’t we gone back? For the last 2+years Valuing Voices has been researching the issue, we have heard from colleagues: ‘we would love to evaluate post-project but we don’t have any money, ‘donors don’t fund this’, ‘it is too expensive’[*]. Funding currently from bilateral donors such as USAID is given in 1-5 year tranches with fixed terms for completion of results and learning from them and one-year close-out processes [1]. Much of the canon of evaluations conducted after close out that we amassed was from international NGOs that had used their private funds to evaluate large donor-funded projects for their own learning. Many aimed also to show leadership in sustainability and admittedly dazzle their funders – join them!.
- We fund capacity building during projects but if we do not return to evaluate how well we have supported our partners and communities to translate this to sustainability, then we fall short. Meetings convened by INTRAC on civil society sustainability are opening new doors for joint learning about factors such as “legitimacy… leadership, purpose, values, and structures” within organizations well beyond any project’s end [2]. The OECD’s DAC criteria for evaluating development assistance define sustainability as: “concerned with measuring whether the benefits of an activity are likely to continue after donor funding has been withdrawn. Projects need to be environmentally as well as financially sustainable“ [3]. We need to extend our view beyond typical criterion for sustainability being a focus primarily on continued funding.
- We need funding to explore whether certain sectors lend themselves to sustainability. In addition to the cases in blog 1, a study by CARE/ OXFAM/ PACT on Cambodian Savings groups finds that we have some revisions to make on how we design and implement with communities to foster sustainability in this sector which typically promises greater sustainability because capital can be recycled [4]. Valuing Voices blogs show indications that once we amass a greater range of post-project evaluations (funders unite), the insights gleaned can illuminate cost efficient paths to more sustained programming, possibly leading to revisions in programming or interventions which have greater likelihood for country-ownership
- Extend the program cycle to include post-project sustainability evaluation. Rare are donors such as the Australian government (forthcoming) and USAID’s Food For Peace that commission such studies. Rare is the initiative such as 3ie that has research funds allocated by major donors to explore an aspect of impact. We miss out on key opportunities to learn from the past for improved project design if we do not return to learn how sustained our outcomes and impacts have been. We miss learning how we could better implement so more national partners could take on the task of sustaining the changes we catalyzed.
- We call on donors to fund a research initiative to comprehensively review sustainability evaluations.
- We call on governments to ask for this in their discussion with donors.
- We call on implementers to invest in such learning to improve the quality of implementation today and sustained impact in the years to come.
Assumptions
Development assistance makes many assumptions about what happens after projects end in terms of people’s self-sufficiency, partners’ capacity to continue to support activities, and projects’ financial independence and people’s ability to step into the shoes of donors and carry on. Unless we take a hard look at our assumptions, we will not move from proving what we expect to learning what is actually there.
Among them are these six assumptions:
- All will be well once we exit; we have implemented so well that of course national participants and partners will be ready and able to carry on without us. We may assume the only important outcomes and impacts are within our Logical Frameworks and Theories of Change. Thus there is no need to return to explore unexpected negative ones, or ways in which the people we strengthened may have innovated in unexpectedly wonderful ways. Aysel Vazirova, a fellow international consultant wrote me: “Post-project evaluations provide data for a deeper analysis of sustainability and help to appreciate numerous avenues taken by the beneficiaries in incorporating development projects into their lives. The theory of change narratives presented by a majority of development programs and projects have a rather disturbing resemblance to the structure of magic tales: (from) Lack – (to) change – (to) happy ending. Post project evaluations have a power to change a rigid structure of this narrative.”
- We assume evaluations are often used to inform new designs, yet dozens of colleagues have lamented that too often this does not happen in the race to new project design. But there is hope. World Wildlife Fund/UK M&E expert Clare Crawford says when following its new management standards, WWF “expects to see the recommendations of an evaluation before the next phase of design can happen (hence evaluations happen a little before the end of a strategic period). WWF-UK, when reading new program plans is mandated to verify if – and how – the recommendations of the last evaluation(s) were made use of in the new design phase. Equally we track management responses to evaluations to see how learning has been applied in current or in future work.” Such a link across the program cycle is not common in our experience and none of the post-project sustained impact evaluations we reviewed said how learning would be used.
- We may assume data continues accessible from the projects we have evaluated, yet our team member Siobhan Green has found that until recently, with the move toward open data, often project data remains the province of the donors and implementers and to the best of our knowledge leaves the country when projects close. While some sectoral data such as health and education data remains local, we are finding in fieldwork that household level data has been rolled up or discarded once projects close, which makes interviews difficult.
- We may assume that the participants and partners are not able to evaluate projects, particularly after the fact. Being vulnerable does not mean that people are not able to share insights or assess how projects helped or not. Methods such as empowerment evaluation and evaluative thinking are powerful supports [5] [6].
- Some may assume that the situation has changed in the intervening years, that there is no benefit in returning to see what results remain. Change is inevitable and sometimes more rapid or dramatic than others. But does that mean we shouldn’t want to understand what happened? This is the greatest disservice of all, for we are selling “sustainable development” so how well have we designed it to be so?
- We assume that learning for our own benefit is enough. A potential client brought me in to discuss my working on a rare post-project evaluation last year. It was to cost hundreds of thousands of dollars and while would occur in several countries. What I discovered was that while the donor really wanted to learn what results remained more than a decade on, I asked ‘how would the countries themselves benefit from this research and findings?’ There was a long silence. Turns out, nothing from the research would benefit or even remain in country. No one had considered the learning needs of the countries themselves. This simply cannot continue if we are to be accountable to those we serve.
Fears
This may be the greatest barrier of all to returning to assess sustainability.
- We assume our projects continue. We may be afraid to look for what will this tell us about the sustainability of our efforts to save lives and livelihoods so we only choose to publicly study what is successful. Valuing Voices has found that across most the post project studies there is some ‘selection bias’, as we repeatedly learned in our research from colleagues that organizations choose to evaluate projects that are most likely to be successfully sustained. For instance, USAID Food For Peace’ study notes, “The countries included in this study—Bolivia, Honduras, India, and Kenya—were also chosen because of their attention to sustainability and exit.” Yet as an Appreciative Inquiry practitioner, I would argue that learning what worked best to know what to do more of may be the best way forward.
- All too often the choice of evaluation design, and sensitivity to findings fly in the face of learning—particularly when findings are negative. This raises fears around a discontinuation of funding (an implementer fear; a beneficiary fear; could also be a recipient government’s fear). Yet as Bill Gates says, “your most unhappy customers are your greatest source of learning.”>
- Participants asked during the project cycle about interventions may be fearful of truth telling because of perceived vulnerabilities around promised future resources, local power imbalances in control over resources, or even political imperatives to adopt a particular position. Alternatively we may not believe them, thinking they may not tell us the truth were that to stop resources.
Those are ours.
- Peter Kimeu, my wise advisor and 20-year friend and colleague from Kenya tells us some fears of the that we need to listen to – those that haunt our national partners and participants.
They are afraid we do not see their real desires:
- “It is ‘not how many have you (the NGO) fed, but how many of us have the capability to feed ourselves and our community?’
- ‘How can we (country national) support our fellow citizens to take our lives and livelihoods into our own hands and excel, sustainably?’
- ‘What is sustainability if it isn’t expanded opportunities, Isn’t the capability of one to make a choice of value/quality life out of the many choices that the opportunities present?”
Will you help us address these challenges? Will you join us in advocating filling the gap in the program cycle, and looking beyond it to how we design and implement with country nationals? Will you, in your own work foster their ownership throughout and beyond? We need to fund learning from sustained impact to transparently discuss assumptions and face our fears. This is a sustained purpose we need to and can fill.
Sources:
[1] Capable Partners Program & FHI 36. (2010). Essential NGO Guide to Managing Your USAID Award: Chapter 6 – Close Out. Retrieved from https://www.ngoconnect.net/sites/default/files/resources/Essential%20NGO%20Guide%20-%20Chapter%206%20-%20Close%20Out.pdf
[2] Hayman, R. (2014, November 5). Civil society sustainability: Stepping up to the challenge. Retrieved from https://www.intrac.org/civil-society-sustainability-stepping-challenge/
[3] OECD. (n.d.). DAC Criteria for Evaluating Development Assistance. Retrieved 2015, from https://web.archive.org/web/20151206171605/http://www.oecd.org/dac/evaluation/daccriteriaforevaluatingdevelopmentassistance.htm
[4] Emerging Markets Consulting. (2013). Sustainability of Savings Group Programs in Cambodia for CARE, Oxfam, and Pact. Retrieved from https://mangotree.org/Resource/Sustainability-of-Savings-Group-Programs-in-Cambodia-for-CARE-Oxfam-and-Pact
[5] Better Evaluation. (n.d.). Empowerment Evaluation. Retrieved from https://www.betterevaluation.org/plan/approach/empowerment_evaluation
[6] Griñó, L., Levine, C., Porter, S., & Roberts, G. (Eds.). (2016). Embracing Evaluative Thinking for Better Outcomes: Four NGO Case Studies. Retrieved from https://www.theclearinitiative.org/resources/embracing-evaluative-thinking-for-better-outcomes-four-ngo-case-studies
[*] It does not have to be. We have done these evaluations for under $170,000, all-inclusive.
by Jindra Cekan | Feb 19, 2016 | Accountability, Aid effectiveness, Asian Development Bank (ADB), CARE, ex-post evaluation, Exit strategies, Food for Peace (FFP), InterAction, Microenterprise, OECD, PACT, Participation, Project Concern International (PCI), Sustainability, Sustainable development, Sustained and Emerging Impacts Evaluation, USAID
What happens after the project ends? Country-national ownership lessons from post-project sustained impacts evaluations (Part 2)
In Part 1 of our blog on lessons learned from post-project evaluations, we explored:
- How we do it matters for great results
- Expect unexpected results
This time we turn to who continues after closeout, and what conditions foster both successful handover and ownership from the onset in order to foster sustained impact.
Who Takes Over? Country nationals
When project handover is integral to the design, development projects needn’t be long-term or expensive. What they need to be is increasingly community-driven. Unless exit strategies are explicit and thorough, sustained impacts are less likely.
1. USAID/ Food for Peace (FANTA/ Tufts)
An ‘exit strategies’ evaluation of 12 projects in four USAID Food for Peace (FFP) countries of Bolivia, Honduras, India, and Kenya carried out in 2009, three to four years after close out detailed mixed results, described here [1]. These were complex food security projects across multiple sectors of: maternal and child health and nutrition; water and sanitation; agriculture, livestock, and rural income generation; natural resource management; school feeding; and micro-savings and loans.
FANTA/Tufts found “providing free resources, such as supplementary food as an incentive for growth monitoring participation or free agricultural marketing services to promote sales, created expectations that could not be sustained once the free resources were no longer offered.” Valuing Voices found similar issues in Niger’s PROSAN (see part 1), with the lack of continued incentives (food and in-kind inputs) led to activities not being continued by community members.
On the other hand, in India, the government took over FFP food ration distribution after closeout. “This phase-over of responsibility to national government programs was effective in the case of supplementary feeding but not in the case of school-feeding (the latter through the midday meals program), due to varying levels of government commitment. India’s government had the resources, capacity (an already existing supply chain), and motivation (commitment) to provide this benefit.” Again, CRS/Niger showed us that decades-long investments in partnerships and 2+ years for phase-over pays off; 80% of outcomes were self-sustained three years on. CIDA Peru also found that “Shared responsibilities and participatory process were instrumental in ensuring sustainability….a shared understanding of project objectives and counterpart interventions was established with Peruvian sector authorities, between donors and local communities” [2].
The lesson learned about close coordination with the partners such as the national government during design and implementation in order for transition to country-ownership and responsibility to be smooth also appeared among multilaterals that Valuing Voices has examined. Three multilateral agencies stand out as having conducted multiple post project evaluation (OECD, JICA and the Asian Development Bank).
We posit that too often in international development, the accountability focus is on fulfilling funder (donor) requirements, rather than accountability to project participants and what is needed to achieve sustained impact for them (Figure 1, below). The optimal case has project funders, implementers and national governments aligning to support those we ostensibly serve: women, men, youth, elders in need of assistance.
@ValuingVoices2015
Are there certain kinds of projects or implementers that manifest optimal accountability? Far more examination is needed, but a promising path is microenterprise.
2. Pact’s WORTH project in Nepal
PACT’s project illuminates that local ownership and structures sustain results and even multiply impact. Implemented from 1999-2001, Pact worked with many local NGOs to reach 125,000 women in 6,000 economic groups across Nepal; of those, one quarter chose to implement village banks. Village Banks cultivated women as agents of change and development in their communities—promoting grassroots sustainability The post project evaluation in 2006 found that:
- Almost two thirds of the original 1,536 village banks were still active eight years after the program began and assets of an average village bank has tripled in the last three years post-project (from $1000 to over $3000 at the time of the evaluation) [3]
- 83% reported that because of WORTH they are able to send more of their children to school [3]
- Women’s economic groups helped start an estimated 425 new groups involving another 11,000 women with neither external assistance nor prompting from the project [3].
Why? The post-project report tells that the banks were not an end in and of themselves, women’s empowerment was: “WORTH groups and banks were explicitly envisaged as more than just microfinance providers; they were seen as organizations that would build up women as agents of change and development in their communities” [3]. Thus local Nepalese sustained and grew their own development.
3. CARE Zanzibar’s Village Savings and Loan Associations were evaluated four years post-project. Similar to PACT, they found the model was sustained and grew:
- Total membership rose from 1,272 in 2002’s closeout to an estimated membership of 4,552 in July 2006, an increase of 258% [4]
- During the most recent payout for all 25 groups, the mean rate of return was 53%, with individual groups’ rates ranging from 10% to 92% [4]
- Participants said that the main changes in the lives as a result of the program were an improved standard of living (22%), improved housing (21%) and increased incomes (20%) [4].
While wonderful, can it only be the responsibility of communities to sustain their gains? How well are we designing for country-ownership and handover to the state?
4. The UN’s OECD has dozens of post project evaluations on its website, funded by member governments.
One study illuminates that while communities may manage to sustain some of the outcomes, structural investment in national capacity to takeover is key. This example evaluated four 10-15 year-long projects funded largely by Germany that were carried out in Indonesia, Sri Lanka, Tanzania and Zambia with a cumulative value of Euro 145.1 million ($180 million). The study was done in 2004, evaluating activities an astonishing 30 years after inception. Results?
- The good news: “living conditions of the target groups have improved in all four project regions,” with specific sustainable project outcomes observed in the “health and education sector, food security, increase in income and employment and the ensuing rise in the standard of living”. Links were made to project-supported improvements in infrastructure, enhanced private sector economy, and the project’s innovations in agriculture.
- The bad news: “there was low institutional sustainability at the level of state executing organizations for all four projects due to inadequate funds, inefficient organizational structures and a lack of coordination”. Thus, viable exit and handover was limited. Structures advanced as part of a development project ran a high risk of not being sustainable.”
5. Three years ago, the Asian Development Bank reviewed 491 project completion reports (desk studies) and undertook a handful of field visits to projects financed between 2001-09. Similar results:
- “Some early evidence suggests that as many as 40% of all new activities are not sustained beyond the first few years after disbursement of external funding” [5]
- “National government ownership, commitment to and financing of the projects were vital to sustainability“ [5]
- “Neither governments nor other international agencies benefit from systematic information on whether projects reached their intended economic or social objectives over the full life of the intervention or in the decade afterwards” [5].
This is a clarion call to all funders to invest in future excellence by returning to the past, learning what worked best and what failed to do so, examine why and begin anew with accountability to our participants!
Conclusion
What can be done?
A) Foster ownership of the process of development through empowerment to begin with, as PACT’s WORTH project did in Nepal and elsewhere. InterAction’s lovely “A Missing Piece in Local Ownership: Evaluation” reminds us “the local ownership agenda must extend to all parts of the program cycle – from design all the way through evaluation. Including those meant to benefit from international assistance (we use the term “participants”) in deciding what should be done and how it should be done is critically important for effectiveness and sustainability” [6].
Ask yourselves how well we involve governments in collaborative design of what they feel they can sustain of our programming after we leave, how and for how long with what resources, linkages, capacity-built and motivation (see FFP study #1).
B) Design and implement in the present while considering sustaining outcomes and impacts in the long-term, as we learned in Part 1 of this blog as well as taking lessons from some emerging guides such as the systematic guidance of PCI’s Resource Guide for Enhancing Potential for Sustainable Impact [7].
C) Dare to return to learn. As Dina Esposito, the Director of USAID/Food For Peace stated, “this rigorous, retrospective [ex-post] approach is not widely done, but is essential if we are to understand the true impacts of our investments. To be effective, development projects must result in changes that last beyond the duration of the project themselves” [8].
Imagine the sustained cost-efficiencies of learning certain sectoral programs lent itself best to sustainability by communities, others needed Ministries to take over, others still needed different support such as private sector – or all of the above. If we look at sustained impact as our true goal, how differently could we work together? How much more efficiently would we use our global resources?
What can we say about the sustained impact post-project evaluations we have featured?
We have covered lessons about how matters in design and implementation; expect unexpected results and who takes over? Country nationals. Much more research and analysis is needed, many more case studies need to be created for us to understand how to foster the best handover as well as national ownership at the beginning, middle and end. Maybe you drew some of the same conclusions we have:
- Post project evaluations provide valuable insights about sustainability.
- Lessons from such evaluations can lead to better programming in current and future projects.
- The voice of national stakeholders—participants and partners, including governments is essential.
- Donors lose amazing opportunities to learn what works now and continues to work unless they fund more sustainable impact evaluations and support investing resources in fostering sustainability during design and implementation.
In Part 3, we will look at what is keeping us from looking to the past for the future (hint: funding, assumptions and fears) and how we can move ahead together…
Please join us in advocating for and funding this vital approach!
Sources:
[1] Food and Nutrition Technical Assistance (FANTA). (n.d.). Effective Sustainability and Exit Strategies for USAID FFP Development Food Assistance Projects. Retrieved from https://www.fantaproject.org/research/exit-strategies-ffp
[2] Canadian International Development Agency (CIDA). (2012). Evaluation of CIDA’s Peru Program. Retrieved 2014, from https://web.archive.org/web/20140807174641/http://www.acdi-cida.gc.ca/INET/IMAGES.NSF/vLUImages/Evaluations2/$file/peru-eng.pdf
[3] Mayoux, L. (2008, June). Women Ending Poverty: The WORTH Program in Nepal – Empowerment through Literacy, Banking and Business 1999-2007. Retrieved from https://www.findevgateway.org/case-study/2008/06/women-ending-poverty-worth-program-nepal-empowerment-through-literacy-banking
[4] Anyango, E., Esipisu, E., Opoku, L., Johnson, S., Malkamaki, M., & Musoke, C. (2006, January). Village Savings and Loan Associations: Experiences from Zanzibar. Retrieved from https://www.findevgateway.org/case-study/2006/01/village-savings-and-loan-associations-experiences-zanzibar
[5] Asian Development Bank. (2010, October 31). Post-Completion Sustainability of Asian Development Bank-Assisted Projects. Retrieved from https://www.adb.org/documents/post-completion-sustainability-asian-development-bank-assisted-projects
[6] Grino, L. (2015, February 19). A Missing Piece in Local Ownership: Evaluation. Retrieved 2015, from https://web.archive.org/web/20150502162547/https://www.interaction.org/blog/missing-piece-local-ownership-evaluation
[7] Choi-Fitzpatrick, J., Schooley, J., Eder, C., & Lomeli, B. (2014). A Resource Guide for Enhancing Potential for Sustainable Impact: Food and Nutrition Security. Retrieved from https://www.fsnnetwork.org/resource-guide-enhancing-potential-sustainable-impact
[8] Rogers, B. L., & Coates, J. (2015, December). Sustaining Development: A Synthesis of Results from a Four-Country Study of Sustainability and Exit Strategies among Development Food Assistance Projects. Retrieved from https://www.fsnnetwork.org/ffp-sustainability-and-exit-strategies-study-synthesis-report
by Jindra Cekan | Feb 13, 2016 | Aid effectiveness, Bill and Melinda Gates Foundation, Catholic Relief Services (CRS), Central Asia, Ethiopia, Ethiopian Red Cross, Evaluation, Federation of the Red Cross, Lutheran World Relief, Mercy Corps, Niger, PACT, Partners for Democratic Change, post-project evaluation, Sustainability, Sustainable development, Sustained and Emerging Impacts Evaluation, USAID
What happens after the project ends?
Lessons from post-project sustained impacts evaluations (Part 1)
We talk a lot about impact of our interventions, but far less is analyzed about the sustained impact of our work in the years after projects close out. We take for granted that successful strategies will continue after projects shut down. Do they always? Maybe they do but we don’t know. Maybe there are innovations and impacts to be learned from…
To answer these question Valuing Voices spent 2 ½ years looking for and analyzing ‘post-project’ evaluations of projects undertaken 2-10 years after projects ended. The result: in our $137 billon international development industry, some 99% of projects remain unevaluated after project close out [1]. Only 17 agencies we have found so far have publically available post-project evaluations; most of them have one, while the OECD and JICA have dozens. Hundreds of studies recommend such learning that is missing from our industry’s program cycle (green slice).
Six decades on, this astonishing finding raises serious questions about stewardship of resources and commitment to learning—particularly learning from participant and partner stakeholders for whom sustainability matters most, and who are tasked with it over the long-term.
A review of post project evaluations generate food for thought about good program design and illustrate the value added of post project perspectives. This rapid review of select ‘ex-post’ evaluations points to three early lessons:
How we do it matters for great results
Expect unexpected results
Who takes over? Country Nationals
First, organizations go back to see how well their projects results were sustained. What we learned was that how well they used participatory processes in how they implemented and handed over mattered a lot for sustainability and that we must expect unexpected results.
How we do it matters for great results
1. Catholic Relief Services/ USAID PROSAN food security project in Niger
Valuing Voices evaluated this food security (agriculture, health and resilience project in 2015 which ran from 2007-2012 (report forthcoming). The $32 million project was implemented in a consortium of CRS, CARE and Helen Keller but this evaluation focused only on CRS areas.
- Interviews with over 500 participants found that three years after project closeout, 80% of project activities still continued, as did many village committees and there were a variety of community innovations [2]
- On average, households can feed themselves through their own production or purchase of food for 8-12 months three years after closeout compared to 6-9 months at closeout three years earlier. Such impact was unexpected. [2]
- 91% of respondents reported improved household health, hygiene, and nutrition [2]
- A 2.5-year participatory exit process from CRS to country stakeholders (local government, an array of local and international NGOs and the private sector ensured continuity and boosted local ownership [2]
- 20% of the activities did not continue, mostly food-assisted NRM and resilience-related [2]
- Youth make up 50% of the population and need to be engaged during the project for long term sustainability to occur. [2]
2. PartnersGlobal (formerly Partners for Democratic Change)
Their mission is to “to build sustainable capacity to advance civil society and a culture of change and conflict management worldwide” uses an approach that is “bottom-up, locally-led rather than foreign-led, based on the belief that change comes from sustainable efforts led by local people, organizations and institutions invested in their own long-term future.” They went back and to review 55 case studies of projects through 22 centers they founded in central and eastern Europe from 1989-2011. They found:
- In 80% of cases, there was advancement of good governance by influencing the participation of civil society working with government [3]
- In 50% of the cases there was increased access to justice and managing and resolving disputes/conflicts, thereby strengthening civil society [3]
- 18 of 22 of the centers that had been established still exist today (82%) [3].
3. Mercy Corps
They did a post-project evaluation in Central Asia in 2007, one and three years after two conflict resolution projects ended which were worth $18 million. These complex community mobilization programs with aims “to empower communities to work together in a participatory manner to address the infrastructure and social needs [while] developing sustainable skills [and] empowers communities to identify and utilize existing resources within the communities and not to depend only on external assistance.”
- 72% of youth report that they continue to use at least one skill they learned during the programs (e.g. teamwork and communication, and skills such as sewing, construction, roofing, journalism and cooking) [4]
- 68% of community members witnessed local government becoming more involved in community activities after the end of the programs as compared with before the programs [4]
- 57% of the communities studied continuing to use one or more of the decision-making practices promoted during the program [4]
- 42% of members, representing 70% of communities, reported that the community had worked collectively on new projects or repairs to existing infrastructure. Participants and partners had implemented almost 100 infrastructure projects by themselves independent of donor funds. [4]
These are terrific expected results.
We also learned to Expect unexpected results
4. Federation of the Red Cross and Ethiopian Red Cross
Valuing Voices combined a final evaluation of “Building Resilient Community: Integrated Food Security Project to Build the Capacity of Dedba, Dergajen & Shibta Vulnerable People to Food Insecurity” that had funding over $3 million in 2009 from the Swedish Red Cross with an assessment of projected sustainability. It was an IFRC/ERCS collaboration with the Ethiopian government to provide credit for food security inputs to 2,259 households, which were to be repaid in cash over time as well water and agriculture/ seedlings for environmental resilience. We answered the DAC criteria for evaluation and found the project overall to be quite good, albeit with weak data tracking systems.
In terms of sustainability, we used participatory methods to learn about what people felt they could self-sustain once the project left their area, so we could shape a similar follow-on project design to be moved elsewhere in Tigray, particularly around the credit for animals.
- While 87% of the loans had been promoted by the government and given for large animals (oxen and cows), and 13% was for small animals (sheep, goats, chickens)…
- But project participants we interviewed, strongly preferred the small animals in terms of being able to sustain them on their own. They felt they could afford these smaller amounts of credit as well as the feed to sustain them, without taking the risk of animal death leaving them with large debt. This was especially true for women, who preferred poultry to all other animals 15:1.
In our quest for fast results, are we asking participants to bear too much risk? As one of our Valuing Voices team asks, Who is responsible for sustainability?
5. Lutheran World Relief
From 2005-2007 Lutheran World Relief intervened in Niger, the world’s poorest country, with a $500,000 Pastoralist Survival and Recovery Programme (ARVIP) drought rehabilitation project funded by the Bill and Melinda Gates Foundation. There were numerous outcomes from targeting sheep, wells and animal fodder to 600 of the poorest women in 10 communities in northern Niger, among them:
- Women’s share of household income increased from 5% to 25% in some households. This was due to the value of the sheep grants, as well as time-savings used for income generation. Access to wells in five of the villages saved women a staggering 7-10 hours every other day from not having to go fetch water 3.5 hours away each way for household and animal needs and were free to weave mats or cook food for sale [5]
- Many said they didn’t have to resort to worse survival strategies during the next hungry season after they received the sheep [5]
What was not expected were these results:
- Many women in several villages reported an impact that was completely unexpected to the implementer and donor which was “our husbands don’t beat us anymore” [5]. This was thanks to both increased respect and income from the sheep as well as access to well-water which led to cleanliness and their ability to be home for their husbands, children and mothers-in-law, rather than fetching water whole days 2-3 times a week. The same was found in PACT’s WORTH empowerment and village banking project in Nepal that wrote, “one in 10 reported that WORTH has actually helped “change her life” because of its impact on domestic violence” [6].
- We defined success too narrowly. Many interviewees were content with the project even though prospects for project-expected drought resilience or sustained food security were less likely. Some women sold the sheep to buy food, pay their children’s school fees or their daughters’ dowries, while some had their sheep sold by their husbands who used them to buy other animals, pay for ceremonies or other expenses. Participants saw the project as bringing them resources and considered it a success. Spending assets on immediate needs is not at all illogical for a community who can feed itself only 4 months a year; for some households, their pressing needs far outweighed the luxury to wait and buffer seasonal food insecurity far down the line.
We hope you agree that allocating funds and attention to post-project sustained impacts evaluations is necessary for the remaining 99% of international development projects as it offers a fantastic learning opportunity about how to ‘do development’ well now and for the future. Without returning to look for what participants and partners valued enough to continue on their own, without returning to learn about unexpected sustained impacts, we rob ourselves of pivotal learning needed for success.
In part two, we look at ownership onward, planning for handover and lessons from who takes over? Country Nationals. In part three, we focus on Funding, Assumptions and Fears.
Please join us in advocating for this! Please think about your own projects… and whether you have considered these things, or need our help. We’re listening!
Sources:
[1] OECD. (2015, December 22). Detailed final 2014 aid figures released by OECD/DAC. Retrieved from http://www.oecd.org/dac/stats/final2014oda.htm
[2] Cekan, J., PhD, Kagendo, R., Towns, A. (2016). Participation by All: The Keys to Sustainability of a CRS Food Security Project in Niger. Retrieved from https://www.crs.org/our-work-overseas/research-publications/participation-all
[3] Carstarphen, N., PhD. (2013, November). Sustainable Investment in Local Capacity for Democracy and Peace: A Global Evaluation of Partners for Democratic Change. Retrieved from https://www.partnersglobal.org/resource/sustainable-investment-in-local-capacity-for-democracy-and-peace/
[4] Westerman, B., & Sheard, S. (2007, December). Sustainability Field Study – Understanding What Promotes Lasting Change at the Community Level. Retrieved from https://reliefweb.int/report/world/sustainability-field-study-understanding-what-promotes-lasting-change-community-level
[5] Cekan, J. (2013). Increasing women’s incomes, increasing peace: Unexpected lessons from Niger. Participatory Learning and Action, (66), 75-82. Retrieved from https://pubs.iied.org/G03661/
[6] Mayoux, L. & Valley Research Group (2008, June). Women Ending Poverty: The WORTH Program in Nepal – Empowerment through Literacy, Banking and Business 1999-2007. Retrieved from https://www.findevgateway.org/case-study/2008/06/women-ending-poverty-worth-program-nepal-empowerment-through-literacy-banking
