by Jindra Cekan | Mar 4, 2015 | Accountability, Aid effectiveness, Czech, Evidence-based policy, Mongolia, Sustainability, Sustainable development, Transparency
What remained of Mongolian kindergartens
In 2012, we have ventured with my colleague Marie Koerner through steppes of Mongolia to learn what remained of the Czech-funded mobile kindergartens. People remembered a great project, nomadic women teaching in mobile kindergartens, children learning playfully in a safe environment and integrating better to primary schools… and their parents having more time for work or smaller siblings. Well, not much remained of it 4 months after extended project end. Despite an official agreement, the Mongolian Ministry of Education did not provide a budget. It rather accepted a bigger donation from the Asian Development Bank for a similar project. Well, the new one did not benefit that many children in so many remote areas, but kindergartens had better qualified teachers, free meals and heating. And the donor was happy. Still, something remained of the Czech project – the positive attitude of the community towards pre-primary education and dedication to involve children at least in summer prep-schools organized by the government. Read the full evaluation report here in the Czech language.
The sustainability stories are diverse, yet factors are often similar. The devil is in the detail – for example, a dedicated village chief can make a big difference.
Sustainability or commercial continuity?
Based on different evaluations of Czech development projects conducted in 2010 to 2013, the Czech Ministry of Foreign Affairs started debating sustainability in 2013. Often, it was linked to commercial continuity – companies, ideally Czech, would continue to have business based on their previous successful engagement. Or at least income generation within the projects was highlighted. Questions arose if at least certain sectors, CSO or companies or types of projects are more sustainable than others, based on income generation or other factors.
What makes development projects sustainable?
Following the policy debate, the Czech platform of Civil Society Organisations – FoRS engaged Marie Koerner and me to analyse all evaluation reports, define what sustainability actually is and how it can be achieved. The key findings are below.
-
Project sustainability is linked to continued benefits for intended beneficiariesafter donor funding has been withdrawn. Not all project activities need to continue. Not all benefits need to be seen as meaningful by the beneficiaries. Income generation is not always possible and does not necessarily contribute to poverty reduction. Especially if the income is generated by the Czech companies, not by the local actors, who are responsible for sustaining project benefits.
-
Sustainability of the Czech Development Cooperation projects varies. Review of 37 publicly available evaluation reports of Czech development projects shows no correlation between sustainability and specific sectors, contracting authorities, implementers or countries of implementation has not been detected. It is rather external and internal factors shown below which influence project sustainability.
-
Sustainability factors are project specific and interdependent. They are external and internal. Often, even external factors can be influenced to achieve higher sustainability.
-
Sustainability can be influenced throughout the project cycle. Each stakeholder plays a certain role at each phase of the project cycle.
-
Good practice is available across sectors, including environment, agriculture, social development including education, economic development including energy as well as global development education and awareness raising. These sustainable Czech projects considered the key factors influencing sustainability in their plans and activities and thus contributed to lasting positive benefits for the recipients.
Overview of factors is displayed below:
Sustainability Factors
Read the publication below, including different good practices. You can download ithere.
What next? What is your experience?
Based on the study, the Czech Development Agency together with FoRS organised an internal expert workshop on 13 March 2014 to debate possible steps for increasing the sustainability of Czech bilateral projects.
The results are yet to be seen. Feel free to share your resources and experience!
by Jindra Cekan | Feb 19, 2015 | Accountability, Evaluation, ex-post evaluation, foreign aid, International aid, International non-profits, Local Participants, Microenterprise, Nepal, Participants, Participation, post-project evaluation, Results, SEEP, self-sustainability, Sustainability, Sustainable development, Village Banks
Making money– microenterprise– is this a way to sustainable livelihoods? PACT’s Nepalese Lessons
Many Americans are steeped in the belief that we must ‘pull ourselves up by our bootstraps’, that hard work and especially faith in small businesses is the way to success. This is one of the many reasons why microfinance so appeals to donors as an investment. Does it work?
The US NGO-umbrella, Interaction, posted some “Aid Works” global results, including “the percentage of USAID-funded microfinance institutions that achieved financial sustainability jumped from 38% in 2000 to 76% in 2012.” Yet there have been numerous detractors of the model and the unsustainability of control over resources/ empowerment [1] [2].
What does one ex-post evaluations that we have on hand tell us? PACT’s USAID-funded WORTH program in Nepal was focused on women ending poverty through business, banking and literacy/ bookkeeping [3]. The project, implemented between 1999 and 2001 worked with 240 local NGOs to reach 125,000 women in 6,000 economic groups across Nepal’s southern Terai (in 2001 a Maoist insurgency led to the groups being on their own) [3]. By then, 1,500 of these groups led by the women themselves (35,000-strong) received training to become informal-sector Village Banks [3]. Working with local NGOs enabled them to reach 100,000 women in a few months due to the NGOs’ presence and connections in the communities. The collaboration worked well due to a shared belief by PACT and the NGOs that dependency is not empowering. As the report says “WORTH groups and banks were explicitly envisaged as more than just microfinance providers; they were seen as organizations that would build up women as agents of change and development in their communities” [3].
In 2006, PACT and Nepalese Valley Research Group looked to see sustainability of the banks, the extent of retained income by the women as well as any effect on community development and broader issues such as domestic abuse [3]. They went to 272 Banks from a random sample of 450 from seven of the 21 WORTH districts. Remarkably, they found even more functioning: 288 (16 more) of them were thriving and – wow- WORTH women had spawned another 400 more groups on their own [3]. Participant interviews were done with members and management as well as those women who had left their Banks and members of groups that had dissolved plus they interviewed a ‘control group’ of poor, non-WORTH women in Village Bank communities.
Was it a universal success? Almost. See the bar chart below showing what impacts the management committee felt the village banks had had on members, which is mostly better off, some the same, some far better off. This held true for the original village bank members and the new bank members.

[3]
The SEEP network reviewed WORTH’s ex-post and found five key findings:
- Wealth creation: A Village Bank today holds average total assets of over Rs. 211,000, or $3,100, more than three times its holdings in 2001. Each woman member of WORTH now has an average equity stake of $116 in her Village Bank [3].
- Sustainability: Approximately two-thirds (64 percent) of the original 1,536 Village Banks are still active eight and a half years after the program began and five to six years after all WORTH-related support ended. That means there are nearly 1,000 surviving groups with approximately 25,000 members [3].
- Replication: A quarter of the existing WORTH groups has helped start an estimated 425 new groups involving another 11,000 women with neither external assistance nor prompting from WORTH itself. If all these groups are currently operating, then more Village Bankers are conducting business today in Nepal than when formal WORTH programming ended in 2001. The report also said 63% of the Village Bank members derived the income from agriculture/ sale of food versus 17% in commerce/ retail trade and the rest in miscellaneous trades. Over 40% of the participants said they borrowed to pay for education and health costs and another 20% to pay off other loans plus for festivals (e.g. birth, death) [3].
- Literacy: 97 percent of respondents reported that literacy is “very important” to their lives; 83 percent reported that because of WORTH they are able to send more of their children to school [3].
- Domestic disputes and violence: Two-thirds of groups reported that members bring their personal or family problems to the group for advice or help. Of these, three-quarters reported helping members deal with issues of domestic disputes and related problems. Forty-three percent of women said that their degree of freedom from domestic violence has changed because of their membership in a WORTH group. One in 10 reported that WORTH has actually helped “change her life” because of its impact on domestic violence [3].
The report outlines other impacts, including self-help actions such as two-thirds of groups being engaged in community action, and three-quarters said that the group has done something to help others in the community. Speaking of community, it is notable that the self-selected women were primarily from wealthier groups (60%), 15% from the middle class, with only 20% from the most disadvantaged castes [3]. Frankly this is not as surprising, as those most willing to take on risk are rarely the poorest until later; 67% of the very poor later wanted to join such a bank (once the risk was shown not to be too high versus income) [3].
The study’s author asks “Yet for all this documented success, WORTH and other savings-led microfinance programs remain among the best kept secret in the world of international development and poverty alleviation. Although together such programs reach some two million poor people, they go almost unnoticed by the $20 billion credit-led microfinance industry… The empowered women in this study—like WORTH women elsewhere in Asia and Africa— have proved themselves equipped to lead a new generation of entrepreneurs who can take WORTH [onward] through a model of social franchising now being pilot-tested [which is] as creative and potentially groundbreaking as is WORTH…WORTH has the potential to become an “international movement that supports women’s efforts to lift themselves, their families, and their communities out of poverty” [3].
So why aren’t are we learning from such projects and scaling them up everywhere? PACT is [4]. They have reached 365,000 women in 14 countries – including Myanmar, Cambodia, Colombia, Swaziland, DRC, Ethiopia, with Nigeria and Malawi starting this year [4]. Coca-Cola awarded $400,000 to PACT in 2013 to replicate WORTH in Vietnam with 2,400 women [5]. Who else is replicating this model? It’s not clear from many excellent microenterprise sites I visited except one tells me that Mastercard Foundation and Aga Khan are looking into wider replication as well. Let’s track their results and ask participants!
Sources:
[1] Bateman, M. (2011, September 20). Microcredit doesn’t work – it’s now official. Retrieved from https://opinion.bdnews24.com/2011/09/20/microcredit-doesn%E2%80%99t-work-%E2%80%93-it%E2%80%99s-now-official/
[2] Vaessan, J., Rivas, A., & Duvendack, M. (2014, November). The Effects of Microcredit on Women’s Control Over Household Spending in Developing Countries: A Systematic Review and Meta-analysis. Retrieved from https://www.findevgateway.org/paper/2014/11/effects-microcredit-womens-control-over-household-spending-developing-countries
[3] Mayoux, L. (2008, June). Women Ending Poverty: The WORTH Program in Nepal – Empowerment through Literacy, Banking and Business 1999-2007. Retrieved from https://www.findevgateway.org/case-study/2008/06/women-ending-poverty-worth-program-nepal-empowerment-through-literacy-banking
[4] PACT. (n.d.). WORTH. Retrieved 2015, from https://web.archive.org/web/20141106013639/http://www.pactworld.org/worth
[5] PACT. (2013, August 13). The Coca-Cola Foundation awards $400,000 grant to Pact. Retrieved from https://www.pactworld.org/article/coca-cola-foundation-awards-400000-grant-pact
by Jindra Cekan | Feb 13, 2015 | Accountability, Aid effectiveness, Bilateral organizations, Czech, Donor Driven Development, Donors, Evidence-based policy, ex-post evaluation, International aid, NGO, post-project evaluation, Republic of Georgia, self-sustainability, Sustainable development
Czech it out! Great evaluation happening in the Czech Republic
One of the delights of living in another country is the surprises one encounters. For me, coming back to our second 'home', it was an evaluative surprise. For by connecting to the Czech Foreign Ministry's Evaluation team, I found evidence of learning from meta-evaluation, doing ex-post evaluation, conscientious tracking of project cost-effectiveness and an openness to self-sustainability research funding and using national evaluators to lead them.
Czech Foreign aid is widespread- "Through development cooperation, the Czech Republic helps to eradicate poverty in less developed parts of the world by means of sustainable socio-economic development. It also contributes to global security and stability, conflict prevention, the promotion of democracy, human rights and fundamental freedoms, and the rule of law". Development assistance is done by several entities, the main two under the Ministry of Foreign Affairs, ORS (Development Cooperation and Humanitarian Aid) and its subsidiary CRA (Czech Development Agency).
The Ministry of Foreign Affairs oversees some fascinating evaluation work. After attending several partner-donor meetings and presentations of a meta-evaluation, an ex-post from an array of projects in Georgia and a discussion of findings across all evaluations in 2014, I am impressed. Why? Because not only are they willing to learn from both successes and failures, openly discussing challenges in learning between grants and contracts, but also because they are tackling programming in 10 countries (e.g. Afghanistan, Bosnia and Herzegovina, Ethiopia, Georgia) with a mere 17 people and a budget of $35 million for 2015.
What are some of the things they are learning?
1. They commissioned a meta-evaluation looking at 20 projects from 2012-2013. What worked well was well described and documented evaluations that were also cost-effective (evaluations were 4% of total costs) and tried to offer constructive solutions to things that did not work well in projects. While some methodologies needed to be better, and reports were hard to access, a major finding was what needs to be improved is inclusion of local recipients in stakeholder analyses, soliciting their views on what the evaluations should focus and on how the projects affected them. Further, during discussions we highlighted the need for an evaluation of outcomes and impacts, not just how evaluations quality was but also which organizations had the best results and why.
2. They commissioned an ex-post evaluation across eight organizations' in the Republic of Georgia (5 Czech, 3 Georgian), of one-year projects with 131 separate activities in civic engagement, media and youth between 2008-2012. The evaluation looked at the short-term effectiveness and longer-term sustainability of activities in the Republic of Georgia. Key findings included good relevance of aid offered, high cost-efficiency, low effectiveness for Georgian decision-making, primarily individual (rather than systemic) sustainability, though some good impact.
Key recommendations from this evaluation-, which Valuing Voices thinks, are universal included:
LEARN BETTER TOGETHER
a. Implement min. 3-year projects, whereby focus in a selected region (or a few regions) on a selected local priority topic, ensure in-depth needs analysis, multi-stakeholder cooperation [including participants], sustainable mechanisms, ongoing local support and enough flexibility as per external factors.
b. Allocate budget for burning human rights issues and for enhancing planning, monitoring, evaluation and learning capacities of Civil Society Organizations.
and
SHARE FINDINGS MORE
c. Coordinate activities with other implementers and donors in the target area and if possible (taking into account the political situation) also with local state institutions and potential implementers
d. Implement multi-stakeholder initiatives in a specific area (health, environment, social inclusion, minorities) with an advocacy component, sharing of results / lessons learnt and a media component
3. Among annual recommendations from Evaluation discussions throughout the year emerged this surprising one on cost-effectiveness. A detailed financial budget is now standard, and expenses for project activities among a majority of (grant-funded) projects and the Czech Development Agency are now required. This enables cost-effectiveness comparisons at least across grants (albeing not across for-profit contracts). In my experience this is unparalleled! (Let me know if other countries do this please!)
Overall, the fact that the Czech Foreign Ministry and implementing partners are willing to look at themselves critically and transparently improve accountability to its ultimate recipients and taxpayers makes me shout Hurrah from all of Prague’s 100 spires! Here is one of them, taken from a Ministry’s window.
by Jindra Cekan | Jan 3, 2015 | Aid effectiveness, Bilateral organizations, DIFD, Evidence-based policy, Food security, Malawi, Mali, Nicaragua, ODI, Participants, self-sustainability, Sustainable development, USAID
Walking in our participants' shoes, Doing Development Differently
You and I like to make informed decisions. We go to restaurants recommended by Yelp or Facebook friends. We refer to Consumer Reports' rankings before we buy appliances and read Amazon reviews before we purchase items, or at least ask random friends what works for them. I just bought a fridge and looked at energy star ratings to buy one that was efficient and climate-friendly. Would you buy an expensive appliance without market research reassuring you of the likely success?
Yet that is what we ask millions of development participants to do every day. In international development, community participants don't have such luxuries as knowing which projects worked well before, or that success in terms of health, income, education is replicable using this model. They invest their time and resources, blindly, hoping for a good return.
Doing Development Differently is a very encouraging initiative by UK's Overseas Development Institute (ODI) which focuses on learning from bottom-up, country-led development. Leni Wild tells us via a Malawi Country Scorecard example that NGO-led innovation during implementation is key as is creating coalitions of shared responsibility to meet participant needs. Natalia Adler shares a Nicaraguan Human-Centered Design where policy makers literally walked the lives of participants, learning from being participant-observers and supporters, not omniscient experts.
For our projects and policies are but a piece of participants' lives, but an important input if done for appropriate impact and sustainability. But do we know if we all mean the same thing? Years ago I asked what 'food security' meant in several Malian communities. In addition to expected answers- 'being able to feed myself and my family'- I got answers such as 'many children' (both the cause of having food and that more children generate sustained food security once adult, employed), and 'children's schooling' (having surplus to send their children to school year-around).
Not only do we need to understand what impact looks like to our participants, and start putting in systems for them to track it and report back to us, but also what does sustainability means to our participants? Also what can they self-sustain? What can we do more skillfully in future project design, learning from what worked best?
* Imagine what project impacts we could learn about if we returned to see, 2-5 years after projects closed out, to see what remarkable unintended impacts were created as in Niger.
* Imagine how self-sustainable project activities could be if we designed future projects based on past project successes (trackin which activities communities around the world were most able to replicate after projects left)?
* Imagine being the non-profit able to claim that the majority of activities were self-sustainable by communities 10 years later, and receiving the best Star ratings by them?!
* Imagine being the Minister of Development in Africa, Asia or Latin America being able to vett incoming projects based on likelihood of achieving Development Star results?
Using national evaluators, building national systems of online IATI-compliant documentation of what their own people consider to be the most sustainable impacts of projects, as well as are missing pieces of the 'development' puzzle. Listening could teach us a lot more than our logical framework (logframe) expectations of impact from our (donor’s) view — especially what communities imagine projects will lead to. Even farther out, imagine if together we calculate projects' economic benefits/ the Return on Investment to participants (crudely put, the 'bang for the buck") of our projects in their view, as well as our efficacy in terms of resource use, that scary thought…
Doing Development Differently is as exciting as are initiatives like USAID Forward in supporting development toward countries, e.g. starting to channel funds directly to local implementers under which falls USAID’s CLA (Collaborating, Learning and Adapting). Another great example is UK’s DIFD’s BRIDGE which incorporates Strategic Learning and adaptation into projects (adapting the during implementation, rather than the quite fixed straight-jacket most projects are tied to on signing agreements or contracts).
For we need to put ourselves out of a job by training local NGOs to supplant us, to support capacity/ create systems in Ministries to take over all but our funding, and especially, to listen to those who know best- communities whose lives we are to improve. Where are you seeing success brewing?
by Jindra Cekan | Dec 16, 2014 | Evaluation, India, Industrial Revolution, International aid, Shrujan, South Africa, Sustainable development, Umzi Wethu, Valuing Voices
Ideas we’re not ready for are the ones we need most
Guest Post: Barbara Geary Truan
Some ideas precede and shape reality, the ones that lie behind everything we create. Other ideas lag behind an often hidden reality: they seek then find it. Galileo Galilei observed the skies for years before coming up with the notion that the earth was revolving around the sun. Both kinds of ideas have something essential to offer development professionals today.
The first tells us that if we can envision a world without poverty, we can create it. But it also begs the question of why, in spite of the last 50 years of development work, more than a billion people today live on less than $1.25, putting them below the internationally accepted norm of extreme poverty. This, I think, is where the second kind of idea comes in.
If we gaze at the development skies, we might begin to sense a centre around which both poverty and wealth rotate; we might come to recognise these opposites as distinct faces of the industrial revolution and the mechanistic thinking that underpins it.
With machines (technology, automatic process) as the ground of our systems, the reigning values are: mass production, identical forms, standard measurements. And the requirement is: unlimited consumption, both of raw materials and finished products. An unintentional result is that the system tends to overlook and undervalue a whole set of powerful variables and realities – those which are distinctly human.
What would happen if we allowed development thinking to cut itself loose from the tentacles of a mechanised view of the world? What if, for example, we introduced a quality like humility – the kind of humility Galileo experienced when gazing at the skies – into our thinking? What if we concentrated as much on what we don’t know as on what we do know? What if we threw out the molds and conveyor belts of development initiatives and decided to let it be an exclusively human endeavour?
I think if we had the courage required for such an undertaking, three exciting things would happen:
1. We would shift the order of the “five Ws”: what, where and when would play second fiddle to WHO and WHY. To stop assuming that these last two questions always have the same generic answers (“poor people” and “we want to help”) can be powerful. These assumptions become labels and labels become reductionist. Making them more specific opens wide two windows of possibility we often forget are there. A specific and thus enriched “who” automatically introduces collaboration, partnership and diversity into our work. A deeply specific, clearly articulated “why” has the potential to keep it honest.
2. We would shift our thinking about time: when time becomes human, we notice two things. First, we get to be in charge of it. Paying attention to detail, allowing time for deep thinking while planning, or shifting time frames to accommodate reality during a project, all give us sovereignty over time. Secondly, we would notice that creating wealth and health where poverty and illness have installed themselves takes time. Put simply, development projects that take a long view are the only ones that really work because, as in any relationship, commitment is a game changer.
3. We would see a different picture: when we allow development to be human, we begin to see and understand the human beings who have the most at stake in the process. And when we meet and know them, we look beyond the mere context of their lives to see how they live their lives. When we do this we see vast – seemingly unlimited – amounts of creativity, ingenuity, intelligence, courage, fortitude, to name just a few notable human qualities for which there are no standard measures.
These ideas aren’t theoretical. A few of the finest development initiatives I know of embody the human approach. In northwestern India, in a geographically isolated region with limited resources, an organisation called Shrujan has been quietly creating wealth and helping artisans sustain themselves while nurturing an integral part of their culture for almost fifty years. In South Africa, a project known as Umzi Wethu has come up with an original, dynamic and transferrable model for completely turning around the lives of vulnerable youth.
There are other examples like these. Valuing Voices exists to celebrate them and to encourage development professionals to look at them carefully and imitate what makes them so successful. Such work whispers of an unmoving sun, nudging us to imagine that there just may be another centre somewhere else in the sky: maybe even right in the heart of the enormous potential of what we label “poor”.
When Jindra invited me to write this blog, she mentioned that at a recent conference on development monitoring and evaluation, someone told her that Valuing Voices was an idea for which people were not yet ready. Hurray, I thought. That means she must really be on to something great.
Barbara Geary Truan holds a master’s degree in international relations from the Fletcher School. She worked for six years at the International Federation of Red Cross and Red Crescent Societies, and for over a decade as an evaluator for a philanthropical initiative that supports original working projects designed to better the world.