Maximizing what we've got… Time is now!
We had a stirring conversation here in D.C. with someone very knowledgeable about sustainability; this person is a strong proponent of local ownership of all development. They also said vehemently, why evaluate the sustainability of projects after closeout; we all know what that will show! What was implied is that our system of international development and aid is so flawed, so broken, that the inevitable result of not focusing on local ownership as the fundamental basis for our work means Nothing. Will. Be. Left…. All. Is. Lost.
We disagree. Our development industry does some good, some bad, and is ever-changing (albeit slowly). Billions of dollars each year are spent trying to improve people's lives and livelihoods around the world, and we've seen great good be done. While. We. Remain.
We know far, far less about what remains after our projects end because less than 1% of the time we return post-project (ex-post) to evaluate anything.
Our problem with time begins with fixed timelines within projects that say we have 1, 3, 5 years to get to success. They work with participants and partners who need to make substantial changes to how they use their resources and beliefs over a relatively short time of a few months to a few years. We expect immediate results from them, changing how they farm (use new seeds, new methods, new ways of interacting with markets) and save money (learn new concepts of profit and interest, repayment and re-lending), and improve their health and that of their families (get prenatal exams, vaccinate your children, exclusively breastfeed without adding water or tea). Everyone in our projects is 'on the clock' from the donor and implementer to partners and participants. This clock ticks down irrevocably as project closeout looms, promised-successes-to-donors at hand or a mirage in the distance. We assume sustained results.
How many of us have ever gone on a diet? How many have learned a new language? How many of us have transferred jobs and had to learn new skills on the job? How quickly have we managed to do all that successfully, all at once?! Probably many. How many of you have had to do this on a fixed timeline? Were you successful when there was a limited, fixed time and you did not set your own pace?
It takes time to implement projects well enough to ensure that most participants ‘got it’, not just the 'early adopters'. It takes time to hand over projects so well that our partners and participants are ready to take over at least some of what we worked so hard to transfer. It took leadership and staff two years in the very successful participatory USAID/ Food For Peace food security project by CRS Niger that was a continuation of similar programming for 15 years. It also takes time to pass for conditions to be ready for our return, to isolate what people could self-sustain from what the project supplied, to learn what was so well designed and implemented during projects that to 'took root' in people's lives, that they have made it their own. We estimate optimal evaluation time is 2-7 years after closeout. Valuing Voices also believes we should not just evaluate the sustainability of outputs and outcomes of what we put in place that we thought they would continue, and the sustained impact of those cumulative investments, but also the emerging, unintended new activities and impacts we never imagined people would innovate from our projects. We are doing just such evaluations in Zimbabwe and Uganda now and hope to do and catalyze much more fieldwork around the world.
And why does it matter? Why shouldn't we write off our time-limited donor-funded projects? Because:
1) It's all we've got. Our current development system is not going anywhere soon, and there is success to learn from.
2) We need to quickly learn from what worked sustainably best and stop wasting time and resources on what we refuse to admit fails because we are too scared to return to see. Go back with the intention to learn what does and focus on doing more of what works.
3) Such analysis – and design of new projects – must have country ownership as a centerpiece throughout the project cycle assumptions, but to throw out decades of good work simply because we are just learning the value of country ownership is foolish.
Finally, here's a lovely example from Brazil of how local, participation (and yes, as my colleague thought, local ownership) works best. And. It. Takes. Time.
"Our results also show that Participatory Budgeting’s influence strengthens over time… Participatory Budgeting’s increasing impact indicates that governments, citizens, and civil society organizations are building new institutions… cities incorporate citizens at multiple moments of the policy process, allowing community leaders and public officials to exchange better information." How often do we return to do what are called longitudinal reviews of our work abroad, using the same rigorous standards we evaluate our domestic projects? Not often. Shouldn't that change?
Only by working together, honoring the value of our participants, that they deserve the same chance at change that we take for granted will things change. We must value both the voices of our participants and our own expertise for development to improve for true aid effectiveness…. Let us begin anew!
Sustainable Development Goals and Foreign Aid–
How Sustainable and Accountable to Whom?
Jindra Cekan, PhD of ValuingVoices
World leaders will paint New York City red next week at the UN Summit adopting the new post-2015 development agenda. The agreed plans set 17 new ‘Sustainable’ Development Goals (SDGs) to be achieved by 2030. These are successors to the Millennium Development Goals (MDGs).
While many of us have heard of them, how many of us know whether we met them and what prospects are there for the Sustainable Development Goals to do well or better? According to Bill Gates the MDGs were “the best idea for focusing the world on fighting global poverty that I have ever seen.” The Brookings Institute goes on to praise the eight MDG targets for aiming high by setting targets such as halving world poverty and reducing child mortality, improving universal primary education and gender equality and empower women etc. . Donor countries pledged three times more than they had until that time (raising the percentage of gross national income for international development assistance from 0.2% to 0.7%, not huge amounts but laudable). From 2000 to 2015 extreme poverty did fall by half (although some argue China and Asia were well on their way before this aid came) and in some countries (Senegal, Cambodia) child deaths fell by half. Global health improved via huge coalitions on immunization and HIV/AIDS. Yet while poverty dropped and health increased, hunger, environment and sanitation targets were not met, for instance, and there are 850 million people still hungry worldwide (11% of all people) . Yet gains far outweigh losses. The new Sustainable Development Goals are to be achieved in 15 years, by 2030. The UN and member nations will track a remarkable 169 indicators, “monitoring progress towards the SDGs at the local, national, regional, and global levels… 
Overall, one would feel rather tickled by these results — not 100% but still amazing given global disparities. Nancy Birdsall of the Center for Global Development thinks measurement is not the goal: “Growing global interconnectedness means that the problems the world faces, that hold back development, are increasingly shared… we’re making a promise to ourselves that we are one world, one planet, one society, one people, who look out for each other…” 
But I’m a fan of measurable results, I must admit. One would logically think that our international development projects funded by U.S. Agency for International Development (USAID), the U.S. Department of Agriculture (USDS), the Millennium Challenge Corporation and others could outline how the caused the good results that some MDGs showed. USAID’s website links its work to the MDGs clearly: “in September 2010, President Obama called for the elevation of development as a key pillar of America’s national security and foreign policy. This set forth a vision of an empowered and robust U.S. Agency for International Development that could lead the world in solving the greatest development challenges of our time and, ultimately, meet the goal of ending extreme poverty in the next generation” . It goes on to talk about work to “Promote sustainable development through high-impact partnerships and local solutions“ . USDA’s Foreign Agricultural Service states their “non-emergency food aid programs help meet recipients’ nutritional needs and also support agricultural development and education. These food assistance programs, combined with trade capacity building efforts, support long-term economic development” . Finally, MCC states they are “committed to delivering results throughout the entire lifecycle of its investments. From before investments begin to their completion and beyond… MCC’s continuum of results is designed to foster learning and accountability” .
Maybe. We don’t actually know because 99% of the time we never return to projects after they end to learn how sustainable they actually were. We could be fostering super-sustainability. Or not.
For international development programming works on 1-5 year programming cycles. Multi-million dollar project requests for proposals are designed and sent out by these funders to non-profit or for-profit implementers. These are awarded to one or more organization, quite rigorously monitored, and most have very good results. Then they end. Since 2000, the US Agency for International Development has spent $280 billion on country-to-country development and humanitarian aid projects as well as funding multilateral aid and in spite of much work evaluating the final impact of projects at the end, they never go back . The EU has spent a staggering $1.4 trillion. USAID has funded one evaluation that has gone back to see what communities and partners could sustain… in the last 30 years, and that is about to be published. A handful of international non-profits have taken matters in their own hands and funded such studies privately. The EU’s track record is even more dismal, with policies being proposed but not done . The World Bank, which has funded over 12,000 projects has an independent evaluation arm, the Independent Evaluation Group. They returned after projects closed out to evaluate results only 33 times and we found only three of them systematically talked to project participants about what was sustained.
The bottom line is, how do we know anything we’ve done in international development or SDGs is sustainable unless we go back to see? What amazing or awful results must we know for future design? If we do not return, are we really accountable to our taxpayers and our real clients: the participants and the national countries foreign aid recipients themselves?
The UN has pledged to have an SGD report card to “measure progress towards sustainable development and help ensure the accountability of all stakeholders for achieving the SDGs….[and a] Global Partnership for Sustainable Development Data, to help drive the Data Revolution….by using data we can ensure accountability for the policies that are implemented to reduce global and local inequities” . I completely agree that having citizen generated data at the local, national, regional, and global levels is so very important “to fill gaps in our knowledge, establish global norms and standards and…help countries develop robust national strategies for data development.” And as the World Bank IEG’s Caroline Heider states, measuring them is complex (e.g. agriculture affected by climate change and measuring changes across sectors is hard) but worthwhile .
While SDG data tells us what donor-funded activities and policies work, very few in international development know how sustainably our programming works for our ultimate clients, our participants and partners. And the price needn’t be high—a recent post-project evaluation we did cost under $120,000 which is a pittance given the project cost over $30 million and reached 500,000 folks. We found clear (mostly successful) lessons. USAID has, after 30 years, funded one post-project evaluation that also has clear cost-effective lessons (forthcoming). Really, in this era of cost-effectiveness, don’t we want data on what worked best (Note to self: do that more) and what worked least (Note to self: stop doing that)?
Learning what participants and partners could self-sustain after we left is actually all we should care about. They want to get beyond aid. Shouldn’t we know if we are getting them there? Self-sustainability of outcomes is a clear indicator of good Return on Investment of our resources and expertise and their time, effort and expertise. It shows us we want to put ourselves out of a job, having built country-led development that really has a future in-country with their own resources.
Two steps are:
1) Donors to add a funding equivalent to 1% of program value for five years after closeout for all projects over $10 million to support local capacity-building of NGOs and national partners to take over implementation plus to evaluate lessons across different sector’s sustainability outcomes.
2) A cross-donor fund for country-led analysis of such learning plus lessons for what capacity needs to be built in-country to take over programming. This needs support from regionally-based knowledge repositories and learning centers in Africa, Asia, etc. Online and tangible centers could house both implementer reports/ evaluations and analyze/ share lessons learned across sectors and countries from post-project evaluations for projects that closed out 2-7 years ago for future design.
Now that is accountability. Let’s advocate for sustainability funding, data and learning now.
What are your suggestions? How can we improve sustainability?
 McArthur, J. (2013, February 21). Own the Goals: What the Millennium Development Goals Have Accomplished. Retrieved from https://www.brookings.edu/articles/own-the-goals-what-the-millennium-development-goals-have-accomplished/
 End Poverty 2015 Millennium Campaign. (n.d.). MDG 1: Eradicate Extreme Poverty and Hunger. Retrieved from https://www.endpoverty2015.org/mdg-success-stories/mdg-1-end-hunger/
 Sharma, S. (2015, August 20). From Aspirations to Reality: How to Effectively Measure the Sustainable Development Goals. Retrieved from https://www.huffpost.com/entry/measuring-the-peoples-age_b_7999640
 Mirchandani, R. (2015, September 22). Does It Matter If We Don’t Achieve the SDGs? A New Podcast with Nancy Birdsall and Michael Elliott. Retrieved from https://www.cgdev.org/blog/does-it-matter-if-we-dont-achieve-sdgs-podcast-nancy-birdsall-and-michael-elliott
 USAID. (n.d.). USAID Forward. Retrieved from https://www.usaid.gov/usaidforward/usaid-forward-2014-archive
 United States Department of Agriculture (USDA). (n.d.). Food Assistance. Retrieved from https://www.fas.usda.gov/topics/food-assistance
 Millennium Challenge Corporation. (n.d.). Our Impact. Retrieved 2016, from https://web.archive.org/web/20160325135258/https://www.mcc.gov/our-impact
 Cekan, J. (2015, March 13). When Funders Move On. Retrieved from https://ssir.org/articles/entry/when_funders_move_on
 Global Issues. (n.d.). Foreign Aid for Development Assistance: Foreign Aid Numbers in Charts and Graphs. Retrieved from https://www.globalissues.org/print/article/35#globalissues-org
 Florio, M. (2009, November/December). Sixth European Conference on Evaluation of Cohesion Policy: Getting Incentives Right — Do We Need Ex Post CBA? Retrieved from https://ec.europa.eu/regional_policy/archive/conferences/evaluation2009/abstracts/florio.doc
 Heider, C. (2015, September 15). Evaluation Beyond 2015: Implications of the SDGs for Evaluation. Retrieved from https://ieg.worldbankgroup.org/blog/evaluation-beyond-2015-implications-sdgs-evaluation
Making money– microenterprise– is this a way to sustainable livelihoods? PACT’s Nepalese Lessons
Many Americans are steeped in the belief that we must ‘pull ourselves up by our bootstraps’, that hard work and especially faith in small businesses is the way to success. This is one of the many reasons why microfinance so appeals to donors as an investment. Does it work?
The US NGO-umbrella, Interaction, posted some “Aid Works” global results, including “the percentage of USAID-funded microfinance institutions that achieved financial sustainability jumped from 38% in 2000 to 76% in 2012.” Yet there have been numerous detractors of the model and the unsustainability of control over resources/ empowerment  .
What does one ex-post evaluations that we have on hand tell us? PACT’s USAID-funded WORTH program in Nepal was focused on women ending poverty through business, banking and literacy/ bookkeeping . The project, implemented between 1999 and 2001 worked with 240 local NGOs to reach 125,000 women in 6,000 economic groups across Nepal’s southern Terai (in 2001 a Maoist insurgency led to the groups being on their own) . By then, 1,500 of these groups led by the women themselves (35,000-strong) received training to become informal-sector Village Banks . Working with local NGOs enabled them to reach 100,000 women in a few months due to the NGOs’ presence and connections in the communities. The collaboration worked well due to a shared belief by PACT and the NGOs that dependency is not empowering. As the report says “WORTH groups and banks were explicitly envisaged as more than just microfinance providers; they were seen as organizations that would build up women as agents of change and development in their communities” .
In 2006, PACT and Nepalese Valley Research Group looked to see sustainability of the banks, the extent of retained income by the women as well as any effect on community development and broader issues such as domestic abuse . They went to 272 Banks from a random sample of 450 from seven of the 21 WORTH districts. Remarkably, they found even more functioning: 288 (16 more) of them were thriving and – wow- WORTH women had spawned another 400 more groups on their own . Participant interviews were done with members and management as well as those women who had left their Banks and members of groups that had dissolved plus they interviewed a ‘control group’ of poor, non-WORTH women in Village Bank communities.
Was it a universal success? Almost. See the bar chart below showing what impacts the management committee felt the village banks had had on members, which is mostly better off, some the same, some far better off. This held true for the original village bank members and the new bank members.
The SEEP network reviewed WORTH’s ex-post and found five key findings:
- Wealth creation: A Village Bank today holds average total assets of over Rs. 211,000, or $3,100, more than three times its holdings in 2001. Each woman member of WORTH now has an average equity stake of $116 in her Village Bank .
- Sustainability: Approximately two-thirds (64 percent) of the original 1,536 Village Banks are still active eight and a half years after the program began and five to six years after all WORTH-related support ended. That means there are nearly 1,000 surviving groups with approximately 25,000 members .
- Replication: A quarter of the existing WORTH groups has helped start an estimated 425 new groups involving another 11,000 women with neither external assistance nor prompting from WORTH itself. If all these groups are currently operating, then more Village Bankers are conducting business today in Nepal than when formal WORTH programming ended in 2001. The report also said 63% of the Village Bank members derived the income from agriculture/ sale of food versus 17% in commerce/ retail trade and the rest in miscellaneous trades. Over 40% of the participants said they borrowed to pay for education and health costs and another 20% to pay off other loans plus for festivals (e.g. birth, death) .
- Literacy: 97 percent of respondents reported that literacy is “very important” to their lives; 83 percent reported that because of WORTH they are able to send more of their children to school .
- Domestic disputes and violence: Two-thirds of groups reported that members bring their personal or family problems to the group for advice or help. Of these, three-quarters reported helping members deal with issues of domestic disputes and related problems. Forty-three percent of women said that their degree of freedom from domestic violence has changed because of their membership in a WORTH group. One in 10 reported that WORTH has actually helped “change her life” because of its impact on domestic violence .
The report outlines other impacts, including self-help actions such as two-thirds of groups being engaged in community action, and three-quarters said that the group has done something to help others in the community. Speaking of community, it is notable that the self-selected women were primarily from wealthier groups (60%), 15% from the middle class, with only 20% from the most disadvantaged castes . Frankly this is not as surprising, as those most willing to take on risk are rarely the poorest until later; 67% of the very poor later wanted to join such a bank (once the risk was shown not to be too high versus income) .
The study’s author asks “Yet for all this documented success, WORTH and other savings-led microfinance programs remain among the best kept secret in the world of international development and poverty alleviation. Although together such programs reach some two million poor people, they go almost unnoticed by the $20 billion credit-led microfinance industry… The empowered women in this study—like WORTH women elsewhere in Asia and Africa— have proved themselves equipped to lead a new generation of entrepreneurs who can take WORTH [onward] through a model of social franchising now being pilot-tested [which is] as creative and potentially groundbreaking as is WORTH…WORTH has the potential to become an “international movement that supports women’s efforts to lift themselves, their families, and their communities out of poverty” .
So why aren’t are we learning from such projects and scaling them up everywhere? PACT is . They have reached 365,000 women in 14 countries – including Myanmar, Cambodia, Colombia, Swaziland, DRC, Ethiopia, with Nigeria and Malawi starting this year . Coca-Cola awarded $400,000 to PACT in 2013 to replicate WORTH in Vietnam with 2,400 women . Who else is replicating this model? It’s not clear from many excellent microenterprise sites I visited except one tells me that Mastercard Foundation and Aga Khan are looking into wider replication as well. Let’s track their results and ask participants!
 Bateman, M. (2011, September 20). Microcredit doesn’t work – it’s now official. Retrieved from https://opinion.bdnews24.com/2011/09/20/microcredit-doesn%E2%80%99t-work-%E2%80%93-it%E2%80%99s-now-official/
 Vaessan, J., Rivas, A., & Duvendack, M. (2014, November). The Effects of Microcredit on Women’s Control Over Household Spending in Developing Countries: A Systematic Review and Meta-analysis. Retrieved from https://www.findevgateway.org/paper/2014/11/effects-microcredit-womens-control-over-household-spending-developing-countries
 Mayoux, L. (2008, June). Women Ending Poverty: The WORTH Program in Nepal – Empowerment through Literacy, Banking and Business 1999-2007. Retrieved from https://www.findevgateway.org/case-study/2008/06/women-ending-poverty-worth-program-nepal-empowerment-through-literacy-banking
 PACT. (n.d.). WORTH. Retrieved 2015, from https://web.archive.org/web/20141106013639/http://www.pactworld.org/worth
 PACT. (2013, August 13). The Coca-Cola Foundation awards $400,000 grant to Pact. Retrieved from https://www.pactworld.org/article/coca-cola-foundation-awards-400000-grant-pact
What should projects accomplish… and for whom?
An unnamed international non-profit client contacted me to evaluate their resilience project mid-stream, to gauge prospects for sustainable handover. EUREKA, I thought! After email discussions with them I drafted an evaluation process that included learning from a variety of stakeholders, ranging from Ministries, local government and the national University who were to take over the programming work about what they thought would be most sustainable once the project ended and how in the next two years the project could best foster self-sustainability by country-nationals. I projected several weeks for in-depth participatory discussions with local youth groups and sentinel communities directly affected by the food security/ climate change onslaught and who benefited from resilience activities to learn what had worked, what didn’t and who would take what self-responsibility locally going forward.
Pleased with myself, I sent off a detailed proposal. The non-profit soon answered that I hadn’t fully understood my task. In their view the main task at hand was to determine what the country needed the non-profit to keep doing, so the donor could be convinced to extend their (U.S.-based) funding. The question at hand became how could I change my evaluation to feed them back this key information for the next proposal design?
Maybe it was me, maybe it was the autumn winds, maybe it was my inability to sufficiently subsume long-term sustainability questions under shorter-term non-profit financing interests that led me to drop this. Maybe the elephant in the living room that is often unspoken is the need for some non-profits to prioritize their own organizational sustainability to ‘do good’ via donor funding rather than working for community self-sustainability.
Maybe donor/funders should share this blame, needing to push funding out, proving success at any cost to get more funding and so the cycle goes on. As a Feedback Lab feature on a Effective Philanthropy report recently stated: “Only rarely do funders ask, ‘What do the people you are trying to help actually think about what you are doing?’ Participants in the CEP study say that funders rarely provide the resources to find the answer. Nor do funders seem to care whether or not grantees are changing behavior and programs in response to how the ultimate beneficiaries respond” .
And how much responsibility do communities themselves hold for not balking? Why are they so often ‘price-takers’ (in economic terms) rather than ‘price-makers’? As wise Judi Aubel asked in a recent evaluation list-serve discussion “When will communities rise up to demand that the “development” resources designed to support/strengthen them be spent on programs/strategies which correspond to their concerns/priorities??”
We can help them do just that by creating good conditions for them to be heard. We can push advocates to work to ensure the incoming Sustainable Development Goals (post-MDGs) listen to what recipient nations feel are sustainable, more than funders. We can help their voices be heard via systems that enable donor/ implementers to learn from citizen feedback, such as Keystone has via their Constituent Voice practice (in January 2015 it is launching an online feedback data sharing platform called the Feedback Commons) or GlobalGiving’s new Effectiveness Dashboard (see Feedback Labs).
We can do it locally in our work in the field, shifting the focus from our expertise to theirs, from our powerfulness to theirs. In field evaluations can use Empowerment Evaluation. We can fund feedback loops pre-RFP (requests for proposals), during project design, implementation and beyond, with the right incentives tools for learning from community and local and national-level input so that country-led development begins to be actual not just a nice platitude. We can fund ValuingVoices’ self-sustainability research on what lasts after projects end. We can conserve project content and data in Open Data formats for long-term learning from country-nationals.
Most of all, we can honour our participants as experts, which is what I strive to do in my work. I’ll leave you with a story from Mali. in 1991 I was doing famine-prevention research in Koulikoro Mali where average rainfall is 100mm a year (4 inches). I accompanied women I was interviewing to a deep well which was 100m deep (300 feet). They used plastic pliable buckets and the first five drew up 90% of the bucket full. When I asked to try, they seriously gave me a bucket. I laughed, as did they when we saw that only 20% of my bucket was full. I had splashed the other 80% out on the way up. Who’s the expert?
How are we helping them get more of what they need, rather than what we are willing to give? How are we prioritizing their needs over our organizational income? How are we #ValuingVoices?
 The Center for Effective Philanthropy. (2014, October 27). Closing the Citizen Feedback Loop. Retrieved December 2014, from https://web.archive.org/web/20141031130101/https://feedbacklabs.org/closing-the-citizen-feedback-loop/
 Better Evaluation. (n.d.). Empowerment Evaluation. Retrieved December 2014, from https://www.betterevaluation.org/plan/approach/empowerment_evaluation
 Sonjara. (2016). Content and Data: Intangible Assets Part V. Retrieved from http://www.sonjara.com/blog?article_id=135