Prospects for long-term sustainability…or lack thereof at the Macro Level (Part 2)

Prospects for long-term sustainability… or lack thereof at the Macro level (Part 2)


During 2003-2004, the OECD conducted a very interesting Ex-Post Evaluation Sustainability Summary that synthesizes four separate Regional Rural Development (RRD) projects, which has illuminated sustainability of the impacts of German RRD projects over nearly a 30-year period.  The goal of this evaluation was to use the perspectives of local experts (evaluators and national consultants) to inform the Federal Ministry for Economic Cooperation and Development about the true impacts of four rural long-term projects.


The four projects evaluated were:

·      Indonesia – Area Development Project in West Pasaman (1980-1992); German contribution: €32 million; Target population: 200,000 urban and rural people

·      Sri Lanka – Regional Rural Development Project (RRDP) in Kandy (1987-2000); German contribution: €8.1 million; Target population: 200,000 rural small-scale producers

·      Tanzania – Tanga Integrated Rural Development Program (TIRDEP) (1972-1993); German contribution: €75 million; Target population: 700,000 rural small-scale producers

·      Zambia – Integrated Rural Development Program (IRDP) in Kabompo (1977-1993); German contribution: €30 million; Target population: 65,000 rural small-scale producers


While these programs “had been implemented in regions with very different underlying general conditions,” involving interventions of varying scales, with different socioeconomic conditions affecting the different countries and have effected a great number of changes in their respective countries, German’s RRD experts also drew similarly positive findings on outcomes and negative findings in terms of processes and the pitfalls of programming without focus on sustainability.


Here is the good news:

·      “Living conditions of the target groups have improved in all four project regions,” with specific sustainable project outcomes observed in the “health and education sector, food security, increase in income and employment and the ensuing rise in the standard of living.” The projects were able to create these changes by improving infrastructure, enhancing the private sector economy, and supporting innovations in agriculture.

·      Project planning was done largely on the German side, however it was determined that there was still a relatively high acceptance of the objectives by the participating stakeholders (locals) due to the establishment of joint objectives by locals and the implementing agencies.  

Yet, there were also some factors that jeopardized the sustainability of positive results, which included the (non)maintenance of the infrastructure as well as the intensification of economic activities that led to negative ecological impacts (such as acidification of the soil and overfishing).

looking backwards

Even more interestingly, the OECD gets kudos from ValuingVoices for analyzing where the projects failed to meet their outcomes and sharing what led to unsustainable outcomes. Here is the bad news:

·      There was low institutional sustainability at the level of state executing organizations for all four projects due to inadequate funds, inefficient organizational structures and a lack of coordination. Thus, viable exit and handover was limited. Due to this lack of institutional sustainability, it was concluded that, “the putting into place of new structures by means of development projects runs a very high risk of not being sustainable,” noting significant differences in local expectations of the partners and the concepts of the RRD. Hence imposing structures ex-post rather than designing with sustainability in mind doesn’t seem to work.

·      There is also an issue of changing standards over time. These projects were completed from the 1970s-1990s, but older development schemes are not considered as relevant today. “The former RRD project concept is no longer pursued” today as a result of changes, such as economic reforms and decentralization, which led to the adoption of new concepts. This kind of risk bedevils all ex-post evaluations, akin to asking perfectly good donkeys why they are not thoroughbreds.

Further, the processes this collection of ex-post studies illuminates are key:


·      “The expenses for the consultancy services of German experts were often considered as disproportionate compared with the hardware supplied. Here, partner expectations obviously were not consistent with those of the German side.” Potentially this is an incentive to use national evaluators. Yet the study concluded: “this serial evaluation has brought about a change in perspectives in part through the deployment of local experts. However, the qualification profile of locally available consultants varies considerable. The results of this serial evaluation do not suggest a general shift of evaluations to local organizations.” This study was 10 years ago so this has also begun to shift, even though pairing a ‘northern’ evaluator with ‘southern’ ones is more the norm.


·      Finally, “the termination of German support was often considered abrupt and incomprehensible. Phasing out was done according to German views and did not take sufficient account of the partners’ views and needs as seen by them…[which] provides grounds for a systematic conflict between the interests of the partners and those of the German side.”  This is key in reforming international development, from artificial timelines to those informed by adaptive learning, that takes into account and is even led by community and local NGO learning and readiness to take the helm themselves.


There is much to learn from OECD reviews such as this one about both the aims and processes for ex-post evaluation. What remains unclear in all ex-post evaluations Valuing Voices has found is what the organizations and donors have learned from them, and to what degree they have applied the lessons learned to the rest of their programming. 

What are your experiences?  Have lessons from evaluations been taken in? How? Why not?


Twigas (Participant self-sustainability) throughout ‘development time’- Design, Implementation, M&E and beyond

Twigas (Participant self-sustainability) throughout 'development time'- Design, Implementation, M&E and beyond


What if we saw our true clients (sometimes vulnerable twigas) as our project participants and wanted the return on investment of projects be maximally sustained?  How would this change how we design, implement, monitor and evaluate projects with country-leadership? With $1.5 trillion dollars in US and EU foreign aid spent since 2000, don't you think that our industry urgently needs feedback on what communities feel will be sustainable now, what interventions offer the likelihood of positive impact beyond the performance of the project’s planned (log-framed) activities, and what activities stood the test of time and our departure?

Shockingly, such learning is not happening today.  YET.

Country-led development through participation has been a clarion call for several years, including USAID Forward and EU’s interest in food security and development finance. The Paris Declaration on Aid Effectiveness is a clarion call supporting country-led development. Multilaterals do some work to evaluate their projects; a handful of international non-profits do a small amount of evaluation post-project; and non-profits do some work using mobile technology to harness community feedback. Africa’s CAADP also manifests country-led and partially country-financed development (5-10% of GDP budgeted for Agriculture).  Aid transparency is gaining support, and ValuingVoices adds a much needed local voice in terms of transparent, effective assistance.  Much more funding is emerging to foster grassroots participation (e.g., Ashoka, Rockefeller and other funding), ranging from lessons learned from 6,000 aid recipients by Time to Listen to books touting Participatory Statistics, and the astonishing growth of mobile crowdsourcing platforms such as Kenya’s Ushahidi and Ghana’s Esoko. IOCE’s new evaluation initiative (VOPE) that is building capacity of national evaluators will also be a key resource.

So that's the good news coming up the riverbank. When should we be talking to communities about their views of what they feel is most sustainable and that they can sustain themselves? ALL THE TIME.

1) During Design

Most of our projects are designed in capital cities by donors who are informed by past projects, research, industry trends, policy imperatives, and are limited by narrowly focused funding streams. Very few projects are designed by those who will live with the results, namely the communities themselves. Donors often develop proposal requirements with little input from locals until start-up, when implementers fine-tune. Communities only get what donors are willing to give and donors want results. Not just temporary results but sustained ones.  Such faulty design and lack of learning from evaluations—rarely asking our true clients, our participants, from the very start—leads to rusting tractors, farmers returning to traditional practices without resources to switch, latrines that are not used, etc. We mean well, but these practices are not working well.  We know this from working over 25 years in international development. We need to change that.


2) During Implementation, at Midterm and Final Evaluation

Communities participate in midterm and final evaluations but often the knowledge is a one-way street, out.  Hardly ever are findings shared with communities and joint re-design done to optimize prospects for self-sustainability of project activities.  Ongoing learning from implementation needs to be transparently captured and shared in open-data format for discussion and adaptation (along USAID’s CLA Approach – Collaborating, Learning and Adapting program activities).  Often sustainability is planned during the last 6 months of implementation, and handed to partners that are weakly-resourced, may contain untested assumptions, and haven’t had a chance to build up the organizational culture needed. Capacity building of partners to take over implementation also requires resources, which need to be budgeted in, and systems that partners co-create to fit their needs and ability to maintain.  Program evaluation data that exists is often locked away in .pdf files in repositories that are not often accessible by countries themselves, and lessons learned within doesn’t often inform future design even among the implementers, much less by locals. We must change that.


3) After Project Close-out

Very rarely do implementing agencies return 3, 5, 10 years after projects close and ask participants what is “still standing” that they managed to sustain themselves. How often do we take community members, local NGOs, national evaluators as the leaders of evaluations of long-term self-sustainability of our projects? Based on my research (www.ValuingVoices/blogs99% of international aid projects are not evaluated for sustainability or impact after project close by anyone, much less by the communities they are designed to serve.  We can change that.

This is what ValuingVoices is all about.


‘Developing them’ fails. Long live country-led development!(?)

'Developing them' fails (Haiti). Long live country-led development! (?)


I am tired of inefficiency and waste of tax dollars and human capacity and time. Some organizations use images of Africans, Asians, Latin Americans and Caribbean people suffering without permission to further their mission to get funds to do programming. United for Sight says it "evokes the idea that the poor are helpless and incapable of helping themselves, thereby cultivating a culture of paternalism." And that's what I am the very most tired of, our paternalism; this is the antithesis of country-led development.  


Maybe we've gone about it a bit wrong. A case in point is a recent report on Haiti, that the US spent $2.8 billion and that there is little to show for it. Now, emergency response is truly a different kettle of fish – it is very expensive to intervene and restore even basic services and many nations have low aid-absorptive capacity. Nonetheless, it opens the door to unbridled waste, low capacity building and little sustainability as well.   As Global Post and AP found, only 5% of 2012 US government spending went through Haitian organizations (and 95% through US and others) so the following dismal results are on our plate:  


* "The number of people still living in grim encampments of quake survivors is now at 170,000" and while "The US promised to build 15,000 permanent homes but completed only 2,649 of them before ending its housing construction program, deciding instead to extend financing to Haitians directly for them to build their own homes." (I think at least we've tried to let people get in debt and find their own way to house themselves :), like us)


* The US is spending $170 million "to attract manufacturing to the new Caracol Industrial Park in Haiti’s North. But the massive investment was to have created 65, 000 jobs but has created fewer than 3,000 jobs, and the project may not have the funds to construct the port needed to export the industrial goods." Further, a US 2013 GAO audit found that there remained a deficit of over $100 million to complete it, with questionable prospects of success. (Not the best Return on Investment)


* “If the dismal record of USAID’s most trusted contractor is any indication, these entities are unreliable to say the least. A government audit of Haiti projects performed by USAID’s largest contractor, Chemonics, shows the company routinely failed to implement its aid projects correctly. In 2012, GlobalPost discovered that a $2-million, US-taxpayer funded USAID/Chemonics project constructed a building for Haiti’s parliament that was unfinished and unusable.”


As one Haitian professional said, "“There have been aid programs for such a long time here, but when you evaluate it, they don’t have durable impacts… [it] isn’t really helping the people with these problems be released from their problems, it only keeps them stuck in poverty.”

So this is what can happen when we know best. Again and again, with great hubris and waste, we know better than the country's nationals. I have worked in "international development" for 25 years and the title has always irked me. Are international development projects named aptly? Isn't it instead a matter of national development projects supported by international expertise and funding instead?!   


There is good news. USAID recently launched its Forward Progress program, which aims to increase the amount of money it sends directly to local companies and NGOs (Congress permitting). "we have to support the institutions, private sector partners and civil society organizations that serve as engines of growth and progress for their own nations. USAID Forward is helping us to do that through new models for public-private partnerships and increased investment directly to partner governments and local organizations." The jury's still out about tangible results locally, but this sounds hopeful.  


A delightful prospect of real country-led development potentially comes from a collaborative international donor-country partnership, CAADP. Comprehensive Africa Agriculture Development Programme's (CAADP) goal is to eliminate hunger and reduce poverty through agriculture by addressing "policy and capacity issues across the entire agricultural sector and African continent. CAADP is entirely African-led and African-owned and represents African leaders' collective vision for agriculture in Africa." Notably, putting their money where their mouths are, "African governments have agreed to increase public investment in agriculture by a minimum of 10 per cent of their national budgets and to raise agricultural productivity by at least 6 per cent."  


So how are is CAADP doing after 10 years? The results are mixed. A 2013 IFPRI evaluation found that 

* "Unlike programs that are largely driven by donor priorities, undermining local institutions in the process, CAADP  is a product of the locally-sourced development goals of participating countries. This approach has resulted in a sense of ownership and better development outcomes" [yet] 

* “The countries with the 10 largest agricultural sectors on the continent… have spent less than 5 percent of their total budget"

But while these results are mixed, so too are donor commitments. The 2011 midterm evaluation found that while there were compacts created in 22 countries, and 6 countries have been awarded GASFP funding totaling $270million, another 8 countries across West Africa (plus Ethiopia) received less than 50% of the funds promised to promote agriculture.

So what will it take for the poor to get the help they need from any of us? Do we value their voices? Is our international aid really helping them? You tell me.