Prospects for long-term sustainability…or lack thereof at the Macro Level (Part 2)

Prospects for long-term sustainability… or lack thereof at the Macro level (Part 2)

 

During 2003-2004, the OECD conducted a very interesting Ex-Post Evaluation Sustainability Summary that synthesizes four separate Regional Rural Development (RRD) projects, which has illuminated sustainability of the impacts of German RRD projects over nearly a 30-year period.  The goal of this evaluation was to use the perspectives of local experts (evaluators and national consultants) to inform the Federal Ministry for Economic Cooperation and Development about the true impacts of four rural long-term projects.

 

The four projects evaluated were:

·      Indonesia – Area Development Project in West Pasaman (1980-1992); German contribution: €32 million; Target population: 200,000 urban and rural people

·      Sri Lanka – Regional Rural Development Project (RRDP) in Kandy (1987-2000); German contribution: €8.1 million; Target population: 200,000 rural small-scale producers

·      Tanzania – Tanga Integrated Rural Development Program (TIRDEP) (1972-1993); German contribution: €75 million; Target population: 700,000 rural small-scale producers

·      Zambia – Integrated Rural Development Program (IRDP) in Kabompo (1977-1993); German contribution: €30 million; Target population: 65,000 rural small-scale producers

 

While these programs “had been implemented in regions with very different underlying general conditions,” involving interventions of varying scales, with different socioeconomic conditions affecting the different countries and have effected a great number of changes in their respective countries, German’s RRD experts also drew similarly positive findings on outcomes and negative findings in terms of processes and the pitfalls of programming without focus on sustainability.

 

Here is the good news:
 

·      “Living conditions of the target groups have improved in all four project regions,” with specific sustainable project outcomes observed in the “health and education sector, food security, increase in income and employment and the ensuing rise in the standard of living.” The projects were able to create these changes by improving infrastructure, enhancing the private sector economy, and supporting innovations in agriculture.

·      Project planning was done largely on the German side, however it was determined that there was still a relatively high acceptance of the objectives by the participating stakeholders (locals) due to the establishment of joint objectives by locals and the implementing agencies.  

Yet, there were also some factors that jeopardized the sustainability of positive results, which included the (non)maintenance of the infrastructure as well as the intensification of economic activities that led to negative ecological impacts (such as acidification of the soil and overfishing).

Even more interestingly, the OECD gets kudos from ValuingVoices for analyzing where the projects failed to meet their outcomes and sharing what led to unsustainable outcomes. Here is the bad news:
 

·      There was low institutional sustainability at the level of state executing organizations for all four projects due to inadequate funds, inefficient organizational structures and a lack of coordination. Thus, viable exit and handover was limited. Due to this lack of institutional sustainability, it was concluded that, “the putting into place of new structures by means of development projects runs a very high risk of not being sustainable,” noting significant differences in local expectations of the partners and the concepts of the RRD. Hence imposing structures ex-post rather than designing with sustainability in mind doesn’t seem to work.

·      There is also an issue of changing standards over time. These projects were completed from the 1970s-1990s, but older development schemes are not considered as relevant today. “The former RRD project concept is no longer pursued” today as a result of changes, such as economic reforms and decentralization, which led to the adoption of new concepts. This kind of risk bedevils all ex-post evaluations, akin to asking perfectly good donkeys why they are not thoroughbreds.
 

Further, the processes this collection of ex-post studies illuminates are key:

 

·      “The expenses for the consultancy services of German experts were often considered as disproportionate compared with the hardware supplied. Here, partner expectations obviously were not consistent with those of the German side.” Potentially this is an incentive to use national evaluators. Yet the study concluded: “this serial evaluation has brought about a change in perspectives in part through the deployment of local experts. However, the qualification profile of locally available consultants varies considerable. The results of this serial evaluation do not suggest a general shift of evaluations to local organizations.” This study was 10 years ago so this has also begun to shift, even though pairing a ‘northern’ evaluator with ‘southern’ ones is more the norm.

 

·      Finally, “the termination of German support was often considered abrupt and incomprehensible. Phasing out was done according to German views and did not take sufficient account of the partners’ views and needs as seen by them…[which] provides grounds for a systematic conflict between the interests of the partners and those of the German side.”  This is key in reforming international development, from artificial timelines to those informed by adaptive learning, that takes into account and is even led by community and local NGO learning and readiness to take the helm themselves.

 

There is much to learn from OECD reviews such as this one about both the aims and processes for ex-post evaluation. What remains unclear in all ex-post evaluations Valuing Voices has found is what the organizations and donors have learned from them, and to what degree they have applied the lessons learned to the rest of their programming. 

What are your experiences?  Have lessons from evaluations been taken in? How? Why not?