by Jindra Cekan | Mar 4, 2015 | Accountability, Aid effectiveness, Czech, Evidence-based policy, Mongolia, Sustainability, Sustainable development, Transparency
What remained of Mongolian kindergartens
In 2012, we have ventured with my colleague Marie Koerner through steppes of Mongolia to learn what remained of the Czech-funded mobile kindergartens. People remembered a great project, nomadic women teaching in mobile kindergartens, children learning playfully in a safe environment and integrating better to primary schools… and their parents having more time for work or smaller siblings. Well, not much remained of it 4 months after extended project end. Despite an official agreement, the Mongolian Ministry of Education did not provide a budget. It rather accepted a bigger donation from the Asian Development Bank for a similar project. Well, the new one did not benefit that many children in so many remote areas, but kindergartens had better qualified teachers, free meals and heating. And the donor was happy. Still, something remained of the Czech project – the positive attitude of the community towards pre-primary education and dedication to involve children at least in summer prep-schools organized by the government. Read the full evaluation report here in the Czech language.
The sustainability stories are diverse, yet factors are often similar. The devil is in the detail – for example, a dedicated village chief can make a big difference.
Sustainability or commercial continuity?
Based on different evaluations of Czech development projects conducted in 2010 to 2013, the Czech Ministry of Foreign Affairs started debating sustainability in 2013. Often, it was linked to commercial continuity – companies, ideally Czech, would continue to have business based on their previous successful engagement. Or at least income generation within the projects was highlighted. Questions arose if at least certain sectors, CSO or companies or types of projects are more sustainable than others, based on income generation or other factors.
What makes development projects sustainable?
Following the policy debate, the Czech platform of Civil Society Organisations – FoRS engaged Marie Koerner and me to analyse all evaluation reports, define what sustainability actually is and how it can be achieved. The key findings are below.
-
Project sustainability is linked to continued benefits for intended beneficiariesafter donor funding has been withdrawn. Not all project activities need to continue. Not all benefits need to be seen as meaningful by the beneficiaries. Income generation is not always possible and does not necessarily contribute to poverty reduction. Especially if the income is generated by the Czech companies, not by the local actors, who are responsible for sustaining project benefits.
-
Sustainability of the Czech Development Cooperation projects varies. Review of 37 publicly available evaluation reports of Czech development projects shows no correlation between sustainability and specific sectors, contracting authorities, implementers or countries of implementation has not been detected. It is rather external and internal factors shown below which influence project sustainability.
-
Sustainability factors are project specific and interdependent. They are external and internal. Often, even external factors can be influenced to achieve higher sustainability.
-
Sustainability can be influenced throughout the project cycle. Each stakeholder plays a certain role at each phase of the project cycle.
-
Good practice is available across sectors, including environment, agriculture, social development including education, economic development including energy as well as global development education and awareness raising. These sustainable Czech projects considered the key factors influencing sustainability in their plans and activities and thus contributed to lasting positive benefits for the recipients.
Overview of factors is displayed below:
Sustainability Factors
Read the publication below, including different good practices. You can download ithere.
What next? What is your experience?
Based on the study, the Czech Development Agency together with FoRS organised an internal expert workshop on 13 March 2014 to debate possible steps for increasing the sustainability of Czech bilateral projects.
The results are yet to be seen. Feel free to share your resources and experience!
by Jindra Cekan | Feb 13, 2015 | Accountability, Aid effectiveness, Bilateral organizations, Czech, Donor Driven Development, Donors, Evidence-based policy, ex-post evaluation, International aid, NGO, post-project evaluation, Republic of Georgia, self-sustainability, Sustainable development
Czech it out! Great evaluation happening in the Czech Republic
One of the delights of living in another country is the surprises one encounters. For me, coming back to our second 'home', it was an evaluative surprise. For by connecting to the Czech Foreign Ministry's Evaluation team, I found evidence of learning from meta-evaluation, doing ex-post evaluation, conscientious tracking of project cost-effectiveness and an openness to self-sustainability research funding and using national evaluators to lead them.
Czech Foreign aid is widespread- "Through development cooperation, the Czech Republic helps to eradicate poverty in less developed parts of the world by means of sustainable socio-economic development. It also contributes to global security and stability, conflict prevention, the promotion of democracy, human rights and fundamental freedoms, and the rule of law". Development assistance is done by several entities, the main two under the Ministry of Foreign Affairs, ORS (Development Cooperation and Humanitarian Aid) and its subsidiary CRA (Czech Development Agency).
The Ministry of Foreign Affairs oversees some fascinating evaluation work. After attending several partner-donor meetings and presentations of a meta-evaluation, an ex-post from an array of projects in Georgia and a discussion of findings across all evaluations in 2014, I am impressed. Why? Because not only are they willing to learn from both successes and failures, openly discussing challenges in learning between grants and contracts, but also because they are tackling programming in 10 countries (e.g. Afghanistan, Bosnia and Herzegovina, Ethiopia, Georgia) with a mere 17 people and a budget of $35 million for 2015.
What are some of the things they are learning?
1. They commissioned a meta-evaluation looking at 20 projects from 2012-2013. What worked well was well described and documented evaluations that were also cost-effective (evaluations were 4% of total costs) and tried to offer constructive solutions to things that did not work well in projects. While some methodologies needed to be better, and reports were hard to access, a major finding was what needs to be improved is inclusion of local recipients in stakeholder analyses, soliciting their views on what the evaluations should focus and on how the projects affected them. Further, during discussions we highlighted the need for an evaluation of outcomes and impacts, not just how evaluations quality was but also which organizations had the best results and why.
2. They commissioned an ex-post evaluation across eight organizations' in the Republic of Georgia (5 Czech, 3 Georgian), of one-year projects with 131 separate activities in civic engagement, media and youth between 2008-2012. The evaluation looked at the short-term effectiveness and longer-term sustainability of activities in the Republic of Georgia. Key findings included good relevance of aid offered, high cost-efficiency, low effectiveness for Georgian decision-making, primarily individual (rather than systemic) sustainability, though some good impact.
Key recommendations from this evaluation-, which Valuing Voices thinks, are universal included:
LEARN BETTER TOGETHER
a. Implement min. 3-year projects, whereby focus in a selected region (or a few regions) on a selected local priority topic, ensure in-depth needs analysis, multi-stakeholder cooperation [including participants], sustainable mechanisms, ongoing local support and enough flexibility as per external factors.
b. Allocate budget for burning human rights issues and for enhancing planning, monitoring, evaluation and learning capacities of Civil Society Organizations.
and
SHARE FINDINGS MORE
c. Coordinate activities with other implementers and donors in the target area and if possible (taking into account the political situation) also with local state institutions and potential implementers
d. Implement multi-stakeholder initiatives in a specific area (health, environment, social inclusion, minorities) with an advocacy component, sharing of results / lessons learnt and a media component
3. Among annual recommendations from Evaluation discussions throughout the year emerged this surprising one on cost-effectiveness. A detailed financial budget is now standard, and expenses for project activities among a majority of (grant-funded) projects and the Czech Development Agency are now required. This enables cost-effectiveness comparisons at least across grants (albeing not across for-profit contracts). In my experience this is unparalleled! (Let me know if other countries do this please!)
Overall, the fact that the Czech Foreign Ministry and implementing partners are willing to look at themselves critically and transparently improve accountability to its ultimate recipients and taxpayers makes me shout Hurrah from all of Prague’s 100 spires! Here is one of them, taken from a Ministry’s window.
