Czech it out! Great evaluation happening in the Czech Republic
Czech it out! Great evaluation happening in the Czech Republic
One of the delights of living in another country is the surprises one encounters. For me, coming back to our second 'home', it was an evaluative surprise. For by connecting to the Czech Foreign Ministry's Evaluation team, I found evidence of learning from meta-evaluation, doing ex-post evaluation, conscientious tracking of project cost-effectiveness and an openness to self-sustainability research funding and using national evaluators to lead them.
Czech Foreign aid is widespread- "Through development cooperation, the Czech Republic helps to eradicate poverty in less developed parts of the world by means of sustainable socio-economic development. It also contributes to global security and stability, conflict prevention, the promotion of democracy, human rights and fundamental freedoms, and the rule of law". Development assistance is done by several entities, the main two under the Ministry of Foreign Affairs, ORS (Development Cooperation and Humanitarian Aid) and its subsidiary CRA (Czech Development Agency).
The Ministry of Foreign Affairs oversees some fascinating evaluation work. After attending several partner-donor meetings and presentations of a meta-evaluation, an ex-post from an array of projects in Georgia and a discussion of findings across all evaluations in 2014, I am impressed. Why? Because not only are they willing to learn from both successes and failures, openly discussing challenges in learning between grants and contracts, but also because they are tackling programming in 10 countries (e.g. Afghanistan, Bosnia and Herzegovina, Ethiopia, Georgia) with a mere 17 people and a budget of $35 million for 2015.
What are some of the things they are learning?
1. They commissioned a meta-evaluation looking at 20 projects from 2012-2013. What worked well was well described and documented evaluations that were also cost-effective (evaluations were 4% of total costs) and tried to offer constructive solutions to things that did not work well in projects. While some methodologies needed to be better, and reports were hard to access, a major finding was what needs to be improved is inclusion of local recipients in stakeholder analyses, soliciting their views on what the evaluations should focus and on how the projects affected them. Further, during discussions we highlighted the need for an evaluation of outcomes and impacts, not just how evaluations quality was but also which organizations had the best results and why.
2. They commissioned an ex-post evaluation across eight organizations' in the Republic of Georgia (5 Czech, 3 Georgian), of one-year projects with 131 separate activities in civic engagement, media and youth between 2008-2012. The evaluation looked at the short-term effectiveness and longer-term sustainability of activities in the Republic of Georgia. Key findings included good relevance of aid offered, high cost-efficiency, low effectiveness for Georgian decision-making, primarily individual (rather than systemic) sustainability, though some good impact.
Key recommendations from this evaluation-, which Valuing Voices thinks, are universal included:
LEARN BETTER TOGETHER
a. Implement min. 3-year projects, whereby focus in a selected region (or a few regions) on a selected local priority topic, ensure in-depth needs analysis, multi-stakeholder cooperation [including participants], sustainable mechanisms, ongoing local support and enough flexibility as per external factors.
b. Allocate budget for burning human rights issues and for enhancing planning, monitoring, evaluation and learning capacities of Civil Society Organizations.
and
SHARE FINDINGS MORE
c. Coordinate activities with other implementers and donors in the target area and if possible (taking into account the political situation) also with local state institutions and potential implementers
d. Implement multi-stakeholder initiatives in a specific area (health, environment, social inclusion, minorities) with an advocacy component, sharing of results / lessons learnt and a media component
3. Among annual recommendations from Evaluation discussions throughout the year emerged this surprising one on cost-effectiveness. A detailed financial budget is now standard, and expenses for project activities among a majority of (grant-funded) projects and the Czech Development Agency are now required. This enables cost-effectiveness comparisons at least across grants (albeing not across for-profit contracts). In my experience this is unparalleled! (Let me know if other countries do this please!)
Overall, the fact that the Czech Foreign Ministry and implementing partners are willing to look at themselves critically and transparently improve accountability to its ultimate recipients and taxpayers makes me shout Hurrah from all of Prague’s 100 spires! Here is one of them, taken from a Ministry’s window.
