Reblog Ex-post Eval Week: Measuring sustainability post-program –go in and stay for the learning! By Holta Trandafili

Ex-post Eval Week: Measuring sustainability post-program –go in and stay for the learning! By Holta Trandafili

 Holta Trandafili
Holta Trandafili

Greetings, I am Holta Trandafili, a researcher and evaluator captivated by sustainability theories and the sustainment of results. I believe that a thoughtful, systematic inquiry of what happens after an intervention ends adds value to what we know about sustainability. Since 2015 I have co-led ten post-program evaluations (also known as ex-posts) in Uganda, Kenya, Sri Lanka, India, Myanmar, and Bolivia. Their findings point to questions and issues of theory, measurement, and sustainability expectations relevant to any program:

  • To what do we compare results to judge success? Is it that 60% of community groups or water points being operational three years after closeout a good result? Should it be 87% or 90%? Why? Should we use the end-line as the measuring yardstick, especially as contexts change? Whose view of success counts?
  • How long should we expect results, or community groups left behind, or activities to continue post-program? Two or ten years or Forever? Why?
  • Is going back once enough to make a judgment on sustainability? What would we find if we went back in 2020 where we evaluated ex-post in 2015 or even 2019?

Lessons Learned: Here are my reflections and resources on sustainability:

To the enthusiastic evaluators ready to start ex-posts

  • Lesson learned: Organizations often carry out ex-posts for accountability. However, greater wealth lays in learning. Make learning part of your evaluation objectives. It took my organization 5 years from the first ex-post to have more open conversations and share our sustainability learning on what to improve: how we design, transition, and measure programs’ impact. Now we are genuinely more accountable.
  • Get involved: Don’t lose heart if your first ex-posts prove difficult to conduct or have mixed results or unearth new questions and insights on sustainability. You are not alone. Find another evaluator that has gone through an ex-post experience and ask them to write a blog, present at a conference, write guidelines, attend a course, or merely meet to vent and dream.

To those already fighting to mainstream ex-post measurement in their organizations or their clients

Hot Tips:

  • Mainstreaming ex-post evaluations is commendable for any institution. In this process we should start making the case to pilot longitudinal ex-post measurements (i.e., going back not once but several points in time). We can truly unpack the issues of temporality and longevity for sustainment of results. See JICA’s example on ex-post monitoring.
  • Invest in theory-driven evaluations like Realist Evaluation to unpack the hidden mechanisms behind which different types of outcomes are sustained, asking: among whom, in what contexts, how, why?

Rad Resources:

ReBlog Ex-post Eval Week: Evaluating Peace for Sustainability by Peter Kimeu Ngui

Reblog Ex-post Eval Week: Evaluating Peace for Sustainability by Peter Kimeu Ngui

Peter Kimeu Ngui
Peter Kimeu Ngui

Peace is with you. My name is Peter Kimeu Ngui, the Founder and CEO of Decent Living Institute of Organic Farming with extensive experience on reflective peace and partnership building in Africa.

Is there anything like a ‘just world’? Can peace be ‘sustainable’? These two closely related questions must be in mind of evaluators of peace for sustainability in a community long after the funding expires. According to Catholic Relief Services’ Justice Lens, three key conditions that must exist in the vision of a just world and used by all sustainable peace evaluators.:

(1) sustainable right relationship for all humanity and nature

(2) the transformation of injustices in society, structures and policies with

(3) fairness of rights and responsibilities for sustained peace to all creation

The process of reflective peace building evaluation, including participatory creation of sustainability indicators demands a collaborative effort by all the stakeholders. This includes victims, perpetrators, peace builders/ advocacy agents who are necessary to establish and maintain a just environment.

Step one of the process involves reflecting on eight keys principles drawn from Pope Francis’ letter that awaken the conscience and human rights standards which fit into an acronym (CORDSSSS) for easy remembrance; Common Good, Option for the Poor, Rights and Responsibilities, Dignity and Equality, Social nature, Subsidiarity, Stewardship and Solidarity. Each team reflects on:

  • the definition of each principle
  • discussing what their world would be like if the principle was fully lived
  • what would be sustainable peace indicators (e.g. Increased peaceful resolutions administered by the community; peaceful co-existence by people in violent conflicts; reduced rape)
  • what needs to be different in the world (location) they live in?

Step two of the process is scenario building using the Appreciative Inquiry method that engages the participants in the evaluation. This involves finding out what:

  • sustainable values, customs, and traditions
  • policies and structures/ institutions exist in the community that promote/ advocate, promote, establish and maintain, monitor, and evaluate for sustainable peace building
  • the participants are led to dreaming of a just environment in their community and what needs be improved and put in place for this scenario to be realized.

Lessons Learned:

The evaluation teams are always and immediately energized to pursue the best scenario for peacebuilding, using the resources locally available.  The reflective peacebuilding process engages participants on real issues that directly affects the community, that the community is fears, worries, concerned with and they are immediately empowered with knowledge, using familiar traditional approach to resolve the violent conflict, and they self-discover that power to transform is with those engaged in the conflict.

The possibility of a just world becomes a reality amassing greater numbers of individuals to work for peace and work through these processes.

Lessons learned are self-created from local case study evaluation processes and appeal greatly to most of the community members

Rad Resources:

Reblog:Ex-post Eval Week: Are we serious about project sustainability and exit? By Abu Ala Hasan

Ex-post Eval Week: Are we serious about project sustainability and exit? By Abu Ala Hasan

I am Abu Ala Hasan, an independent consultant. I am an anthropologist, have been working in the NGO sector for 17 years. A large portion of my work are evaluations and research.

In 2018 we (Jindra Cekan and I) were selected to design exit strategy and learning documentation for a client. When we started, we found, despite having a plan for devising exit strategy, the implementing organisations involved only paid attention to that at the end of the project, having only three months.

Most startling was the fact that the informants, ranging from partner NGO officials,  to community members involved gave us blank looks when asked about readiness for exit. Few NGO executives said that they would be able to continue some of the activities themselves. Almost none of the community members could answer exit questions; rather they tried to explain about the project services and their benefits.

People sitting around a table with large piece of paper with squares drawn in and round stickers in several of the squares.

Though it was thought provoking, it was not surprising at all due to existing NGO culture, where in practice, interventions are initiated and decisions made by the NGOs or funders, rather than the community.

So, why does it happen?  First and foremost, exit is not taken seriously. Rather than trying to ensure sustainability of the project from the beginning or designing the project to achieve it, it is added, as if, an auxiliary component; apparently to fulfil criteria by outsiders. The word ‘exit strategy’ was mentioned twice and ‘sustainability’ five times in the action plan, a 38- page document, and we saw few activities.

While it takes time, the trend of project cycle has been increasingly heading towards shorter duration. A decade ago most projects were planned for five years or longer but now one to three years is the norm. This is not only counterproductive for sustainability but also detrimental to bring about significant social change (outcome/impact). The community members’ reactions indicated that sustainability was unfamiliar; it appeared, they were not informed or there was no such discussion with them. Planning for sustainability was not taken seriously.

Hot Tips:

To ensure sustainability and proper exit:

  • There should be sincere commitment and effort towards sustainability and exit from NGOs
  • Sustainability and exit planning should be built in the project implementation process from design stage. Projects should be designed for sustainability and not ignored
  • Involvement of the community members, their voice and participation in decision making at all stages is very important for sustainability and proper exit
  • Except for emergency projects, development projects should be longer in duration and properly evaluated periodically

Rad Resources:

For exit and sustainability planning these resources could be helpful:

This week, AEA365 is celebrating Ex-post Eval Week during which blog 

Upcoming Jan Webinar: Lessons from Nordic / the Netherlands’ ex-post project evaluations: 14 Jan 2021

Upcoming Webinar: Lessons from Nordic / the

Netherlands’ ex-post project evaluations: 14 Jan 2021

In June-August 2020, Preston Stewart, our Valuing Voices intern, conducted through government databases of the four Nordic countries – Norway, Finland, Sweden, and Denmark – and the Netherlands to identify ex-post evaluations of government-sponsored projects. The findings and recommendations for action are detailed in a four-part white paper series, beginning with a paper called The Search.

We would like to invite you to a presentation of our findings, and a discussion of what can be learned about:

1) the search process,

2) how ex-post evaluations are defined and categorized,

3) what was done well by each country’s ex-posts,

4) sustainability-related findings and lessons, and

5) what M&E experts in each country can improve on ex-post evaluation practices.

One big finding is that there were only 32 evaluations that seemed to be ex-post project, and only 1/2 of them actually were at least 2 years after project closure.

 

We have many more lessons about conflicting definitions, that ex-post evaluation is not the norm in the evaluations processes of the five governments, that development programs could, if committed to ex-post evaluation, learn about sustainability by engaging with the findings from many more such evaluations, and to increase accountability to the public and for transparent learning, ex-post evaluations should be shared in public, easy-to-access online repositories.

Join us!

REGISTER: https://www.eventbrite.com/e/webinar-lessons-from-nordic-the-netherlands-ex-post-project-evaluations-tickets-132856426147 

Your ticket purchase entitles you to the webinar, its meeting recording, associated documents, and online Sustainability Network membership for resources and discussion. Payment on a sliding, pay-as-you-can scale.

Interview repost: Why Measure & Evaluate Corporate Sustainability Projects?

Interview repost: Why Measure & Evaluate Corporate Sustainability Projects?

A CSR firm in Central Europe asked me to talk about sustainability, evaluation, and Corporate Social Responsibility. We had a terrific interview – please click thru to: https://www.besmarthead.com/cs/media/post/smartheadtalks-2-jindra-cekan-eng

 

 

The topics we discussed include:

* Sustainability in CSR involves value creation through benefits-creation for both companies and consumers

* Projects feeding into the United Nations Sustainable Development Goals may not be sustained over the long-term and we must return to evaluate what could be sustained and what emerged ex-post

* Scale measurable and meaningful impact investment through the transformation of development nonprofits’ programming + approach

* Three activities in sustainability that companies can start doing right away, that you would recommend to any company anywhere in the world

And Very Happy Holidays to all… may our lives and world be more sustainable in 2021….