Building the Evidence Base for Post Project Evaluation: A report to the Faster Forward Fund

» Posted by on Aug 2, 2017 in evaluability checklist, ex-post evaluation, Faster Forward Fund (3F), measuring sustainability checklist, Participation, post-project evaluation, Project cycle, SEIE, Sustainability, Sustained and Emerging Impact Evaluation (SEIE), Sustained and Emerging Impacts Evaluation, Valuing Voices | 1 comment

Building the Evidence Base for Post Project Evaluation: A report to the Faster Forward Fund

We are delighted to share Valuing Voices’ report on the value added of post-project evaluation, which compares findings from eight end-of-project and subsequent post project evaluations.  Many of you are aware of how rarely post project evaluations are undertaken.  As a result, there is little real evidence about project impact on long-term sustainability.   Valuing Voices received a grant from Michael Scriven’s Faster Forward Fund to begin to address this gap.

Our findings show that post project evaluations can contribute to better understanding of sustainability impacts, and reveal unexpected and emerging outcomes years after project close. They also indicate ways in which we can design and implement for sustainability.

Finding suitable projects for this review was difficult because so few post project evaluations are done, fewer are publically available, and fewer still had comparable final evaluations and included local voices.  Agencies that fund post project evaluations offer a range of reasons for doing so: to learn, to promote a success, to inform replication or scale, to provide justification for future funding, to promote accountabilities.  However, many funding agencies consider post project evaluation a luxury or not necessary.  JICA and OECD are notable exceptions in this regard.

Highlights include:

  • The review highlights the range of methods that have been used in post project evaluations, and point to the advantages of planning for sustainability measurement from the outset of the project.
  • The cases reviewed in the study highlight the (sometime dramatic) difference between the anticipated trajectory of a project, what is happening as the project ends, and what actually continued, was adapted, ceased or changed course after close out.
  • Taxonomies, knowledge management about evaluation, data retrieval/ retention, analysis, use and dissemination are elements of sustained impact evaluation that require attention.
  • Little documentation is available about how post project evaluations have actually informed and influenced organizational learning, sectoral dialogue or future programming.
  • Post project evaluations shed particularly interesting light on what emerged post-project that was entirely due to the efforts and resources of participants and partners after project investments stopped. More on these Sustained and Emerging Impacts Evaluations (SEIEs) at Better Evaluation.

As part of this report, Valuing Voices created an evaluability checklist for assessing whether a post project evaluation is viable, as well as a checklist for measuring sustainability starting at the beginning of the project cycle.

We welcome your comments on this report and checklists, and encourage you to share it in your networks and get us feedback on their use.  Please use the report and findings to advocate for more post project sustainability impact evaluations which will contribute to greater evidence-based learning about project sustainability.  Valuing Voices is among a handful of organizations who do post-project evaluations and we can either conduct one or refer you to another who does.

 

Thank you,

Laurie Zivetz, MPH, PhD and Jindra Cekan, PhD, with Kate Robins, MPH, PhD of Valuing Voices

 

 

31 years of Valuing Voices of national participants, project partners, donors and technical staff. Let’s have sustained impact!

Twitter 

1 Comment

  1. “I like the mention of a “Theory of Sustainability” for each project as a part of/complement to a Theory of Change…In other words, a sound TOC contains the assumptions around sustainability trajectories, contextual factors, and other assumptions that will ultimately dictate the sustainability of project outcomes/impacts.

    In some project design processes I’ve wondered whether we should use a TOC goal that goes beyond the project lifespan. I.e. if your five-year project begins in 2015 and your Results Framework goal is “By 2020, participants will…” we could set the corresponding Theory of Change goal as “By 2025, participants will…” thereby forcing us to build in TOC assumptions about what we expect beyond the life of activity. You’d still only technically be liable to deliver what’s in the RF, but you’d have to show convincingly why you believe certain assumptions will hold true for the longer term goal, which would force you to start thinking about life after the project from Day One. We could even make this a standard procedure for CRS project TOCs” CRS l Field RegionaStaffer

Trackbacks/Pingbacks

  1. Impact Investing – International Development’s New Holy Grail? - […] aid industry is so focused on showing results while conditions are (relatively) conducive, that far fewer than 1% of all…
  2. Local Accountability and Transparency… During and Post Project? | Valuing Voices - […] The scant number of post project sustainability evaluations have shown how rarely our international donor funded partners return to…
  3. Local Accountability and Transparency… During and Post Project? - […] The scant number of post project sustainability evaluations have shown how rarely our international donor funded partners return to partners…
  4. Assuming Sustainability and Impact is Dangerous to Development (+ evaluation criteria) | Valuing Voices - […] and impact, and assume positive development trajectories (Sridharan) unless we consistently do “ex-post” project evaluations such as these from our…
  5. Wishing for ex-post evaluation Christmas Lights rather than Needles in Haystacks | Valuing Voices - […] doing ex-post evaluation after this final evaluation. For more lessons on how random and misconstrued ex-posts can be, see…

Submit a Comment

Your email address will not be published. Required fields are marked *

*

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

Translate »