How ‘new’ are our projects… and who is aiming at the right outcomes?
How ‘new’ are our projects… and who is aiming at the right outcomes?
Valuing Voices exciting news is we have received research grant funding from the esteemed evaluator, Michael Scriven’s Faster Forward Fund. We’re looking into the value-added of (ex-post) Sustained and Emerging Impacts Evaluations (SEIEs) and we are doing the research now. We will be documenting methods used and discuss how best to evaluate such sustained impacts after project close-out. Very exciting stuff in this staggeringly ‘new’ field of evaluation.
During this research, a senior international development evaluation expert told us that they can’t return to evaluate now-closed projects because they aren’t the same projects anymore (after closeout) and we are no longer responsible for the results. That took my breath away.
All new projects come from old projects… we recycle old project design most of the time, occasionally making substantive changes in targeting or design but much of how we design and implement remains the same. And while we thoroughly evaluate them during implementation, learning ex-post is a key missing link which all projects in the future can benefit from as we do similar interventions and track similar outcomes year after year but we rarely know which ones were sustained or emerged anew. There absolutely are aspects that get adapted but there are only so many ways to heal the sick, improve crop growth, save money, learn to read and so on, and there is a world we need to learn about what enabled some to be sustained and even morph into new results!
This excellent article, Do NGOs (non-governmental organization) help?, notes that “due to donor pressure [NGOs] are increasingly forced to respond with a discrete project with x number of deliverable outcomes” [1]. It goes on to cite D. Sriskandarajah, the secretary-general of Civicus, a global network of civil society organizations and activists, wrote: ‘We have become a part of the problem rather than the solution… Since demonstrating bang for your buck has become all-important, we divide our work into neat projects, taking on only those endeavours that can produce easily quantifiable outcomes. Reliant on funding to service our own sizeable organizations, we avoid approaches or issues that might threaten our brand or upset our donors. We trade in incremental change’” [1].
We also settle for results while we control them, and don’t ask unpopular questions about who is to sustain these results, with what resources, and for goodness sake, why sustained impact was not funded, designed, implemented and monitored/ evaluated from the very onset in our rush to measurable results?
As this great NGO article by Dinyar Godrej goes on to say, “most media scrutiny of NGO accountability is of how they use funds, their accountability to donors. But what of their accountability towards the recipients of their interventions” [1]? They have no lobbyists to persuade our funders they would like this but nto that, and often such lobbying for their needs falls to the very NGOs that have won these large contracts and tasked with implementing a dizzying array of mandatory input, output, outcome and some impact indicators. We do care deeply about results! US State Department/ USAID has a “Standard Foreign Assistance Master Indicator List” of 2,300 lines in an excel spreadsheet [2]. (There are more indicators still– custom and cross-cutting indicators, the mind boggles).
Wow. But are we asking the right questions? Are we asking what was sustained after all this hard work was done and ended? Rarely. Who should be?! “It is perhaps unrealistic to expect such large structural changes to be delivered by NGOs when governments don’t tackle them either.”
For the rub is this. When we take development over from national governments, largely do not involve country nationals in the funding, design and M&E of projects, then how sustained can these projects still be after we go? Millions are invested, then disappear… Last year, at local debrief at the end of one SEIE Valuing Voices did, the state of affairs became crystal clear when a government official asked us “Can you ever find some funds to fund us to do our own independent evaluations? Even if it is not the projects that they did themselves? We would be happy to get that support…”
When are we no longer responsible for doing great, sustained work? Valuing Voices will let you know what we found regarding the best ways to do SEIEs more. Stay tuned.
What do you think?
P.S. This blog topic prompted me to look for statistics on the number or percentage of funded projects that were renewed. Nothing. Does anyone know how many or what % of projects were extended/ funded again after showing good results? (Often this happens in the form that a successful project in one area of the country gets either funded again or repeated elsewhere in the country or in the world, as have two of our own SEIEs, Niger and Ethiopia). For that matter, what made them so excellent to be replicated? What can we learn?
Sources:
[1] Godrej, D. (2014, December 1). NGOs – Do They Help? Retrieved from https://newint.org/features/2014/12/01/ngos-keynote/
[2] US Department of State. (n.d.). Standard Foreign Assistance Indicators. Retrieved 2017, from https://web.archive.org/web/20170404072145/https://www.state.gov/f/indicators/index.htm
