Aminata and I both want to be proud…
I met Aminata in Mali in 1990 during my doctoral research. She was a Bambara farmer and an impressive woman, with pride in her community. She was a helpful informant during my research on how communities cope with famine, and how famine early-warning systems could support them to do so more effectively (Note: 'early' warning is often far too late to prevent anything, I found, as donors don't want to intervene until far too late).
She's stayed in my mind these 24 years, and I want her to be proud of our international development work. I too want to be proud of what we accomplish to alleviate poverty and ill health by improving lives and livelihoods. We do so much in agriculture/ natural resource management, health and nutrition, education, etc. Yet do we start with communities' burning needs? End with their evaluating us? Or do we mostly use them to implement our ideas?
I've been part of the problem, part of the development 'industry' for 25 years. While I've used participatory methods such as PRA/ RRA and Appreciative Inquiry, I've mainly been focused on answering donor questions as to how successful we were, rather than helping communities ask the questions they want to answer and get the resources to succeed. I am no longer as proud of that work. We've left African Aminatas, Asian Aainas, Latina Adrienna's and millions like them behind in our rush to implement, to get more funds, to succeed, win more, 'develop' more…..
In founding ValuingVoices I put myself in their shoes, seeing projects come, projects go, temporary changes coming, some unintended results coming, some building on projects coming, but mostly going. It's never hard to celebrate local capacity when surrounded by vibrant voices, teaching us about their communities and their needs. What is hard is when so much of those decisions are not in their hands, but are at the mercy of donors who often decided not to support them (after our fieldwork showing both need and great capacity was presented): 'this region was not a priority' or 'we are now working on a new sectoral focus or a new initiative and this is no longer in our priorities.'
I am that much more deeply encouraged by a recent article on nutrition that points the way toward building these women’s pride as well as my own by starting with their voices and with donors being willing to listen. Nutrition research in Rwanda suggests that scaling up community-based solutions is the way forward. They decreased (stunting) malnutrition by 18% in five years which is very fast by "'setting up an almost universal community-based health insurance scheme… with the help of [each village determining its own way to tackle malnutrition… and not packaged interventions provided by donors" said Fidele Ngabo, director of Maternal Child Health. The article says "the Rwandan model could be used in other African countries, where foreign donor-driven initiatives tend to focus on treatment and technical solutions…. Change will only come when nutrition research is led by Africa, and interventions are designed to meet a country’s priorities, according to the findings of a two-year European Union-funded SUNRAY (SUstainable Nutrition Research for Africa in the Years to come)."
(a future Aminata?)
This groundbreaking research highlights the issue so many of us encounter. "Researchers from developed countries search for African partners for joint research, based on funding and research priorities defined outside Africa… so, despite enormous amounts of money spent on nutrition research and interventions, malnutrition rates have not fallen [in Benin]. Instead, the research agenda should be based on needs identified within the continent. Calls for research proposals of donors should match this agenda… [and prioritize] the locally identified needs and priorities." That sounds almost heretical in some circles.
In Benin, "researchers and policymakers wait for 'the dictate of donors before taking action. Hence, donor-funded programmes aren’t sustainable. As soon as they end, all activities are stopped, and acquired benefits and good practices are lost,' said Eunice Nago Koukoubou of the Université d'Abomey-Calavi in Benin, an author of the published findings." Yet the crux of the problem is that donor funding is filling a gap not filled by national governments. It isn't even so much that they are unwilling, rather in the case of Benin, “the government is trying to raise funds for the [strategic plan where nutrition should be central in development]", which also needs technical capacity building, a means of carrying on data retention and learning and a means of sharing findings with each other. What is needed is a "partnership between African researchers, 'who have more credibility and knowledge of the context', and Western researchers with the resources and opportunities (e.g. the African Economic Research Consortium).
Until we get to national partnership, let's improve our local ones. Let's really listen to participants in co-designing activities in projects we still think are best. Let's build on and share SUNRAY's approach of starting with what Aminantas, Aainas, Adriennas want and get Rwanda's great results. Let's partner in the truest sense of the world. After all, this excellent article underscores that "ultimately it is about political will…. [some] who feel they lack resources to tackle their long-standing battle with chronic malnutrition have to realize that “your children are not the donors' children, they are yours." They are our children, at least we need to treat them with respect as if they were.
Prospects for long-term sustainability… or lack thereof at the Macro level (Part 2)
During 2003-2004, the OECD conducted a very interesting Ex-Post Evaluation Sustainability Summary that synthesizes four separate Regional Rural Development (RRD) projects, which has illuminated sustainability of the impacts of German RRD projects over nearly a 30-year period. The goal of this evaluation was to use the perspectives of local experts (evaluators and national consultants) to inform the Federal Ministry for Economic Cooperation and Development about the true impacts of four rural long-term projects.
The four projects evaluated were:
· Indonesia – Area Development Project in West Pasaman (1980-1992); German contribution: €32 million; Target population: 200,000 urban and rural people
· Sri Lanka – Regional Rural Development Project (RRDP) in Kandy (1987-2000); German contribution: €8.1 million; Target population: 200,000 rural small-scale producers
· Tanzania – Tanga Integrated Rural Development Program (TIRDEP) (1972-1993); German contribution: €75 million; Target population: 700,000 rural small-scale producers
· Zambia – Integrated Rural Development Program (IRDP) in Kabompo (1977-1993); German contribution: €30 million; Target population: 65,000 rural small-scale producers
While these programs “had been implemented in regions with very different underlying general conditions,” involving interventions of varying scales, with different socioeconomic conditions affecting the different countries and have effected a great number of changes in their respective countries, German’s RRD experts also drew similarly positive findings on outcomes and negative findings in terms of processes and the pitfalls of programming without focus on sustainability.
Here is the good news:
· “Living conditions of the target groups have improved in all four project regions,” with specific sustainable project outcomes observed in the “health and education sector, food security, increase in income and employment and the ensuing rise in the standard of living.” The projects were able to create these changes by improving infrastructure, enhancing the private sector economy, and supporting innovations in agriculture.
· Project planning was done largely on the German side, however it was determined that there was still a relatively high acceptance of the objectives by the participating stakeholders (locals) due to the establishment of joint objectives by locals and the implementing agencies.
Yet, there were also some factors that jeopardized the sustainability of positive results, which included the (non)maintenance of the infrastructure as well as the intensification of economic activities that led to negative ecological impacts (such as acidification of the soil and overfishing).
Even more interestingly, the OECD gets kudos from ValuingVoices for analyzing where the projects failed to meet their outcomes and sharing what led to unsustainable outcomes. Here is the bad news:
· There was low institutional sustainability at the level of state executing organizations for all four projects due to inadequate funds, inefficient organizational structures and a lack of coordination. Thus, viable exit and handover was limited. Due to this lack of institutional sustainability, it was concluded that, “the putting into place of new structures by means of development projects runs a very high risk of not being sustainable,” noting significant differences in local expectations of the partners and the concepts of the RRD. Hence imposing structures ex-post rather than designing with sustainability in mind doesn’t seem to work.
· There is also an issue of changing standards over time. These projects were completed from the 1970s-1990s, but older development schemes are not considered as relevant today. “The former RRD project concept is no longer pursued” today as a result of changes, such as economic reforms and decentralization, which led to the adoption of new concepts. This kind of risk bedevils all ex-post evaluations, akin to asking perfectly good donkeys why they are not thoroughbreds.
Further, the processes this collection of ex-post studies illuminates are key:
· “The expenses for the consultancy services of German experts were often considered as disproportionate compared with the hardware supplied. Here, partner expectations obviously were not consistent with those of the German side.” Potentially this is an incentive to use national evaluators. Yet the study concluded: “this serial evaluation has brought about a change in perspectives in part through the deployment of local experts. However, the qualification profile of locally available consultants varies considerable. The results of this serial evaluation do not suggest a general shift of evaluations to local organizations.” This study was 10 years ago so this has also begun to shift, even though pairing a ‘northern’ evaluator with ‘southern’ ones is more the norm.
· Finally, “the termination of German support was often considered abrupt and incomprehensible. Phasing out was done according to German views and did not take sufficient account of the partners’ views and needs as seen by them…[which] provides grounds for a systematic conflict between the interests of the partners and those of the German side.” This is key in reforming international development, from artificial timelines to those informed by adaptive learning, that takes into account and is even led by community and local NGO learning and readiness to take the helm themselves.
There is much to learn from OECD reviews such as this one about both the aims and processes for ex-post evaluation. What remains unclear in all ex-post evaluations Valuing Voices has found is what the organizations and donors have learned from them, and to what degree they have applied the lessons learned to the rest of their programming.
What are your experiences? Have lessons from evaluations been taken in? How? Why not?
'Developing them' fails (Haiti). Long live country-led development! (?)
I am tired of inefficiency and waste of tax dollars and human capacity and time. Some organizations use images of Africans, Asians, Latin Americans and Caribbean people suffering without permission to further their mission to get funds to do programming. United for Sight says it "evokes the idea that the poor are helpless and incapable of helping themselves, thereby cultivating a culture of paternalism." And that's what I am the very most tired of, our paternalism; this is the antithesis of country-led development.
Maybe we've gone about it a bit wrong. A case in point is a recent report on Haiti, that the US spent $2.8 billion and that there is little to show for it. Now, emergency response is truly a different kettle of fish – it is very expensive to intervene and restore even basic services and many nations have low aid-absorptive capacity. Nonetheless, it opens the door to unbridled waste, low capacity building and little sustainability as well. As Global Post and AP found, only 5% of 2012 US government spending went through Haitian organizations (and 95% through US and others) so the following dismal results are on our plate:
* "The number of people still living in grim encampments of quake survivors is now at 170,000" and while "The US promised to build 15,000 permanent homes but completed only 2,649 of them before ending its housing construction program, deciding instead to extend financing to Haitians directly for them to build their own homes." (I think at least we've tried to let people get in debt and find their own way to house themselves :), like us)
* The US is spending $170 million "to attract manufacturing to the new Caracol Industrial Park in Haiti’s North. But the massive investment was to have created 65, 000 jobs but has created fewer than 3,000 jobs, and the project may not have the funds to construct the port needed to export the industrial goods." Further, a US 2013 GAO audit found that there remained a deficit of over $100 million to complete it, with questionable prospects of success. (Not the best Return on Investment)
* “If the dismal record of USAID’s most trusted contractor is any indication, these entities are unreliable to say the least. A government audit of Haiti projects performed by USAID’s largest contractor, Chemonics, shows the company routinely failed to implement its aid projects correctly. In 2012, GlobalPost discovered that a $2-million, US-taxpayer funded USAID/Chemonics project constructed a building for Haiti’s parliament that was unfinished and unusable.”
As one Haitian professional said, "“There have been aid programs for such a long time here, but when you evaluate it, they don’t have durable impacts… [it] isn’t really helping the people with these problems be released from their problems, it only keeps them stuck in poverty.”
So this is what can happen when we know best. Again and again, with great hubris and waste, we know better than the country's nationals. I have worked in "international development" for 25 years and the title has always irked me. Are international development projects named aptly? Isn't it instead a matter of national development projects supported by international expertise and funding instead?!
There is good news. USAID recently launched its Forward Progress program, which aims to increase the amount of money it sends directly to local companies and NGOs (Congress permitting). "we have to support the institutions, private sector partners and civil society organizations that serve as engines of growth and progress for their own nations. USAID Forward is helping us to do that through new models for public-private partnerships and increased investment directly to partner governments and local organizations." The jury's still out about tangible results locally, but this sounds hopeful.
A delightful prospect of real country-led development potentially comes from a collaborative international donor-country partnership, CAADP. Comprehensive Africa Agriculture Development Programme's (CAADP) goal is to eliminate hunger and reduce poverty through agriculture by addressing "policy and capacity issues across the entire agricultural sector and African continent. CAADP is entirely African-led and African-owned and represents African leaders' collective vision for agriculture in Africa." Notably, putting their money where their mouths are, "African governments have agreed to increase public investment in agriculture by a minimum of 10 per cent of their national budgets and to raise agricultural productivity by at least 6 per cent."
So how are is CAADP doing after 10 years? The results are mixed. A 2013 IFPRI evaluation found that
* "Unlike programs that are largely driven by donor priorities, undermining local institutions in the process, CAADP is a product of the locally-sourced development goals of participating countries. This approach has resulted in a sense of ownership and better development outcomes" [yet]
* “The countries with the 10 largest agricultural sectors on the continent… have spent less than 5 percent of their total budget"
But while these results are mixed, so too are donor commitments. The 2011 midterm evaluation found that while there were compacts created in 22 countries, and 6 countries have been awarded GASFP funding totaling $270million, another 8 countries across West Africa (plus Ethiopia) received less than 50% of the funds promised to promote agriculture.
So what will it take for the poor to get the help they need from any of us? Do we value their voices? Is our international aid really helping them? You tell me.
Let's start turning the oceanliner of development to dignified sustainability – today!
No time like the present, our participants are waiting for dignified development to fully arrive. Dignity is the "quality or state of being worthy of esteem or respect." When we design projects with communities with long-term self-sustainability as the core value, we are respecting them.
And we must start local. As Acumen Fellow Natalie Grillon says from Uganda: "Ask more questions, listen and learn. I’m always trying to get better at listening to learn before I act so that my actions can lead to productive results based on consensus and conversations rather than assumptions. Don’t believe that you know the full story. The farmers for whom I work know their crop, know their land and know what they want."
She goes on to tell us how she stopped doing things for the project staff: " I could offer more in working alongside my colleagues to learn how to do new things together, like using a new database or thinking about ways to motivate teams… I could feel the sense of empowerment and excitement that could come from learning even a simple new thing and which only required a few minutes of our time" rather than just doing it herself for them.
This is what Valuing Voices is all about — seeing our participants as our true clients, as listening to what they want rather than designing development requests for proposal and even project proposals themselves in capital cities far from input by these very clients. Would any corporation in their right mind design without asking potential clients, without pre-testing and tweaking interventions before a full launch? Imagine how products would fare without marketing campaigns touting their benefits over other products? This doesn't happen often in development because participants are most often seen as 'price takers', grateful for what we offer, not matter if the fit is not perfect. Such an approach is not only inefficient, it is deeply disrespectful.
On the other hand, there are successes to be seen along the lines of localizing development. In addition to successes found in my other blogs (see Plan, Mercy Corps, LWR and even PACT – forthcoming), Devex recently posted an article praising Nuru and Millenium Villages for "working to develop local leaders who will fully take over the various programs they’ve begun after the foreigners leave in two years… [using] the opportunity to build up local leaders and engage them in developing their own communities." Hallelujah! Devex says Munk's critique of Millenium Villages is that "Millennium violates two basic principles of good development: It’s not scalable and it’s not sustainable". Nuru believes "poor people hold the keys to their own development."
Absolutely! Not only should design and implementation center around community wants and capacities, there are all sorts of project activities that communities can help sustain themselves:
* Agriculture and livelihoods (income generation, micro-credit, marketing),
* Natural resource management (climate-smart agriculture, reforestation),
* Literacy and numeracy.
While there are other things larger than communities that need ongoing external support for, how often are there referendums on what they would prioritize? The UN recently named 2014-2024 the sustainable energy decade and infrastructure such as roads, water systems as well as trade including the World Trade Organization's Doha Trade talks for improve the trading prospects of developing countries are vital, how often have citizens been asked? Ashoka's Changemakers supports projects that create feedback loops (like our own in East Africa: http://www.changemakers.com/project/valuing-voices-kenya).
Communities are deeply grateful for assistance yet they want to to have a voice, to steer the ship more themselves. The Listening Project found that "agencies should slow down and take more time to understand people’s capabilities, priorities, preferences, and ideas… [participants] don’t want to have aid agencies to be more extractive in how they gather information. They want to be part of the decision making process of aid efforts. This goes beyond two-way communication and requires rethinking many of our assumptions and processes to find ways to truly collaborate and support those who are affected."
Among the project's 6,000 interviewees, some wondered "why no one seems to check on whether the assistance provided has made a positive difference in recipients’ lives. It is important for aid agencies to have processes and mechanisms to receive and provide feedback to communities and to be accountable for their actions—and particularly for any harm that has been done."
Yes! We need to help those projects such as Nuru whose farmers are supported in self-sufficiency, countries that support communities evaluating our projects, for a start. That is the dignified life path to take as a development professional – as their peers.
Is Development Doomed? No! Youth and community input are key to sustainability
Paul Theroux has a deeply depressing, if quite realistic, view of how development has failed Africa. "Zambian-born economist Dambisa Moyo calls aid a "debilitating drug," arguing that "real per-capita income [in Africa] today is lower than it was in the 1970s, and more than 50% of the population — over 350 million people — live on less than a dollar a day, a figure that has nearly doubled in two decades." He notes that our desire to help is the very death knell for Africa: "50 years later, the education system in Malawi is still faltering. Why? Because teaching as a profession in Malawi, and many parts of Africa, is undervalued, if not despised, and poorly paid. Besides, you can always find a foreign teacher willing to do the work: American, British, Japanese, Australian." Later he notes that in our desire to help, we can harm: "A mud hut and thatched roof is, in fact, renewed every few years; the cement structures and tin roofs built by well-meaning NGOs create maintenance problems." If we keep doing things For Africa and the 'developing' world and don't support their own self-development, who wins? If we keep designing development projects because we know what they need, how sustainable will they be?
So what is to be done? What already exists is the OECD's DAC. It has lovely and broad-ranging criteria for evaluating impact at project's end. They use five kinds of measures. Now we need African, Asian, Latin American evaluators to see how relevant these are and how they'd change the measures with communities. I suspect that looking at projects from the participants' viewpoint rather than the donors could change how we look at these categories, and measure them:
1) Relevance (e.g. are the activities and outputs of the programme consistent with the overall goal and the attainment of its objectives and intended impacts and effects?)
2) Effectiveness ( (e.g. to what extent were the objectives achieved / are likely to be achieved and why?)
3) Efficiency (e.g. were activities cost- and time-efficient?)
4) Impact (e.g. what real difference has the activity made to the beneficiaries?)
And this is key:
5) Sustainability – "measuring whether the benefits of an activity are likely to continue after donor funding has been withdrawn. Projects need to be environmentally as well as financially sustainable… e.g. to what extent did the benefits of a programme or project continue after donor funding ceased? What were the major factors which influenced the achievement or non-achievement of sustainability of the programme or project?")
Some organizations use these measure for final evaluation, the worse news is that there are very few of them, there is pretty faulty knowledge sharing within organizations and across projects, so there's no guarantee that lessons from one project will be learned by the next. Even worse, no communities are at the helm of these evaluations, and the very way in which they are structured lead to skewed results. What can go wrong? In Ashoka's great Feedback Labs, project participants' "feedback was excessively positive. In focus group discussions we asked participants how comfortable they felt giving us feedback, including criticisms and suggestions. Their message was consistent: “you are our benefactors, we are grateful for any help provided, we would never criticize you"
* Minimize respondent bias by using national evaluators independent of project or donor staff, working with communities;
* Involve community feedback on its effectiveness from the project's start, building their buy-in ONLY if we make changes during implementation based on it, and use it to shape future projects.
* Involve youth as key respondents as their lives and livelihoods depend on vibrant communities who have been supported to create their best futures (rather than implementing projects that we international donors feel they should have and that help us get more funds while feeding our development industry). We can use SMS technology to crowdsource and inform what works versus what doesn't as UNICEF Uganda did to check on aid received, and we can use it to inform knowledge youth need to know as Pathfinder International for Mozambican family planning knowledge.
Why should we do that? Because they are the future. As Forbes cites a new Ashoka Article on youth as changemakers: "Currently, 70% of Africans are under the age of 30. By 2040, 50% of the world’s youth will be African, most of whom will be women and girls. With nearly half of the youth population in Africa currently unemployed or inactive—and 72% living on less than $2 per day—communities are finding it more urgent than ever to enable new avenues and solutions for the creation of quality jobs that make way for youth livelihoods and productivity. How these communities address this challenge could shape the future of the world in unprecedented ways….[one path is supporting] community-rooted innovators, social entrepreneurs with a long history of working to help communities become “unstuck” and help youth become changemakers. These innovators build solutions based on a deep understanding of the socio-economic context within their specific communities. They weave initiatives that flip problems into cross-sector, sustainable, community-driven solutions." We can support this through having them be the key engines for innovation in communities, shaping our development projects to meet their current needs.
So let's get started! Where have you seen success? What are the dangers of these approaches?
Bringing Mandela to grassroots evaluation
After South Africa's Nelson Mandela died, accolades flowed as did deep sorrow for a passing of a man of such forgiveness and peace. I too was one of millions who went to anti-apartheid rallies and rejoiced when he too stood before us after he was freed.
What struck me most were the posts that looked at how we could continue to manifest him: 1) South African business leader and head of GAIN, Jay Naidoo "…our beloved Madiba, our founding father of democracy… pledged 'Never, never and never again shall it be that this beautiful land will again experience the oppression of one by another…'
Madiba exuded the same serenity which in his life had filled the hearts of everyone he touched – the billions of the poor, marginalized and oppressed in the world…. even in death he was able to call out the humanity within each of us; to care, to embrace our people with integrity, humility. Leaving this solemn shrine I sensed the disciplined determination. We connected back to his fiery commitment and vowed not abandon dream of a better life for all. As we descended the stairs the helplessness and the fear evaporated. The streaming column of humanity was united by a single-minded focus on reclaiming the legacy of Mandela. They are reclaiming our shared past. And we were unafraid again.
We should not be looking for another Mandela. This week is an opportunity for us to search for the Mandela within each one of us. We want our leaders to live up to the morality of Madiba. We want our leaders to be honest. And to do what the father of our nation did so well: listen. Listen to the people before they stop listening to you; because the people have found the courage and fearlessness of Mandela to demand the promise of freedom now."
Secondly, amidst the chorus of accolades, a practical suggestion appeared, one which speaks to me about the longing for Africans to evaluate Africa. A leading member of AFREA (the African Evaluation Association), Issaka Traore suggested: "What about a Nelson Mandela Award for all upcoming AfrEA Conferences starting with the next one in Cameroon. We can do this in two ways:
1. Nelson Mandela Award for ‘Made in Africa approach to Evaluation’. With this Award at every conference AfrEA will give an Award to any Evaluator or group of Evaluators including Scholars, who have produced an Evaluation Paper or Evaluative Research Paper on ‘African communities knowledge, know-how or practices/skills in Evaluation’.
2. 7th AfrEA Conference Nelson Mandela Evaluation Prize. The organizing committee could launch a Prize for any Evaluator, group of Evaluator, National Evaluation who/that will compile all messages on Madiba following his death, pointing out content of these messages related to: the Relevance of his struggle, the Impact of his fight in South-Africa and Worldwide and finally the Sustainability of his vision."
Yes! How do we do this? There are newly appreciated participatory methodologies and studies such as Empowerment Evaluation and “Who Counts? The Power of Participatory Statistics.. New ways to understand development project participants includes 6,000 participants on aid impact research Time to Listen, plus other research that is fundamentally shifting the focus to “customers, clients, co-creators, or my current favorite, constituents”.
We need to incorporate their voices more strongly now — international development projects can use national evaluators to independently evaluate projects with communities, even if that adds a few days to current evaluation timelines. There is encouraging evidence that international projects are prioritizing the use of national evaluators, as Eval Partners' VOPE (a pairing of international and national evaluators) promotes, and a recent SIDW/ Charney review of USAID-funded developmental evaluation attests to. We can use mobile text/SMS technology to crowdsource feedback from participants on project impact and sustainability throughout current project implementation, as non-profits such as CRS, Mercy Corps and the CORE health consortium are successfully doing.
For as we try to manifest Mandela, we need to hear the voices of our participants. A Tanzanian friend reminded me that "we Africans have a saying. 'Until Lions have their own historians, the history of the jungle will always glorify a man'." So we need to support country-led evaluation, locally-led impact determination, designing and shaping their own development and history.