The study report notes a variety of pathways and procedures for evaluation and monitoring. Many donor agencies commission evaluations of development projects they fund NGOs to implement. Some larger NGOs themselves carry out internal project evaluations, or undertake impact assessments of their own or others' activities. The range of criteria NGOs employ when monitoring impact is continually on the increase and now broadly resembles the wide array of criteria applied by consultants and others in the course of donor-commissioned evaluations. For instance, much faith is placed in relevance to local needs or circumstances, achievement of objectives, efficiency, effectiveness and sustainability.

Comparing NGO-run and donor-commissioned cases, the researchers found donor-commissioned studies tended to lack poverty analysis and cost-effectiveness analysis, whereas NGO-run evaluations tended to be more outspoken and critical in pinpointing weaknesses than donor-commissioned studies. As to what evaluations of NGOs revealed in terms of strengths and weaknesses, it was apparent that NGOs were more successful when implementing social projects and delivering services but considerably less so when moving into the economic sphere. More general lessons were the importance of building capacity for self-assessment among NGOs and taking astute account of external factors such the wider social and economic context, as potentially distorting influences.

Some NGOs voiced reservations over evaluations that focus exclusively on impact. If donors began to fund NGOs solely on the basis of impact criteria, the latter's ability to reach the poor could be impaired. Innovative and experimental work might be stillborn and attributes like originality and willingness to take risks that differentiated NGOs from other development players, might be eroded.

The Finnish study concludes that knowledge of the nature of development 'impact' needs a boost. Another priority need is improved methods of evaluation, with a sharper focus on appraisal, planning, establishing baselines and monitoring outcomes. Both needs are pressing, according to the OECD report, because impact data obtained by prevailing means can be misapplied in ways that highlight NGO weaknesses and play down NGO strengths. The report also calls for:

• more dialogue between donors and NGOs towards improved evaluation and monitoring yardsticks, with a view to expanding common ground and reducing potential conflict on this score
• further networking and information exchange among NGOs, backed by donors, to enable them share data on practical impact and evaluation methods
• more donor funds to help NGOs strengthen their own capacity to undertake evaluations and to encourage the identification and ownership of 'home-grown' indicators of performance
• further work to examine apparent trade-offs between reaching the poor and achieving financial sustainability on a wide scale
• fresh studies to clarify the role of beneficiaries vis-à-vis stake holders in evaluations
• greater focus in future impact assessment studies on sectoral and thematic issues.

Contributor(s): R. Riddell et al

Source(s):
1. Searching for Impact and Methods: NGO Evaluation Synthesis Study. OECD/DAC Expert Group on Evaluation, by R. Riddell et al. (1997) on behalf of the Expert Group on Evaluation of the Organisation for Economic Cooperation and Development
2. Searching for Impact and Methods: NGO Evaluation Synthesis Study: Volume II Appendices. Ministry for Foreign Affairs of Finland. Blue Series 1997:9 More information.

Funded by: Ministry of Foreign Affairs, Finland

Date: 98 December 17

Further Information:
R. Riddell
Research Fellow ODI, London
c/o Dr. Abby Riddell
DFID-PSU
PO Box 50050
Lusaka
Zambia

Tel: 00 260 1 250903
Email: r.riddell@pop3.zamnet.zm