AIC Small Project Report Evaluating participatory planning.pdf (1.27 MB)
Report: 'Evaluating the impacts of participatory planning for urban water infrastructure and rural livelihoods adaptation in Indonesia'
report
posted on 2017-02-07, 06:03 authored by Seona Meharg, James Butler, Dewi Kirono, Neil Lazarow, Hannah Barrowman, Kate DugganIn 2009-2014 the Department of Foreign Affairs and Trade (DFAT)-CSIRO Research for Development (R4D) Alliance (‘the Alliance’) brought together the research skills of CSIRO and its international research partners with the development knowledge and networks of the Australian aid program to enhance poverty reduction in the Southeast Asian region. The partnership provided an opportunity to develop innovative approaches to international aid delivery by better understanding the relationship between poverty and the environment. Six major collaborative projects were implemented, covering Vietnam (two projects), Bangladesh (one), Indonesia (two) and the Greater Mekong Region (one). The research investigated complex development challenges in the domains of climate, water resources, sustainable cities, and food security.
This Australia Indonesia Centre (AIC) Small Grants project enabled us to refine an evaluation methodology developed as part of the Alliance. In this case, the two Indonesian projects and partners were the focus: Makassar Sustainable Urban Development (with Hasanuddin University), and Climate Adaptation Strategies for Rural Livelihoods in Nusa Tenggara Barat Province (with the University of Mataram). The Small Grants project involved repeating evaluation workshops and stakeholder interviews in April-June 2015 that had first been undertaken at the end of the Alliance projects in April-June 2014, thus enabling the tracking of progress along each project’s Impact Pathway 1 year after project completion.
A significant limitation amongst many current impact evaluation methodologies is that a narrow set of approaches is often used to evaluate complex development issues. This means that impacts may be only partially described, and this is compounded by a lack of clarity around the validation and attribution of impacts to particular interventions. The Alliance approach applied mixed methods to overcome many of these challenges, allowing us to evaluate and quantify the impact of the individual projects, and to provide an opportunity for reflection by the project teams and hence further build capacity through learning.
Our approach was based on three facets:
1. A Theory of Change and Impact Pathway exercise and diagram which created a ‘roadmap’ for each project’s assumed progress and related outputs, outcomes, impacts and goals.
2. A self-reflection workshop amongst the Indonesian research partners which mapped key achievements against the project’s Impact Pathway, and discussed reasons for lack of progress and necessary remedial actions.
3. An impact evaluation survey of the research team and boundary partners, which asked interviewees to score 18 indicators linked to phases of the Theory of Change and Impact Pathway, providing additional perspectives and triangulation of results from the self-reflection workshops.
The results of the exercises demonstrated that with a small additional investment, project impacts can be effectively estimated and attributed. In addition, the process of self-reflection was shown to rekindle project teams’ efforts to maintain momentum, and to tackle barriers to impact that they had identified. We suggest that the Alliance methodology demonstrated by this Small Grants project could be usefully applied to other projects in the AIC portfolio.
