Generating better evidence for sustainable development research and evaluation
IIED is developing a body of work that seeks to understand how to develop better evidence for sustainable development research and evaluation.
An IIED workshop in September 2016 that focused on conceptualising 'better evidence' (Photo: Celie Manuel/IIED)
The concept of 'evidence' in the field of international development is broad and its definition and importance contested.
From the school of thought that says "if you can't measure it, you are guessing", to experiments in post-normal science for wicked problems, and impact evaluation proselytizing, engagement with evidence by actors in the development arena is diverse and serves many different purposes.
Most recently, the concept of 'alternative facts' has emerged, and there have been accusations that we are being subjected to culturally constructed ignorance, purposefully created by special interest groups working hard to create confusion and suppress 'the truth'; that we live now in a post-factual agnotological world.
Although there is an ever greater need for high quality evidence for driving processes towards better development outcomes, in a post-truth climate, there is a risk that prioritisation of high quality evidence may be threatened by insufficient investment and disagreement about what that evidence should look like.
Within IIED we are asking ourselves whether and how we can use the experiential learning of development NGOs and community-based organisations – who are working on multifaceted interventions in diverse contexts and achieving differing levels of impact – to shape and build better evidence.
We have consulted internally, both with the 'old guard' and with young innovators, and we have consulted externally with implementing partners and practitioners. An initial result of this consultation has been the identification of eight key characteristics defining what we call 'better evidence'.
Better evidence:
- Is well triangulated, using different sources of information and embracing multiple perspectives
- Is transparently based upon data and information processed through knowledge and experience, including local knowledge and collective experiences
- Addresses local issues and seeks to influence discourse, policy and/or practice at national and global
- Is communicable, well communicated and listened to
- Is action-oriented towards creating positive change
- Challenges perceived wisdom and/or reinforces alternative narratives where relevant
- Is cognisant of its own limitations in being the best evidence under the circumstances (such as financial, contextual, cultural, political), and
- Has a high level of validity and reliability, and is transparent about its own methodological basis.
A group of IIED researchers and evaluators have explored how methodological innovations and time-tested approaches may contribute to better evidence for sustainable development, and developed a better evidence philosophy for sustainable development. This includes seven case studies of methodological approaches that can generate better evidence.
A better evidence philosophy for sustainable development
Conceiving of better evidence is itself a dynamic process that improves the main products of our work through enquiry, investigation and research. But this does not deny the internal and external dialectical nature of the better evidence process.
We are involved in negotiating what constitutes evidence and how we can improve the way we generate, communicate and use it.
Methods toolkit
Community-driven data collection in informal settlements
This paper looks at the value of data gathering by local communities and how their evidence can help inform discussions about local priorities and help to build positive relationships with governments.
Knowledge-based participatory action research
This paper describes an approach that combines local and traditional knowledge systems with conventional research to generate qualitative and quantitative knowledge that is scientifically credible.
Participatory resource mapping
This paper describes an approach that enables communities to discuss and map out their physical environment and the resources they depend on in collaborative discussions with decision makers to find solutions to community planning problems.
Process tracing with Bayesian updating
This paper looks at process tracing with Bayesian updating, a case-based impact evaluation method that is useful for investigating how and why a particular change has come about using the systematic examination of evidence and establishing a specific degree of confidence in that evidence.
This paper explores how quasi-experimental methods can be used to demonstrate intervention effectiveness in local communities in a participatory and consultative manner, and living up to industry standards of transparency and validity.
Social assessment for protected areas
This paper describes an accessible, step-by-step approach to the social assessment of protected areas that uses rigorous and mixed data collection methods, such as randomised sampling, household surveys, focus group discussions and participatory workshops.
Theory-based impact evaluation
This paper looks at different approaches to theory-based impact evaluation methods that seek to find an explanation for what causes a given outcome, using different methods of data collection in situations where there are multiple complex causes and effects.
