TY - JOUR

T1 - What role should randomized control trials play in providing the evidence base for conservation?

AU - Pynegar, Edwin L.

AU - Gibbons, James M.

AU - Asquith, Nigel M.

AU - Jones, Julia P.G.

PY - 2021/3

Y1 - 2021/3

N2 - There is general agreement that conservation decision-making should be evidence-informed, but many evaluations of intervention effectiveness do not attempt to account for confounding variables and so provide weak evidence. Randomised Control Trials (RCTs), in which experimental units are randomly allocated to treatment or control groups, offer an intuitive means of calculating the effect size of an intervention through establishing a reliable counterfactual and avoid the pitfalls of alternative quasi-experimental approaches. However, RCTs may not be the most appropriate way to answer some kinds of evaluation question, are not feasible in all circumstances, and factors such as spillover and behavioural effects risk prejudicing their quality. Some of these challenges may be greater in situations where the intervention aims to influence ecological outcomes through changing human behaviour (socio-ecological interventions). The external validity – the extent to which findings are generalizable – of RCT impact evaluation has also been questioned. We offer guidance and a series of criteria for deciding when RCTs may be a useful approach for evaluating the impact of conservation interventions, and what must be considered to ensure an RCT is of high quality. We illustrate this with examples from one of the few RCTs of a socio-ecological intervention – an incentive-based conservation program in the Bolivian Andes. Those who care about evidence-informed environmental management should aim to avoid a re-run of the polarized debate surrounding RCTs’ use in fields such as development economics and take a pragmatic approach to impact evaluation, while also actively integrating learning from these fields. If this can be achieved, they will have a useful role to play in robust impact evaluation.

AB - There is general agreement that conservation decision-making should be evidence-informed, but many evaluations of intervention effectiveness do not attempt to account for confounding variables and so provide weak evidence. Randomised Control Trials (RCTs), in which experimental units are randomly allocated to treatment or control groups, offer an intuitive means of calculating the effect size of an intervention through establishing a reliable counterfactual and avoid the pitfalls of alternative quasi-experimental approaches. However, RCTs may not be the most appropriate way to answer some kinds of evaluation question, are not feasible in all circumstances, and factors such as spillover and behavioural effects risk prejudicing their quality. Some of these challenges may be greater in situations where the intervention aims to influence ecological outcomes through changing human behaviour (socio-ecological interventions). The external validity – the extent to which findings are generalizable – of RCT impact evaluation has also been questioned. We offer guidance and a series of criteria for deciding when RCTs may be a useful approach for evaluating the impact of conservation interventions, and what must be considered to ensure an RCT is of high quality. We illustrate this with examples from one of the few RCTs of a socio-ecological intervention – an incentive-based conservation program in the Bolivian Andes. Those who care about evidence-informed environmental management should aim to avoid a re-run of the polarized debate surrounding RCTs’ use in fields such as development economics and take a pragmatic approach to impact evaluation, while also actively integrating learning from these fields. If this can be achieved, they will have a useful role to play in robust impact evaluation.

KW - Counterfactual

KW - RCTs

KW - effectiveness

KW - evidence

KW - impact evaluation

KW - randomization

KW - randomized control trials

U2 - 10.1017/S0030605319000188

DO - 10.1017/S0030605319000188

M3 - Article

VL - 55

SP - 235

EP - 244

JO - Oryx

JF - Oryx

SN - 0030-6053

IS - 2

ER -