Browse by author
Lookup NU author(s): Dr Gavin Stewart
This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0).
© 2022 The Authors. Journal of Applied Ecology published by John Wiley & Sons Ltd on behalf of British Ecological Society. Variable study quality is a challenge for all the empirical sciences, but perhaps particularly for disciplines such as ecology where experimentation is frequently hampered by system complexity, scale and resourcing. The resulting heterogeneity, and the necessity of subsequently combining the results of different study designs, is a fundamental issue for evidence synthesis. We welcome the recognition of this issue by Christie et al. (2019) and their attempt to provide a generic approach to study quality assessment and meta-analytic weighting through an extensive simulation study. However, we have reservations about the true generality and usefulness of their derived study ‘accuracy weights’. First, the simulations of Christie et al. rely on a single approach to effect size calculation, resulting in the odd conclusion that before-after control-impact (BACI) designs are superior to randomised controlled trials (RCTs), which are normally considered the gold standard for causal inference. Second, the so-called ‘study quality’ scores have long been criticised in the epidemiological literature for failing to accurately summarise individual, study-specific drivers of bias and have been shown to be likely to retain bias and increase variance relative to meta-regression approaches that explicitly model such drivers. Synthesis and applications. We suggest that ecological meta-analysts spend more time critically, and transparently, appraising actual studies before synthesis, rather than relying on generic weights or weighting formulas to solve assumed issues; sensitivity analyses and hierarchical meta-regression are likely to be key tools in this work.
Author(s): Pescott OL, Stewart GB
Publication type: Article
Publication status: Published
Journal: Journal of Applied Ecology
Year: 2022
Volume: 59
Issue: 5
Pages: 1187-1190
Print publication date: 01/05/2022
Online publication date: 22/03/2022
Acceptance date: 20/10/2020
Date deposited: 20/04/2022
ISSN (print): 0021-8901
ISSN (electronic): 1365-2664
Publisher: Wiley-Blackwell Publishing Ltd.
URL: https://doi.org/10.1111/1365-2664.14153
DOI: 10.1111/1365-2664.14153
Altmetrics provided by Altmetric
