Toggle Main Menu Toggle Search

Open Access padlockePrints

ROBITT: A tool for assessing the risk-of-bias in studies of temporal trends in ecology

Lookup NU author(s): Dr Matthew Grainger, Dr Gavin Stewart



This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0).


© 2022 The Authors. Methods in Ecology and Evolution published by John Wiley & Sons Ltd on behalf of British Ecological Society. Aggregated species occurrence and abundance data from disparate sources are increasingly accessible to ecologists for the analysis of temporal trends in biodiversity. However, sampling biases relevant to any given research question are often poorly explored and infrequently reported; this can undermine statistical inference. In other disciplines, it is common for researchers to complete ‘risk-of-bias’ assessments to expose and document the potential for biases to undermine conclusions. The huge growth in available data, and recent controversies surrounding their use to infer temporal trends, indicate that similar assessments are urgently needed in ecology. We introduce ROBITT, a structured tool for assessing the ‘Risk-Of-Bias In studies of Temporal Trends in ecology’. ROBITT has a similar format to its counterparts in other disciplines: it comprises signalling questions designed to elicit information on the potential for bias in key study domains. In answering these, users will define study inferential goal(s) and relevant statistical target populations. This information is used to assess potential sampling biases across domains relevant to the research question (e.g. geography, taxonomy, environment), and how these vary through time. If assessments indicate biases, then users must clearly describe them and/or explain what mitigating action will be taken. Everything that users need to complete a ROBITT assessment is provided: the tool, a guidance document and a worked example. Following other disciplines, the tool and guidance document were developed through a consensus-forming process across experts working in relevant areas of ecology and evidence synthesis. We propose that researchers should be strongly encouraged to include a ROBITT assessment when publishing studies of biodiversity trends, especially when using aggregated data. This will help researchers to structure their thinking, clearly acknowledge potential sampling issues, highlight where expert consultation is required and provide an opportunity to describe data checks that might go unreported. ROBITT will also enable reviewers, editors and readers to establish how well research conclusions are supported given a dataset combined with some analytical approach. In turn, it should strengthen evidence-based policy and practice, reduce differing interpretations of data and provide a clearer picture of the uncertainties associated with our understanding of reality.

Publication metadata

Author(s): Boyd RJ, Powney GD, Burns F, Danet A, Duchenne F, Grainger MJ, Jarvis SG, Martin G, Nilsen EB, Porcher E, Stewart GB, Wilson OJ, Pescott OL

Publication type: Article

Publication status: Published

Journal: Methods in Ecology and Evolution

Year: 2022

Volume: 13

Issue: 7

Pages: 1497-1507

Print publication date: 04/07/2022

Online publication date: 28/03/2022

Acceptance date: 21/03/2022

Date deposited: 21/04/2022

ISSN (electronic): 2041-210X

Publisher: Wiley-Blackwell Publishing Ltd.


DOI: 10.1111/2041-210X.13857


Altmetrics provided by Altmetric


Funder referenceFunder name
European Research Council