Toggle Main Menu Toggle Search

Open Access padlockePrints

Practicalities of using a modified version of the Cochrane Collaboration risk of bias tool for randomised and non-randomised study designs applied in a health technology assessment setting

Lookup NU author(s): Rob Pickard


Full text for this publication is not currently held within this repository. Alternative links are provided below where available.


We describe our experience of using a modified version of the Cochrane risk of bias (RoB) tool for randomised and non-randomised comparative studies.ObjectivesTo assess time to complete RoB assessmentTo assess inter-rater agreementTo explore the association between RoB and treatment effect sizeMethodsCochrane risk of bias assessment was performed on a sample of full text primary reports included in a systematic review comparing operative techniques for radical prostatectomy. Inter-rater agreement was assessed using the kappa statistic.ResultsTwenty-four studies were judged as high overall RoB, 13 were judged as low RoB and 11 were unclear. The weighted Kappa value was 0.35 indicating fair agreement. The median (range) time taken to rate each study was 30min (10-49). The effect estimate for all studies was 0.61 (95% credible interval (CrI) 0.46-0.83) and 0.73 (95% CrI 0.29-1.75) for low risk studies.ConclusionsAlthough the process was time consuming, using a modified version of the RoB tool proved useful for demonstrating conservative effect estimates. That we only achieved a fair agreement between reviewers demonstrates the urgent need for further validation to improve inter-rater agreement. We suggest additional RoB levels could improve inter-rater reliability. (c) 2013 Crown copyright.

Publication metadata

Author(s): Robertson C, Ramsay C, Gurung T, Mowatt G, Pickard R, Sharma P, UK Robotic Laparoscopic Prostatect

Publication type: Article

Publication status: Published

Journal: Research Synthesis Methods

Year: 2014

Volume: 5

Issue: 3

Pages: 200-211

Print publication date: 01/09/2014

Online publication date: 14/11/2013

ISSN (print): 1759-2879

ISSN (electronic): 1759-2887

Publisher: Wiley-Blackwell Publishing


DOI: 10.1002/jrsm.1102


Altmetrics provided by Altmetric