Toggle Main Menu Toggle Search

Open Access padlockePrints

Graphical approaches for the control of generalized error rates

Lookup NU author(s): Professor James WasonORCiD



This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0).


© 2020 The Authors. Statistics in Medicine published by John Wiley & Sons, Ltd. When simultaneously testing multiple hypotheses, the usual approach in the context of confirmatory clinical trials is to control the familywise error rate (FWER), which bounds the probability of making at least one false rejection. In many trial settings, these hypotheses will additionally have a hierarchical structure that reflects the relative importance and links between different clinical objectives. The graphical approach of Bretz et al (2009) is a flexible and easily communicable way of controlling the FWER while respecting complex trial objectives and multiple structured hypotheses. However, the FWER can be a very stringent criterion that leads to procedures with low power, and may not be appropriate in exploratory trial settings. This motivates controlling generalized error rates, particularly when the number of hypotheses tested is no longer small. We consider the generalized familywise error rate (k-FWER), which is the probability of making k or more false rejections, as well as the tail probability of the false discovery proportion (FDP), which is the probability that the proportion of false rejections is greater than some threshold. We also consider asymptotic control of the false discovery rate, which is the expectation of the FDP. In this article, we show how to control these generalized error rates when using the graphical approach and its extensions. We demonstrate the utility of the resulting graphical procedures on three clinical trial case studies.

Publication metadata

Author(s): Robertson DS, Wason JMS, Bretz F

Publication type: Article

Publication status: Published

Journal: Statistics in Medicine

Year: 2020

Volume: 39

Issue: 23

Pages: 3135-3155

Print publication date: 15/10/2020

Online publication date: 17/06/2020

Acceptance date: 10/05/2020

Date deposited: 08/07/2020

ISSN (print): 0277-6715

ISSN (electronic): 1097-0258

Publisher: John Wiley and Sons Ltd


DOI: 10.1002/sim.8595

PubMed id: 32557848


Altmetrics provided by Altmetric


Funder referenceFunder name
Biometrika Trust
UK Medical Research Council (