Toggle Main Menu Toggle Search

Open Access padlockePrints

Combining judgements from correlated experts

Lookup NU author(s): Dr Kevin Wilson, Dr Malcolm Farrow


Full text for this publication is not currently held within this repository. Alternative links are provided below where available.


When combining the judgements of experts, there are potential correlations between the judgements. This could be as a result of individual experts being subject to the same biases consistently, different experts being subject to the same biases or experts sharing backgrounds and experience. In this chapter we consider the implications of these correlations for both behavioural and mathematical approaches to expert judgement aggregation. We introduce the ideas of mathematical and behavioural aggregation and identify the possible dependencies which may exist in expert judgement elicitation. We describe a number of mathematical methods for expert judgement aggregation, which fall into two broad categories; opinion pooling and Bayesian methods. We qualitatively evaluate which of these methods can incorporate correlations between experts. We also consider behavioural approaches to expert judgement aggregation and the potential effects of correlated experts in this context. We discuss the results of an investigation which evaluated the correlation present in 45 expert judgement studies and the effect of correlations on the resulting aggregated judgements from a subset of the mathematical methods. We see that, in general, Bayesian methods which incorporate correlations outperform mathematical methods which do not.

Publication metadata

Author(s): Wilson KJ, Farrow M

Editor(s): Luis C. Dias, Alec Morton, John Quigley

Publication type: Book Chapter

Publication status: Published

Book Title: Elicitation: the Science and Art of Structuring Judgement

Year: 2018

Volume: 261

Pages: 211-240

Print publication date: 20/12/2017

Online publication date: 18/11/2017

Acceptance date: 20/06/2017

Series Title: International Series in Operations Research & Management Science

Publisher: Springer

Place Published: New York


DOI: 10.1007/978-3-319-65052-4_9

Notes: eISBN: 9783319650524

Library holdings: Search Newcastle University Library for this item

ISBN: 9783319650517