Combining judgements from correlated experts

Wilson, KJ; Farrow, M

doi:10.1007/978-3-319-65052-4_9

Combining judgements from correlated experts

Lookup NU author(s): Professor Kevin Wilson, Dr Malcolm Farrow

Downloads

Full text for this publication is not currently held within this repository. Alternative links are provided below where available.

Abstract

When combining the judgements of experts, there are potential correlations between the judgements. This could be as a result of individual experts being subject to the same biases consistently, different experts being subject to the same biases or experts sharing backgrounds and experience. In this chapter we consider the implications of these correlations for both behavioural and mathematical approaches to expert judgement aggregation. We introduce the ideas of mathematical and behavioural aggregation and identify the possible dependencies which may exist in expert judgement elicitation. We describe a number of mathematical methods for expert judgement aggregation, which fall into two broad categories; opinion pooling and Bayesian methods. We qualitatively evaluate which of these methods can incorporate correlations between experts. We also consider behavioural approaches to expert judgement aggregation and the potential effects of correlated experts in this context. We discuss the results of an investigation which evaluated the correlation present in 45 expert judgement studies and the effect of correlations on the resulting aggregated judgements from a subset of the mathematical methods. We see that, in general, Bayesian methods which incorporate correlations outperform mathematical methods which do not.