Toggle Main Menu Toggle Search

Open Access padlockePrints

An investigation of dependence in expert judgement studies with multiple experts

Lookup NU author(s): Dr Kevin Wilson



This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND).


Expert judgement plays an important role in forecasting and elsewhere as it can be used to quantify models when no data are available and to improve predictions from models when combined with data. In order to provide defensible estimates of unknowns in an analysis the judgements of multiple experts can be elicited. Mathematical aggregation methods can be used to combine these individual judgements into a single judgement for the decision maker. However, most mathematical aggregation methods assume judgements coming from experts that are independent. This is unlikely to be the case in practice. This paper investigates dependence in expert judgement studies, both within and between experts. It gives the most comprehensive analysis to date by considering all studies in the TU Delft database. It then assesses the practical significance of the dependencies identified in the studies by comparing the performance of several mathematical aggregation methods with varying dependence assumptions. Between expert correlations were more prevalent than within expert correlations. For studies which contained between expert correlations, models which include these improved forecasts. The implications for the use of expert judgement in forecasting are discussed.

Publication metadata

Author(s): Wilson KJ

Publication type: Article

Publication status: Published

Journal: International Journal of Forecasting

Year: 2017

Volume: 33

Issue: 1

Pages: 325–336

Print publication date: 01/01/2017

Online publication date: 22/03/2016

Acceptance date: 30/11/2015

Date deposited: 30/11/2015

ISSN (print): 0169-2070

ISSN (electronic): 1872-8200

Publisher: Elsevier


DOI: 10.1016/j.ijforecast.2015.11.014


Altmetrics provided by Altmetric