Sources of correlation between experts: Empirical results from two extremes
Through two studies, this report seeks to identify the sources of correlation, or dependence, between experts' estimates. Expert estimates are relied upon as sources of data whenever experimental data is lacking, such as in risk analyses and reliability assessments. Correlation between experts is a problem in the elicitation and subsequent use of subjective estimates. Until now, there have been no data confirming sources of correlation, although the experts' background is commonly speculated to be one. Two different populations of experts were administered questions in their areas of expertise. Data on their professional backgrounds and means of solving the questions were elicited using techniques from educational psychology and ethnography. The results from both studies indicate that the way in which an expert solves the problem is the major source of correlation. The experts' background can not be shown to be an important source of correlation nor to influence his choice of method for problem solution. From these results, some recommendations are given for the elicitation and use of expert opinion.
- Research Organization:
- Los Alamos National Lab., NM (USA); Nuclear Regulatory Commission, Washington, DC (USA). Office of Nuclear Regulatory Research
- DOE Contract Number:
- W-7405-ENG-36
- OSTI ID:
- 6624079
- Report Number(s):
- NUREG/CR-4814; LA-10918-MS; ON: TI87007619
- Country of Publication:
- United States
- Language:
- English
Similar Records
Intelligent-systems design: development of a framework and empirical assessment of knowledge presentation and reasoning in an expert-system interface
Eliciting and analyzing expert judgment: A practical guide