The weighted priors approach for combining expert opinions in logistic regression experiments

Quinlan, Kevin R.; Anderson-Cook, Christine M.; Myers, Kary L.

doi:10.1080/08982112.2017.1319956

Title: The weighted priors approach for combining expert opinions in logistic regression experiments

Abstract

When modeling the reliability of a system or component, it is not uncommon for more than one expert to provide very different prior estimates of the expected reliability as a function of an explanatory variable such as age or temperature. Our goal in this paper is to incorporate all information from the experts when choosing a design about which units to test. Bayesian design of experiments has been shown to be very successful for generalized linear models, including logistic regression models. We use this approach to develop methodology for the case where there are several potentially non-overlapping priors under consideration. While multiple priors have been used for analysis in the past, they have never been used in a design context. The Weighted Priors method performs well for a broad range of true underlying model parameter choices and is more robust when compared to other reasonable design choices. Finally, we illustrate the method through multiple scenarios and a motivating example. Additional figures for this article are available in the online supplementary information.

Authors:

Quinlan, Kevin R. ^[1]; Anderson-Cook, Christine M. ^[2]; Myers, Kary L. ^[2]

Pennsylvania State Univ., University Park, PA (United States). Dept. of Statistics
Los Alamos National Lab. (LANL), Los Alamos, NM (United States)

Publication Date:: Mon Apr 24 00:00:00 EDT 2017

Research Org.:: Los Alamos National Lab. (LANL), Los Alamos, NM (United States)

Sponsoring Org.:: USDOE

OSTI Identifier:: 1360704

Report Number(s):: LA-UR-16-24989
Journal ID: ISSN 0898-2112

Grant/Contract Number:: AC52-06NA25396

Resource Type:: Accepted Manuscript

Journal Name:: Quality Engineering

Additional Journal Information:: Journal Volume: 29; Journal Issue: 3; Journal ID: ISSN 0898-2112

Publisher:: American Society for Quality Control

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING; Mathematics

Citation Formats


                    Quinlan, Kevin R., Anderson-Cook, Christine M., and Myers, Kary L. The weighted priors approach for combining expert opinions in logistic regression experiments.  United States: N. p., 2017. 
Web.  doi:10.1080/08982112.2017.1319956.

Copy to clipboard


                    Quinlan, Kevin R., Anderson-Cook, Christine M., & Myers, Kary L. The weighted priors approach for combining expert opinions in logistic regression experiments.  United States.  https://doi.org/10.1080/08982112.2017.1319956

Copy to clipboard


                    Quinlan, Kevin R., Anderson-Cook, Christine M., and Myers, Kary L. Mon .  
"The weighted priors approach for combining expert opinions in logistic regression experiments".  United States.  https://doi.org/10.1080/08982112.2017.1319956.  https://www.osti.gov/servlets/purl/1360704.

Copy to clipboard


                    
@article{osti_1360704,

  title        = {The weighted priors approach for combining expert opinions in logistic regression experiments},

  author       = {Quinlan, Kevin R. and Anderson-Cook, Christine M. and Myers, Kary L.},

  abstractNote = {When modeling the reliability of a system or component, it is not uncommon for more than one expert to provide very different prior estimates of the expected reliability as a function of an explanatory variable such as age or temperature. Our goal in this paper is to incorporate all information from the experts when choosing a design about which units to test. Bayesian design of experiments has been shown to be very successful for generalized linear models, including logistic regression models. We use this approach to develop methodology for the case where there are several potentially non-overlapping priors under consideration. While multiple priors have been used for analysis in the past, they have never been used in a design context. The Weighted Priors method performs well for a broad range of true underlying model parameter choices and is more robust when compared to other reasonable design choices. Finally, we illustrate the method through multiple scenarios and a motivating example. Additional figures for this article are available in the online supplementary information.},

  doi          = {10.1080/08982112.2017.1319956},

  journal      = {Quality Engineering},

  number       = 3,

  volume       = 29,

  place        = {United States},

  year         = {Mon Apr 24 00:00:00 EDT 2017},

  month        = {Mon Apr 24 00:00:00 EDT 2017}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1080/08982112.2017.1319956

Other availability

Search WorldCat to find libraries that may hold this journal

Citation Metrics:

Cited by: 3 works

Citation information provided by
Web of Science

Save / Share:

Export Metadata

Save to My Library

Works referenced in this record:

Experimental Design for Binary Data
journal, March 1983

Abdelbasit, K. M.; Plackett, R. L.
Journal of the American Statistical Association, Vol. 78, Issue 381
DOI: 10.1080/01621459.1983.10477936

On the existence of maximum likelihood estimates in logistic regression models
journal, January 1984

Albert, A.; Anderson, J. A.
Biometrika, Vol. 71, Issue 1
DOI: 10.1093/biomet/71.1.1

Optimal Bayesian design applied to logistic regression experiments
journal, February 1989

Chaloner, Kathryn; Larntz, Kinley
Journal of Statistical Planning and Inference, Vol. 21, Issue 2
DOI: 10.1016/0378-3758(89)90004-9

Monte Carlo sampling methods using Markov chains and their applications
journal, April 1970

Hastings, W. K.
Biometrika, Vol. 57, Issue 1
DOI: 10.1093/biomet/57.1.97

Optimum Designs in Regression Problems
journal, June 1959

Kiefer, J.; Wolfowitz, J.
The Annals of Mathematical Statistics, Vol. 30, Issue 2
DOI: 10.1214/aoms/1177706252

Prediction of Reliability of an Arbitrary System from a Finite Population
journal, December 2010

Lu, Lu; Anderson-Cook, Christine M.
Quality Engineering, Vol. 23, Issue 1
DOI: 10.1080/08982112.2010.495259

The Coordinate-Exchange Algorithm for Constructing Exact Optimal Experimental Designs
journal, February 1995

Meyer, Ruth K.; Nachtsheim, Christopher J.
Technometrics, Vol. 37, Issue 1
DOI: 10.1080/00401706.1995.10485889

A Bayesian Model for Integrating Multiple Sources of Lifetime Information in System-Reliability Assessments
journal, April 2011

Reese, C. Shane; Wilson, Alyson G.; Guo, Jiqiang
Journal of Quality Technology, Vol. 43, Issue 2
DOI: 10.1080/00224065.2011.11917851

Works referencing / citing this record:

How to Host An Effective Data Competition: Statistical Advice for Competition Design and Analysis
journal, February 2019

Anderson‐Cook, Christine M.; Myers, Kary L.; Lu, Lu
Statistical Analysis and Data Mining: The ASA Data Science Journal, Vol. 12, Issue 4
DOI: 10.1002/sam.11404

Similar Records in DOE PAGES and OSTI.GOV collections:

Bayesian Design of Experiments for Logistic Regression to Evaluate Multiple Forensic Algorithms

Journal Article Quinlan, Kevin R. ; Anderson-Cook, Christine Michaela - Applied Stochastic Models in Business and Industry

When evaluating the performance of several forensic classification algorithms, it is desirable to construct a design that considers a variety of performance levels for each of the algorithms. We describe a strategy to use Bayesian design of experiments with multiple prior estimates to capture anticipated performance. Our goal is to characterize results from the different classification algorithms as a function of multiple explanatory variables and use this to choose a design about which units to test. Bayesian design of experiments has been successful for generalized linear models, including logistic regression models. Here, we develop methodology for the case where theremore »« less
Cited by 2
https://doi.org/10.1002/asmb.2359

Full Text Available
SU-F-BRD-01: A Logistic Regression Model to Predict Objective Function Weights in Prostate Cancer IMRT

Journal Article Boutilier, J ; Chan, T ; Lee, T ; ... - Medical Physics

Purpose: To develop a statistical model that predicts optimization objective function weights from patient geometry for intensity-modulation radiotherapy (IMRT) of prostate cancer. Methods: A previously developed inverse optimization method (IOM) is applied retrospectively to determine optimal weights for 51 treated patients. We use an overlap volume ratio (OVR) of bladder and rectum for different PTV expansions in order to quantify patient geometry in explanatory variables. Using the optimal weights as ground truth, we develop and train a logistic regression (LR) model to predict the rectum weight and thus the bladder weight. Post hoc, we fix the weights of the leftmore »« less
https://doi.org/10.1118/1.4889055
Minimally Informative Prior Distributions for PSA

Conference Kelly, Dana L ; Youngblood, Robert W ; Vedros, Kurt G

A salient feature of Bayesian inference is its ability to incorporate information from a variety of sources into the inference model, via the prior distribution (hereafter simply “the prior”). However, over-reliance on old information can lead to priors that dominate new data. Some analysts seek to avoid this by trying to work with a minimally informative prior distribution. Another reason for choosing a minimally informative prior is to avoid the often-voiced criticism of subjectivity in the choice of prior. Minimally informative priors fall into two broad classes: 1) so-called noninformative priors, which attempt to be completely objective, in that themore »« less
Full Text Available
Expert systems for implementing group consensus opinion in complex, weighted decisions

Conference Carr, K R

The development of an expert system for instrumentation and control operations is sometimes particularly difficult because (a) the expert system must implement the consensus opinion of a group of several human experts in all circumstances, (b) there may be 20 or more complex, dynamic, interrelated factors that influence the decision, each of which must be properly considered and weighed, and (c) the decision may involve choosing from several alternative courses of action, rather that a simple yes-no'' choice. These demanding requirements can discourage or, in some cases, prevent altogether the development of expert systems that are seriously needed and wouldmore »« less
Full Text Available
Bayesian Approach to Estimation of Water Table Elevations Using Historical Rasters as Prior Information 2019 - 20430

Conference Carson, John ; Jordan, Amy ; Rice, Amy ; ...

In cases of complex but only partially known geology and a lack of spatial control in observation well locations, water table elevation estimation is very challenging. In some cases, auxiliary information, such as observations of the movement of tracers, operation of injection and extraction wells, and calibration of groundwater models against the historical elevation data, can be combined with expert judgement to estimate flow directions in areas of sparse data and to aid in the production of more reliable contour maps (and associated rasters) than could be produced by relying on sparse well elevation data alone. Given a historical sequencemore »« less

Other research related to this record: