skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Bayesian Design of Experiments for Logistic Regression to Evaluate Multiple Forensic Algorithms

Abstract

When evaluating the performance of several forensic classification algorithms, it is desirable to construct a design that considers a variety of performance levels for each of the algorithms. We describe a strategy to use Bayesian design of experiments with multiple prior estimates to capture anticipated performance. Our goal is to characterize results from the different classification algorithms as a function of multiple explanatory variables and use this to choose a design about which units to test. Bayesian design of experiments has been successful for generalized linear models, including logistic regression models. Here, we develop methodology for the case where there are several potentially nonoverlapping priors for anticipated performance under consideration. The weighted priors method performs well for a broad range of true underlying model parameter choices and is more robust when compared to other candidate design choices. Additionally, we show how this can be applied in the multivariate input case and provide some useful summary measures. The shared information plot is used to evaluate design point allocation, and the D-value difference plot allows for the comparison of design performance across multiple potential parameter values in higher dimensions. Here, we illustrate the methods with several examples.

Authors:
 [1]; ORCiD logo [2]
  1. Pennsylvania State Univ., University Park, PA (United States). Dept. of Statistics
  2. Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
Publication Date:
Research Org.:
Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
Sponsoring Org.:
USDOE National Nuclear Security Administration (NNSA)
OSTI Identifier:
1481136
Report Number(s):
LA-UR-17-30911
Journal ID: ISSN 1524-1904
Grant/Contract Number:  
AC52-06NA25396
Resource Type:
Accepted Manuscript
Journal Name:
Applied Stochastic Models in Business and Industry
Additional Journal Information:
Journal Volume: 34; Journal Issue: 6; Journal ID: ISSN 1524-1904
Publisher:
Wiley
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; Mathematics; Bayesian D‐optimal designs; design of experiments; D‐value difference plot; forensic statistics; shared information plot

Citation Formats

Quinlan, Kevin R., and Anderson-Cook, Christine Michaela. Bayesian Design of Experiments for Logistic Regression to Evaluate Multiple Forensic Algorithms. United States: N. p., 2018. Web. doi:10.1002/asmb.2359.
Quinlan, Kevin R., & Anderson-Cook, Christine Michaela. Bayesian Design of Experiments for Logistic Regression to Evaluate Multiple Forensic Algorithms. United States. doi:10.1002/asmb.2359.
Quinlan, Kevin R., and Anderson-Cook, Christine Michaela. Tue . "Bayesian Design of Experiments for Logistic Regression to Evaluate Multiple Forensic Algorithms". United States. doi:10.1002/asmb.2359. https://www.osti.gov/servlets/purl/1481136.
@article{osti_1481136,
title = {Bayesian Design of Experiments for Logistic Regression to Evaluate Multiple Forensic Algorithms},
author = {Quinlan, Kevin R. and Anderson-Cook, Christine Michaela},
abstractNote = {When evaluating the performance of several forensic classification algorithms, it is desirable to construct a design that considers a variety of performance levels for each of the algorithms. We describe a strategy to use Bayesian design of experiments with multiple prior estimates to capture anticipated performance. Our goal is to characterize results from the different classification algorithms as a function of multiple explanatory variables and use this to choose a design about which units to test. Bayesian design of experiments has been successful for generalized linear models, including logistic regression models. Here, we develop methodology for the case where there are several potentially nonoverlapping priors for anticipated performance under consideration. The weighted priors method performs well for a broad range of true underlying model parameter choices and is more robust when compared to other candidate design choices. Additionally, we show how this can be applied in the multivariate input case and provide some useful summary measures. The shared information plot is used to evaluate design point allocation, and the D-value difference plot allows for the comparison of design performance across multiple potential parameter values in higher dimensions. Here, we illustrate the methods with several examples.},
doi = {10.1002/asmb.2359},
journal = {Applied Stochastic Models in Business and Industry},
number = 6,
volume = 34,
place = {United States},
year = {2018},
month = {7}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Citation Metrics:
Cited by: 1 work
Citation information provided by
Web of Science

Save / Share: