Bayesian Design of Experiments for Logistic Regression to Evaluate Multiple Forensic Algorithms
When evaluating the performance of several forensic classification algorithms, it is desirable to construct a design that considers a variety of performance levels for each of the algorithms. We describe a strategy to use Bayesian design of experiments with multiple prior estimates to capture anticipated performance. Our goal is to characterize results from the different classification algorithms as a function of multiple explanatory variables and use this to choose a design about which units to test. Bayesian design of experiments has been successful for generalized linear models, including logistic regression models. Here, we develop methodology for the case where there are several potentially nonoverlapping priors for anticipated performance under consideration. The weighted priors method performs well for a broad range of true underlying model parameter choices and is more robust when compared to other candidate design choices. Additionally, we show how this can be applied in the multivariate input case and provide some useful summary measures. The shared information plot is used to evaluate design point allocation, and the D-value difference plot allows for the comparison of design performance across multiple potential parameter values in higher dimensions. Here, we illustrate the methods with several examples.
- Research Organization:
- Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
- Sponsoring Organization:
- USDOE National Nuclear Security Administration (NNSA)
- Grant/Contract Number:
- AC52-06NA25396
- OSTI ID:
- 1481136
- Report Number(s):
- LA-UR-17-30911
- Journal Information:
- Applied Stochastic Models in Business and Industry, Vol. 34, Issue 6; ISSN 1524-1904
- Publisher:
- WileyCopyright Statement
- Country of Publication:
- United States
- Language:
- English
Web of Science
How to Host An Effective Data Competition: Statistical Advice for Competition Design and Analysis
|
journal | February 2019 |
Similar Records
Bayesian projection pursuit regression
Nuclear Forensics for Irradiated Materials using Kernel Regression Models