A Decision Theoretic Approach To Optimizing Machine Learning Decisions with Prediction Uncertainty
- Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
- Sandia National Lab. (SNL-CA), Livermore, CA (United States)
While the use of machine learning (ML) classifiers is widespread, their output is often not part of any follow-on decision-making process. To illustrate, consider the scenario where we have developed and trained an ML classifier to find malicious URL links. In this scenario, network administrators must decide whether to allow a computer user to visit a particular website, or to instead block access because the site is deemed malicious. It would be very beneficial if decisions such as these could be made automatically using a trained ML classifier. Unfortunately, due to a variety of reasons discussed herein, the output from these classifiers can be uncertain, rendering downstream decisions difficult. Herein, we provide a framework for: (1) quantifying and propagating uncertainty in ML classifiers; (2) formally linking ML outputs with the decision-making process; and (3) making optimal decisions for classification under uncertainty with single or multiple objectives.
- Research Organization:
- Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States); Sandia National Laboratories (SNL-CA), Livermore, CA (United States)
- Sponsoring Organization:
- USDOE National Nuclear Security Administration (NNSA)
- DOE Contract Number:
- NA0003525
- OSTI ID:
- 1899419
- Report Number(s):
- SAND2022-16299; 712184
- Country of Publication:
- United States
- Language:
- English
Similar Records
Decision Science for Machine Learning (DeSciML)
Incorporating uncertainty into electric utility projections and decisions