Non-conformity Scores for High-Quality Uncertainty Quantification from Conformal Prediction

Technical Report · Sat Aug 17 04:00:00 EDT 2024

DOI:https://doi.org/10.2172/2430248· OSTI ID:2430248

Adams, Jason Ross ^[1]; Berman, Brandon ^[1]; Deka, Rina ^[2]; Michalenko, Joshua James ^[1]

---

1. Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
2. North Carolina State University, Raleigh, NC (United States)

High-quality uncertainty quantification (UQ) is a critical component of enabling trust in deep learning (DL) models and is especially important if DL models are to be deployed in high-consequence applications. Conformal prediction (CP) methods represent an emerging nonparametric approach for producing UQ that is easily interpretable and, under weak assumptions, provides a guarantee regarding UQ quality. This report describes the research outputs of an Exploratory Express Laboratory Directed Research and Development (LDRD) project at Sandia National Laboratories. This project focused on how best to implement CP methods for DL models. This report introduces new methodology for obtaining high-quality UQ from DL models using CP methods, describes a novel system of assessing UQ quality, and provides experimental results that demonstrate the quality of the new methodology and utility of the UQ quality assessment system. Avenues for future research and discussion of potential impacts at Sandia and in the wider research community are also given.

• Research Organization: Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States)
• Sponsoring Organization: USDOE National Nuclear Security Administration (NNSA); USDOE Laboratory Directed Research and Development (LDRD) Program
• DOE Contract Number: NA0003525
• OSTI ID: 2430248
• Report Number(s): SAND--2023-10422R
• Country of Publication: United States
• Language: English

Related Subjects

97 MATHEMATICS AND COMPUTING