Search Menu
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information
Search Scholarly Publications

Title: Deep neural network uncertainty quantification for LArTPC reconstruction

Journal Article · · Journal of Instrumentation
 [1];  [2];  [2]
  1. Stanford University, CA (United States)
  2. SLAC National Accelerator Laboratory (SLAC), Menlo Park, CA (United States)
+ Show Author Affiliations

We evaluate uncertainty quantification (UQ) methods for deep learning applied to liquid argon time projection chamber (LArTPC) physics analysis tasks. As deep learning applications enter widespread usage among physics data analysis, neural networks with reliable estimates of prediction uncertainty and robust performance against overconfidence and out-of-distribution (OOD) samples are critical for their full deployment in analyzing experimental data. While numerous UQ methods have been tested on simple datasets, performance evaluations for more complex tasks and datasets are scarce. Here we assess the application of selected deep learning UQ methods on the task of particle classification using the PiLArNet monte carlo 3D LArTPC point cloud dataset. We observe that UQ methods not only allow for better rejection of prediction mistakes and OOD detection, but also generally achieve higher overall accuracy across different task settings. We assess the precision of uncertainty quantification using different evaluation metrics, such as distributional separation of prediction entropy across correctly and incorrectly identified samples, receiver operating characteristic curves (ROCs), and expected calibration error from observed empirical accuracy. We conclude that ensembling methods can obtain well calibrated classification probabilities and generally perform better than other existing methods in deep learning UQ literature.

Research Organization:
SLAC National Accelerator Laboratory (SLAC), Menlo Park, CA (United States)
Sponsoring Organization:
USDOE Office of Science (SC), High Energy Physics (HEP); Zoox, Inc
Grant/Contract Number:
AC02-76SF00515
OSTI ID:
2326991
Journal Information:
Journal of Instrumentation, Vol. 18, Issue 12; ISSN 1748-0221
Publisher:
Institute of Physics (IOP)Copyright Statement
Country of Publication:
United States
Language:
English

References (14)

Semantic segmentation with a sparse convolutional neural network for event reconstruction in MicroBooNE journal March 2021
Strictly Proper Scoring Rules, Prediction, and Estimation journal March 2007
Transforming classifier scores into accurate multiclass probability estimates conference January 2002
Scalable deep convolutional neural networks for sparse, locally dense liquid argon time projection chamber data journal July 2020
On Wasserstein Two-Sample Testing and Related Families of Nonparametric Tests journal January 2017
Underground operation of the ICARUS T600 LAr-TPC: first results journal July 2011
Deep Residual Learning for Image Recognition conference June 2016
A review of uncertainty quantification in deep learning: Techniques, applications and challenges journal December 2021
A review of semantic segmentation using deep neural networks journal November 2017
Geant4—a simulation toolkit
  • Agostinelli, S.; Allison, J.; Amako, K.
  • Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, Vol. 506, Issue 3 https://doi.org/10.1016/S0168-9002(03)01368-8
journal July 2003
Uncertainty quantification for deep learning in particle accelerator applications journal November 2021
A Survey on Bayesian Deep Learning journal September 2020
Uncertainty quantification in scientific machine learning: Methods, metrics, and comparisons journal March 2023
An introduction to ROC analysis journal June 2006

Similar Records

Related Subjects

47 OTHER INSTRUMENTATION