Evidential Deep Learning: Enhancing Predictive Uncertainty Estimation for Earth System Science Applications

Journal Article · 30 September 2024 · Artificial Intelligence for the Earth Systems

DOI: https://doi.org/10.1175/AIES-D-23-0093.1 · OSTI ID:2481631

Schreck, John S. ^[1]; Gagne, David John ^[1]; Becker, Charlie ^[1]; Chapman, William E. ^[1]; Elmore, Kim ^[2]; Fan, Da ^[3]; Gantos, Gabrielle ^[1]; Kim, Eliot ^[1]; Kimpara, Dhamma ^[4]; Martin, Thomas ^[5]; Molina, Maria J. ^[6]; Przybylo, Vanessa M. ^[7]; Radford, Jacob ^[8]; Saavedra, Belen ^[1]; Willson, Justin ^[1]; Wirz, Christopher ^[1]

---

1. NSF National Center for Atmospheric Research, Boulder, Colorado
2. Cooperative Institute for Severe and High-Impact Weather Research and Operations, National Severe Storms Laboratory, Norman, Oklahoma
3. Department of Meteorology and Atmospheric Science, The Pennsylvania State University, University Park, Pennsylvania
4. Department of Computer Science, University of Colorado Boulder, Boulder, Colorado
5. University Corporation for Atmospheric Research, Unidata, Boulder, Colorado
6. Department of Atmospheric and Oceanic Science, University of Maryland, College Park, College Park, Maryland, NSF National Center for Atmospheric Research, Boulder, Colorado
7. University at Albany, State University of New York, Albany, New York
8. Cooperative Institute for Research in the Atmosphere, Colorado State University, Fort Collins, Colorado

Abstract Robust quantification of predictive uncertainty is a critical addition needed for machine learning applied to weather and climate problems to improve the understanding of what is driving prediction sensitivity. Ensembles of machine learning models provide predictive uncertainty estimates in a conceptually simple way but require multiple models for training and prediction, increasing computational cost and latency. Parametric deep learning can estimate uncertainty with one model by predicting the parameters of a probability distribution but does not account for epistemic uncertainty. Evidential deep learning, a technique that extends parametric deep learning to higher-order distributions, can account for both aleatoric and epistemic uncertainties with one model. This study compares the uncertainty derived from evidential neural networks to that obtained from ensembles. Through applications of the classification of winter precipitation type and regression of surface-layer fluxes, we show evidential deep learning models attaining predictive accuracy rivaling standard methods while robustly quantifying both sources of uncertainty. We evaluate the uncertainty in terms of how well the predictions are calibrated and how well the uncertainty correlates with prediction error. Analyses of uncertainty in the context of the inputs reveal sensitivities to underlying meteorological processes, facilitating interpretation of the models. The conceptual simplicity, interpretability, and computational efficiency of evidential neural networks make them highly extensible, offering a promising approach for reliable and practical uncertainty quantification in Earth system science modeling. To encourage broader adoption of evidential deep learning, we have developed a new Python package, Machine Integration and Learning for Earth Systems (MILES) group Generalized Uncertainty for Earth System Science (GUESS) (MILES-GUESS) ( https://github.com/ai2es/miles-guess ), that enables users to train and evaluate both evidential and ensemble deep learning. Significance Statement This study demonstrates a new technique, evidential deep learning, for robust and computationally efficient uncertainty quantification in modeling the Earth system. The method integrates probabilistic principles into deep neural networks, enabling the estimation of both aleatoric uncertainty from noisy data and epistemic uncertainty from model limitations using a single model. Our analyses reveal how decomposing these uncertainties provides valuable insights into reliability, accuracy, and model shortcomings. We show that the approach can rival standard methods in classification and regression tasks within atmospheric science while offering practical advantages such as computational efficiency. With further advances, evidential networks have the potential to enhance risk assessment and decision-making across meteorology by improving uncertainty quantification, a longstanding challenge. This work establishes a strong foundation and motivation for the broader adoption of evidential learning, where properly quantifying uncertainties is critical yet lacking.

• Sponsoring Organization: USDOE
• Grant/Contract Number: SC0022070
• OSTI ID: 2481631
• Journal Information: Artificial Intelligence for the Earth Systems, Journal Name: Artificial Intelligence for the Earth Systems Journal Issue: 4 Vol. 3; ISSN 2769-7525
• Publisher: American Meteorological SocietyCopyright Statement
• Country of Publication: United States
• Language: English