Search Menu
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information
Search Scholarly Publications

Title: Designing accurate emulators for scientific processes using calibration-driven deep models

Journal Article · · Nature Communications
ORCiD logo [1];  [2]; ORCiD logo [1]; ORCiD logo [1];  [1]; ORCiD logo [1]; ORCiD logo [1]
  1. Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
  2. Arizona State Univ., Tempe, AZ (United States)
+ Show Author Affiliations

Predictive models that accurately emulate complex scientific processes can achieve speed-ups over numerical simulators or experiments and at the same time provide surrogates for improving the subsequent analysis. Consequently, there is a recent surge in utilizing modern machine learning methods to build data-driven emulators. In this work, we study an often overlooked, yet important, problem of choosing loss functions while designing such emulators. Popular choices such as the mean squared error or the mean absolute error are based on a symmetric noise assumption and can be unsuitable for heterogeneous data or asymmetric noise distributions. We propose Learn-by-Calibrating, a novel deep learning approach based on interval calibration for designing emulators that can effectively recover the inherent noise structure without any explicit priors. Using a large suite of use-cases, we demonstrate the efficacy of our approach in providing high-quality emulators, when compared to widely-adopted loss function choices, even in small-data regimes.

Research Organization:
Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States)
Sponsoring Organization:
USDOE National Nuclear Security Administration (NNSA)
Grant/Contract Number:
AC52-07NA27344
OSTI ID:
1706206
Alternate ID(s):
OSTI ID: 1738904
Report Number(s):
LLNL-JRNL-808697; 1013469
Journal Information:
Nature Communications, Vol. 11, Issue 1; ISSN 2041-1723
Publisher:
Nature Publishing GroupCopyright Statement
Country of Publication:
United States
Language:
English

References (36)

Uncertainties in Parameters Estimated with Neural Networks: Application to Strong Gravitational Lensing journal November 2017
The JAG inertial confinement fusion simulation dataset for multi-modal scientific deep learning. In Lawrence Livermore National Laboratory (LLNL) Open Data Initiative dataset January 2020
Quantile Regression for Analyzing Heterogeneity in Ultra-High Dimension journal March 2012
Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data journal October 2019
Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey journal January 2020
Bayesian deep convolutional encoder–decoder networks for surrogate modeling and uncertainty quantification journal August 2018
Learn-By-Calibrating: Using Calibration As A Training Objective conference May 2020
A data-driven statistical model for predicting the critical temperature of a superconductor journal November 2018
CaloGAN: Simulating 3D high energy particle showers in multilayer electromagnetic calorimeters with generative adversarial networks journal January 2018
Opportunities and obstacles for deep learning in biology and medicine journal April 2018
Methods for interpreting and understanding deep neural networks journal February 2018
Analysis of Strength of Concrete Using Design of Experiments and Neural Networks journal August 2006
Strictly Proper Scoring Rules, Prediction, and Estimation journal March 2007
Smoothed state estimates under abrupt changes using sum-of-norms regularization journal April 2012
PMLB: a large benchmark suite for machine learning evaluation and comparison journal December 2017
Orthogonal Matching Pursuit for Sparse Quantile Regression conference December 2014
Lawrence Livermore National Laboratory (LLNL) Open Data Initiative dataset January 2020
A second life of the Matthias’s rules journal June 2016
Recent Progress on Generative Adversarial Networks (GANs): A Survey journal January 2019
Neural networks for variational problems in engineering journal September 2008
Leveraging uncertainty information from deep neural networks for disease detection journal December 2017
Multimodal Machine Learning: A Survey and Taxonomy journal February 2019
Towards Concise Models of Grid Stability conference October 2018
Searching for exotic particles in high-energy physics with deep learning journal July 2014
A HYDRA UQ Workflow for NIF Ignition Experiments
  • Langer, Steven H.; Spears, Brian; Peterson, J. Luc
  • 2016 Second Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization (ISAV) https://doi.org/10.1109/ISAV.2016.006
conference November 2016
Learning for Single-Shot Confidence Calibration in Deep Neural Networks Through Stochastic Inferences conference June 2019
Accurate Telemonitoring of Parkinson's Disease Progression by Noninvasive Speech Tests journal April 2010
Understanding Deep Neural Networks through Input Uncertainties conference May 2019
A guide to deep learning in healthcare journal January 2019
A Procedure for Integrating Geologic Concepts Into History Matching conference April 2013
Machine learning for molecular and materials science journal July 2018
Building Calibrated Deep Models via Uncertainty Matching with Auxiliary Interval Predictors journal April 2020
Representation Learning: A Review and New Perspectives journal August 2013
Improved surrogates in inertial confinement fusion with manifold and cycle consistencies journal April 2020
Nanophotonic particle simulation and inverse design using artificial neural networks journal June 2018
A data-driven statistical model for predicting the critical temperature of a superconductor journal November 2018

Similar Records

Related Subjects

97 MATHEMATICS AND COMPUTING
Computer science
Scientific data