DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Computer Model Calibration with Time Series Data Using Deep Learning and Quantile Regression

Journal Article · · SIAM/ASA Journal on Uncertainty Quantification
DOI: https://doi.org/10.1137/20m1382581 · OSTI ID:1960307
 [1]; ORCiD logo [1];  [2];  [3]
  1. Univ. of Cincinnati, OH (United States)
  2. Miami Univ., Oxford, OH (United States)
  3. Argonne National Lab. (ANL), Lemont, IL (United States)

Computer models play a key role in many scientific and engineering problems. One major source of uncertainty in computer model experiments is input parameter uncertainty. Computer model calibration is a formal statistical procedure to infer input parameters by combining information from model runs and observational data. The existing standard calibration framework suffers from inferential issues when the model output and observational data are high-dimensional dependent data, such as large time series, due to the difficulty in building an emulator and the nonidentifiability between effects from input parameters and data-model discrepancy. To overcome these challenges, we, in this work, propose a new calibration framework based on a deep neural network (DNN) with long short-term memory layers that directly emulates the inverse relationship between the model output and input parameters. Adopting the “learning with noise” idea, we train our DNN model to filter out the effects from data-model discrepancy on input parameter inference. We also formulate a new way to construct interval predictions for DNN using quantile regression to quantify the uncertainty in input parameter estimates. Through a simulation study and real data application with the Weather Research and Forecasting Model Hydrological modeling system (WRF-Hydro), we show our approach can yield accurate point estimates and well-calibrated interval estimates for input parameters.

Research Organization:
Argonne National Laboratory (ANL), Argonne, IL (United States)
Sponsoring Organization:
USDOE Office of Science (SC)
Grant/Contract Number:
AC02-06CH11357
OSTI ID:
1960307
Journal Information:
SIAM/ASA Journal on Uncertainty Quantification, Journal Name: SIAM/ASA Journal on Uncertainty Quantification Journal Issue: 1 Vol. 10; ISSN 2166-2525
Publisher:
Society for Industrial and Applied Mathematics (SIAM)Copyright Statement
Country of Publication:
United States
Language:
English

References (28)

Deep echo state networks with uncertainty quantification for spatio‐temporal forecasting journal November 2018
Why and when can deep-but not shallow-networks avoid the curse of dimensionality: A review journal March 2017
Comparison of Deep Neural Networks and Deep Hierarchical Models for Spatio-Temporal Data journal March 2019
Framewise phoneme classification with bidirectional LSTM and other neural network architectures journal July 2005
Random Forest Prediction Intervals journal June 2019
Large Sample Properties of Simulations Using Latin Hypercube Sampling journal May 1987
Some Asymptotic Results for Learning in Single Hidden-Layer Feedforward Network Models journal December 1989
Calibrating an Ice Sheet Model Using High-Dimensional Binary Spatial Data journal January 2016
Uncertainty Quantification for Computer Models With Spatial Output Using Calibration-Optimal Bases journal September 2018
A Generalized Gaussian Process Model for Computer Experiments With Binary Time Series journal May 2019
Learning about physical parameters: the importance of model discrepancy journal October 2014
Using additive noise in back-propagation training journal January 1992
LSTM recurrent networks learn simple context-free and context-sensitive languages journal January 2001
Bayesian calibration of computer models journal August 2001
A frequentist approach to computer model calibration
  • Wong, Raymond K. W.; Storlie, Curtis B.; Lee, Thomas C. M.
  • Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 79, Issue 2 https://doi.org/10.1111/rssb.12182
journal May 2016
Learning to Forget: Continual Prediction with LSTM journal October 2000
Training with Noise is Equivalent to Tikhonov Regularization journal January 1995
The Effects of Adding Noise During Backpropagation Training on a Generalization Performance journal April 1996
Computer Model Calibration Using High-Dimensional Output journal June 2008
Computer model validation with functional output journal October 2007
Fast dimension-reduced climate model calibration and the effect of data aggregation journal June 2014
Efficient calibration for imperfect computer models journal December 2015
Parallel partial Gaussian process emulation for computer models with massive output journal September 2016
Deep Learning: A Bayesian Perspective journal December 2017
Design and Analysis of Computer Experiments journal November 1989
A parallel workflow implementation for PEST version 13.6 in high-performance computing for WRF-Hydro version 5.0: a case study over the midwestern United States journal January 2019
A Generalized Gaussian Process Model for Computer Experiments With Binary Time Series dataset January 2023
Calibrating an Ice Sheet Model Using High-Dimensional Binary Spatial Data collection January 2016