Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

Karhunen–Loève deep learning method for surrogate modeling and approximate Bayesian parameter estimation

Journal Article · · Advances in Water Resources
 [1];  [1];  [2];  [3];  [4];  [5]
  1. Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
  2. US Geological Survey, Reston, VA (United States); University Corporation for Atmospheric Research (UCAR), Boulder, CO (United States)
  3. University Corporation for Atmospheric Research (UCAR), Boulder, CO (United States)
  4. US Geological Survey, Madison, WI (United States)
  5. Pacific Northwest National Laboratory (PNNL), Richland, WA (United States); Univ. of Illinois at Urbana-Champaign, IL (United States)
+ Show Author Affiliations

We evaluate the performance of the Karhunen-Loève Deep Neural Network (KL-DNN) framework for surrogate modeling and approximate Bayesian parameter estimation in partial differential equation models. In the surrogate model, the Karhunen-Loève (KL) expansions are used for the dimensionality reduction of the number of unknown parameters and variables, and a deep neural network is employed to relate the reduced space of parameters to that of the state variables. The KL-DNN surrogate model is used to formulate a maximum-a-posteriori-like least-squares problem, which is randomized to draw samples of the posterior distribution of the parameters. We test the proposed framework for a hypothetical unconfined aquifer via comparison with the forward MODFLOW and inverse PEST++ iterative ensemble smoother (IES) solutions as well as the state-of-the-art Fourier neural operator (FNO) and deep operator networks (DeepONets) operator learning surrogate models. Our results show that the KL-DNN surrogate model outperforms FNO and DeepONet for forward predictions. For solving inverse problems, the randomized algorithm provides the same or more accurate Bayesian predictions of the parameters than IES as evidenced by the higher log-predictive probability of both the estimated parameter field and the forecast hydraulic head. The posterior mean obtained from the randomized algorithm is closer to the reference parameter field than that obtained with FNO as the maximum a posteriori estimate.

Research Organization:
Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
Sponsoring Organization:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR); USGS; National Science Foundation (NSF)
Grant/Contract Number:
AC05-76RL01830
OSTI ID:
2570716
Report Number(s):
PNNL-SA--212479
Journal Information:
Advances in Water Resources, Journal Name: Advances in Water Resources Vol. 203; ISSN 0309-1708
Publisher:
ElsevierCopyright Statement
Country of Publication:
United States
Language:
English

References (42)

A discretization and multigrid solver for a Darcy–Stokes system of three dimensional vuggy porous media journal December 2008
Levenberg–Marquardt forms of the iterative ensemble smoother for efficient history matching and uncertainty quantification journal May 2013
Ensemble Randomized Maximum Likelihood Method as an Iterative Ensemble Smoother journal December 2011
Efficient Uncertainty Quantification of Reservoir Properties for Parameter Estimation and Production Forecasting journal June 2019
On the relationships between SVD, KLT and PCA journal January 1981
Application of the pilot point method to the identification of aquifer transmissivities journal October 1991
Bayesian uncertainty quantification for flows in heterogeneous porous media using reversible jump Markov chain Monte Carlo methods journal March 2010
Physics-informed neural networks for multiphysics data assimilation with application to subsurface transport journal July 2020
Learning viscoelasticity models from indirect data using deep neural networks journal December 2021
A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data journal April 2022
SVD perspectives for augmenting DeepONet flexibility and interpretability journal January 2023
Improved training of physics-informed neural networks for parabolic differential equations with sharply perturbed initial conditions journal September 2023
Conditional Karhunen–Loève regression model with Basis Adaptation for high-dimensional problems: Uncertainty quantification and inverse modeling journal January 2024
A python framework for environmental model uncertainty analysis journal November 2016
A model-independent iterative ensemble smoother for efficient history-matching and uncertainty quantification in very high dimensions journal November 2018
A scalable model-independent iterative data assimilation tool for sequential and batch estimation of high dimensional model parameters and states journal April 2022
Multiscale finite-volume method for compressible multiphase flow in porous media journal August 2006
Stochastic spectral methods for efficient Bayesian solution of inverse problems journal June 2007
Dimensionality reduction and polynomial chaos acceleration of Bayesian inference in inverse problems journal April 2009
Generalized multiscale finite element methods (GMsFEM) journal October 2013
A deep-learning-based surrogate model for data assimilation in dynamic subsurface flow problems journal July 2020
Physics-informed machine learning with conditional Karhunen-Loève expansions journal February 2021
Simulation of strongly non-Gaussian processes using Karhunen–Loeve expansion journal April 2005
A hybrid regularized inversion methodology for highly parameterized environmental models: HYBRID REGULARIZATION METHODOLOGY journal October 2005
Calibration-constrained Monte Carlo analysis of highly parameterized models using subspace techniques: CALIBRATION-CONSTRAINED SSMC ANALYSIS journal January 2009
Physics‐Informed Deep Neural Networks for Learning Parameters and Constitutive Relationships in Subsurface Flow Problems journal May 2020
Physics‐Informed Machine Learning Method for Large‐Scale Data Assimilation Problems journal May 2022
Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators journal March 2021
Differentiable modelling to unify machine learning and physical models for geosciences journal July 2023
Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems journal July 1995
Revisiting “An Exercise in Groundwater Model Calibration and Prediction” After 30 Years: Insights and New Directions journal June 2019
Evaluating Lower Computational Burden Approaches for Calibration of Large Environmental Models journal June 2021
Risk‐Based Wellhead Protection Decision Support: A Repeatable Workflow Approach journal September 2021
An Exercise in Ground‐Water Model Calibration and Prediction journal May 1988
Dynamic Programming journal July 1966
Hybrid Multiscale Finite Volume Method for Advection-Diffusion Equations Subject to Heterogeneous Reactive Boundary Conditions journal January 2016
Accelerating Markov Chain Monte Carlo with Active Subspaces journal January 2016
A Randomized Maximum A Posteriori Method for Posterior Sampling of High Dimensional Nonlinear Bayesian Inverse Problems journal January 2018
Uncertainty Quantification of Deep Neural Network Predictions for Time-Dependent Responses with Functional PCA conference January 2023
Toward Reproducible Environmental Modeling for Decision Support: A Worked Example journal February 2020
Uncertainty estimation with deep learning for rainfall–runoff modeling journal March 2022
Model Reduction And Neural Networks For Parametric PDEs journal July 2021

Figures / Tables (15)

Fig. 1(p. 6)figure Fig. 1
Fig. 2(p. 7)figure Fig. 2
Table 1(p. 7)table Table 1
Table 2(p. 7)table Table 2
Fig. 3(p. 8)figure Fig. 3
Fig. 4(p. 9)figure Fig. 4
Fig. 5(p. 9)figure Fig. 5
Fig. 6(p. 10)figure Fig. 6
Table 3(p. 10)table Table 3
Fig. 7(p. 11)figure Fig. 7
Fig. 8(p. 11)figure Fig. 8
Fig. 10(p. 12)figure Fig. 10
Fig. 9(p. 12)figure Fig. 9
Fig. 11(p. 13)figure Fig. 11
Fig. 12(p. 13)figure Fig. 12

Similar Records

Bayesian reduced-order deep learning surrogate model for dynamic systems described by partial differential equations
Journal Article · Wed Jun 19 00:00:00 EDT 2024 · Computer Methods in Applied Mechanics and Engineering · OSTI ID:2574308
B-PINNs: Bayesian Physics-informal Neural Networks for Forward and Inverse PDE Problems with Noisy Data
Journal Article · Thu Jan 14 23:00:00 EST 2021 · Journal of Computational Physics · OSTI ID:1763962

Related Subjects

Approximate Bayesian inference
Conditioned Karhunen-Loève expansion
Parameter estimation
operator learning