Bayesian reduced-order deep learning surrogate model for dynamic systems described by partial differential equations

Wang, Yuanzhe; Zong, Yifei; McCreight, James L.; Hughes, Joseph D.; Tartakovsky, Alexandre M.

doi:10.1016/j.cma.2024.117147

Bayesian reduced-order deep learning surrogate model for dynamic systems described by partial differential equations

Journal Article · Wed Jun 19 00:00:00 EDT 2024 · Computer Methods in Applied Mechanics and Engineering

DOI:https://doi.org/10.1016/j.cma.2024.117147· OSTI ID:2574308

^[1]; ^[1]; ^[2]; ^[3]; ^[4]

Univ. of Illinois at Urbana-Champaign, IL (United States)
Univ. Corporation for Atmospheric Research, Boulder, CO (United States); US Geological Survey, Chicago, IL (United States)
Univ. Corporation for Atmospheric Research, Boulder, CO (United States)
Univ. of Illinois at Urbana-Champaign, IL (United States); Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)

We propose a reduced-order deep-learning surrogate model for dynamic systems described by time-dependent partial differential equations. This method employs space–time Karhunen–Loève expansions (KLEs) of the state variables and space-dependent KLEs of space-varying parameters to identify the reduced (latent) dimensions. Subsequently, a deep neural network (DNN) is used to map the parameter latent space to the state variable latent space. An approximate Bayesian method is developed for uncertainty quantification (UQ) in the proposed KL-DNN surrogate model. The KL-DNN method is tested for the linear advection–diffusion and nonlinear diffusion equations, and the Bayesian approach for UQ is compared with the deep ensembling (DE) approach, commonly used for quantifying uncertainty in DNN models. It was found that the approximate Bayesian method provides a more informative distribution of the PDE solutions in terms of the coverage of the reference PDE solutions (the percentage of nodes where the reference solution is within the confidence interval predicted by the UQ methods) and log predictive probability. The DE method is found to underestimate uncertainty and introduce bias. For the nonlinear diffusion equation, we compare the KL-DNN method with the Fourier Neural Operator (FNO) method and find that KL-DNN is 10% more accurate and needs less training time than the FNO method.

View Accepted Manuscript (DOE)

Research Organization:: Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)

Grant/Contract Number:: AC05-76RL01830

OSTI ID:: 2574308

Report Number(s):: PNNL-SA--214371

Journal Information:: Computer Methods in Applied Mechanics and Engineering, Journal Name: Computer Methods in Applied Mechanics and Engineering Vol. 429; ISSN 0045-7825

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

References (23)

Numerical Studies of Three-dimensional Stochastic Darcy’s Equation and Stochastic Advection-Diffusion-Dispersion Equation Lin, G.; Tartakovsky, A. M. Journal of Scientific Computing, Vol. 43, Issue 1 https://doi.org/10.1007/s10915-010-9346-5	journal	January 2010
Ensemble Randomized Maximum Likelihood Method as an Iterative Ensemble Smoother Chen, Yan; Oliver, Dean S. Mathematical Geosciences, Vol. 44, Issue 1 https://doi.org/10.1007/s11004-011-9376-z	journal	December 2011
Efficient Uncertainty Quantification of Reservoir Properties for Parameter Estimation and Production Forecasting McKenna, Sean A.; Akhriev, Albert; Echeverría Ciaurri, David Mathematical Geosciences, Vol. 52, Issue 2 https://doi.org/10.1007/s11004-019-09810-y	journal	June 2019
Modeling uncertainty in flow simulations via generalized polynomial chaos Xiu, Dongbin; Karniadakis, George Em Journal of Computational Physics, Vol. 187, Issue 1 https://doi.org/10.1016/S0021-9991(03)00092-5	journal	May 2003
Unsaturated flow in heterogeneous soils with spatially distributed uncertain hydraulic parameters Tartakovsky, Daniel M.; Lu, Zhiming; Guadagnini, Alberto Journal of Hydrology, Vol. 275, Issue 3-4 https://doi.org/10.1016/S0022-1694(03)00042-8	journal	May 2003
Simulation of multi-dimensional random fields by Karhunen–Loève expansion Zheng, Zhibao; Dai, Hongzhe Computer Methods in Applied Mechanics and Engineering, Vol. 324 https://doi.org/10.1016/j.cma.2017.05.022	journal	September 2017
Projection-based reduced order modeling and data-driven artificial viscosity closures for incompressible fluid flows Prakash, Aviral; Zhang, Yongjie Jessica Computer Methods in Applied Mechanics and Engineering, Vol. 425 https://doi.org/10.1016/j.cma.2024.116930	journal	May 2024
A deep-learning-based surrogate model for data assimilation in dynamic subsurface flow problems Tang, Meng; Liu, Yimin; Durlofsky, Louis J. Journal of Computational Physics, Vol. 413 https://doi.org/10.1016/j.jcp.2020.109456	journal	July 2020
Uncertainty quantification in scientific machine learning: Methods, metrics, and comparisons Psaros, Apostolos F.; Meng, Xuhui; Zou, Zongren Journal of Computational Physics, Vol. 477 https://doi.org/10.1016/j.jcp.2022.111902	journal	March 2023
Models for space–time random functions Grigoriu, Mircea Probabilistic Engineering Mechanics, Vol. 43 https://doi.org/10.1016/j.probengmech.2015.11.004	journal	January 2016
Transient flow in bounded randomly heterogeneous domains: 1. Exact conditional moment equations and recursive approximations Tartakovsky, Daniel M.; Neuman, Shlomo P. Water Resources Research, Vol. 34, Issue 1 https://doi.org/10.1029/97WR02118	journal	January 1998
Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators Lu, Lu; Jin, Pengzhan; Pang, Guofei Nature Machine Intelligence, Vol. 3, Issue 3 https://doi.org/10.1038/s42256-021-00302-5	journal	March 2021
pyABC: distributed, likelihood-free inference Klinger, Emmanuel; Rickert, Dennis; Hasenauer, Jan Bioinformatics, Vol. 34, Issue 20 https://doi.org/10.1093/bioinformatics/bty361	journal	May 2018
Revisiting “An Exercise in Groundwater Model Calibration and Prediction” After 30 Years: Insights and New Directions Hunt, Randall J.; Fienen, Michael N.; White, Jeremy T. Groundwater, Vol. 58, Issue 2 https://doi.org/10.1111/gwat.12907	journal	June 2019
MODFLOW as a Configurable Multi‐Model Hydrologic Simulator Langevin, Christian D.; Hughes, Joseph D.; Provost, Alden M. Groundwater, Vol. 62, Issue 1 https://doi.org/10.1111/gwat.13351	journal	September 2023
An Exercise in Ground‐Water Model Calibration and Prediction Freyberg, David L. Groundwater, Vol. 26, Issue 3 https://doi.org/10.1111/j.1745-6584.1988.tb00399.x	journal	May 1988
Randomize-Then-Optimize: A Method for Sampling from Posterior Distributions in Nonlinear Inverse Problems Bardsley, Johnathan M.; Solonen, Antti; Haario, Heikki SIAM Journal on Scientific Computing, Vol. 36, Issue 4 https://doi.org/10.1137/140964023	journal	January 2014
A Randomized Maximum A Posteriori Method for Posterior Sampling of High Dimensional Nonlinear Bayesian Inverse Problems Wang, Kainan; Bui-Thanh, Tan; Ghattas, Omar SIAM Journal on Scientific Computing, Vol. 40, Issue 1 https://doi.org/10.1137/16M1060625	journal	January 2018
Transport Map Accelerated Markov Chain Monte Carlo Parno, Matthew D.; Marzouk, Youssef M. SIAM/ASA Journal on Uncertainty Quantification, Vol. 6, Issue 2 https://doi.org/10.1137/17M1134640	journal	January 2018
A New Look at Proper Orthogonal Decomposition Rathinam, Muruhan; Petzold, Linda R. SIAM Journal on Numerical Analysis, Vol. 41, Issue 5 https://doi.org/10.1137/S0036142901389049	journal	January 2003
Hamiltonian Monte Carlo in Inverse Problems. Ill-Conditioning and Multimodality Langmore, Ian; Dikovsky, M.; Geraedts, S. International Journal for Uncertainty Quantification, Vol. 13, Issue 1 https://doi.org/10.1615/Int.J.UncertaintyQuantification.2022038478	journal	January 2023
Toward Reproducible Environmental Modeling for Decision Support: A Worked Example White, Jeremy T.; Foster, Linzy K.; Fienen, Michael N. Frontiers in Earth Science, Vol. 8 https://doi.org/10.3389/feart.2020.00050	journal	February 2020
Model Reduction And Neural Networks For Parametric PDEs Bhattacharya, Kaushik; Hosseini, Bamdad; Kovachki, Nikola B. The SMAI journal of computational mathematics, Vol. 7 https://doi.org/10.5802/smai-jcm.74	journal	July 2021

Similar Records

Karhunen–Loève deep learning method for surrogate modeling and approximate Bayesian parameter estimation

Journal Article · Mon Jun 16 00:00:00 EDT 2025 · Advances in Water Resources · OSTI ID:2570716

B-PINNs: Bayesian Physics-informal Neural Networks for Forward and Inverse PDE Problems with Noisy Data

Journal Article · Thu Jan 14 23:00:00 EST 2021 · Journal of Computational Physics · OSTI ID:1763962

B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data

Journal Article · Thu Oct 15 00:00:00 EDT 2020 · Journal of Computational Physics · OSTI ID:2282008

Related Subjects

Bayesian uncertainty quantification
Dimensional reduction
Dynamic systems
Machine learning
Surrogate models

Bayesian reduced-order deep learning surrogate model for dynamic systems described by partial differential equations

Citation Formats

References (23)

Similar Records

Related Subjects