Randomized physics-informed neural networks for Bayesian data assimilation

Zong, Yifei; Barajas-Solano, David A.; Tartakovsky, Alexandre M.

doi:10.1016/j.cma.2024.117670

Randomized physics-informed neural networks for Bayesian data assimilation

Journal Article · Wed Jan 08 00:00:00 EST 2025 · Computer Methods in Applied Mechanics and Engineering

DOI:https://doi.org/10.1016/j.cma.2024.117670· OSTI ID:2497874

^[1]; ^[2]; ^[3]

University of Illinois at Urbana-Champaign, IL (United States)
Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
University of Illinois at Urbana-Champaign, IL (United States); Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)

We propose a randomized physics-informed neural network (rPINN) method for uncertainty quantification in inverse partial differential equation problems. The rPINN method samples the distribution by solving a stochastic optimization problem obtained by randomizing the PINN loss function. The effectiveness of the rPINN method is tested for linear and nonlinear Poisson equations and the diffusion equation with a spatially heterogeneous diffusion coefficient. The rPINN method produces approximations to the posterior with good predictive capacity for all considered problems. We compare rPINN with the Hamiltonian Monte Carlo (HMC), a standard method for sampling the posterior distribution of PINN solutions. HMC and rPINN produce similar distributions for the linear Poisson equation, but rPINN is, on average, 27 times faster than HMC. For the nonlinear Poisson and diffusion equations, the HMC method fails to converge as HMC chains cannot fully explore the posterior distribution of PINN parameters in a reasonable amount of time. We also show that for the considered problems, rPINN outperforms other sampling methods, including the Stein variational gradient descent and deep ensemble methods.

View Journal Article

Research Organization:: Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)

Grant/Contract Number:: AC05-76RL01830

OSTI ID:: 2497874

Alternate ID(s):: OSTI ID: 2564680

Report Number(s):: PNNL-SA--203527

Journal Information:: Computer Methods in Applied Mechanics and Engineering, Journal Name: Computer Methods in Applied Mechanics and Engineering Vol. 436; ISSN 0045-7825

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

References (25)

Ensemble Randomized Maximum Likelihood Method as an Iterative Ensemble Smoother Chen, Yan; Oliver, Dean S. Mathematical Geosciences, Vol. 44, Issue 1 https://doi.org/10.1007/s11004-011-9376-z	journal	December 2011
Identification of groundwater contamination sources and hydraulic parameters based on bayesian regularization deep neural network Pan, Zidong; Lu, Wenxi; Fan, Yue Environmental Science and Pollution Research, Vol. 28, Issue 13 https://doi.org/10.1007/s11356-020-11614-1	journal	January 2021
Bayesian neural networks and density networks MacKay, David J. C. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, Vol. 354, Issue 1 https://doi.org/10.1016/0168-9002(94)00931-7	journal	January 1995
Bayesian approach for neural networks—review and case studies Lampinen, Jouko; Vehtari, Aki Neural Networks, Vol. 14, Issue 3 https://doi.org/10.1016/S0893-6080(00)00098-8	journal	April 2001
Bayesian Physics Informed Neural Networks for real-world nonlinear dynamical systems Linka, Kevin; Schäfer, Amelie; Meng, Xuhui Computer Methods in Applied Mechanics and Engineering, Vol. 402 https://doi.org/10.1016/j.cma.2022.115346	journal	December 2022
Improved training of physics-informed neural networks for parabolic differential equations with sharply perturbed initial conditions Zong, Yifei; He, QiZhi; Tartakovsky, Alexandre M. Computer Methods in Applied Mechanics and Engineering, Vol. 414 https://doi.org/10.1016/j.cma.2023.116125	journal	September 2023
Bayesian reduced-order deep learning surrogate model for dynamic systems described by partial differential equations Wang, Yuanzhe; Zong, Yifei; McCreight, James L. Computer Methods in Applied Mechanics and Engineering, Vol. 429 https://doi.org/10.1016/j.cma.2024.117147	journal	September 2024
Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations Raissi, M.; Perdikaris, P.; Karniadakis, G. E. Journal of Computational Physics, Vol. 378 https://doi.org/10.1016/j.jcp.2018.10.045	journal	February 2019
Physics-informed machine learning with conditional Karhunen-Loève expansions Tartakovsky, A. M.; Barajas-Solano, D. A.; He, Q. Journal of Computational Physics, Vol. 426 https://doi.org/10.1016/j.jcp.2020.109904	journal	February 2021
B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data Yang, Liu; Meng, Xuhui; Karniadakis, George Em Journal of Computational Physics, Vol. 425 https://doi.org/10.1016/j.jcp.2020.109913	journal	January 2021
When and why PINNs fail to train: A neural tangent kernel perspective Wang, Sifan; Yu, Xinling; Perdikaris, Paris Journal of Computational Physics, Vol. 449 https://doi.org/10.1016/j.jcp.2021.110768	journal	January 2022
Uncertainty quantification in scientific machine learning: Methods, metrics, and comparisons Psaros, Apostolos F.; Meng, Xuhui; Zou, Zongren Journal of Computational Physics, Vol. 477 https://doi.org/10.1016/j.jcp.2022.111902	journal	March 2023
Physics-constrained bayesian neural network for fluid flow reconstruction with sparse and noisy data Sun, Luning; Wang, Jian-Xun Theoretical and Applied Mechanics Letters, Vol. 10, Issue 3 https://doi.org/10.1016/j.taml.2020.01.031	journal	March 2020
Bayesian neural network for rainfall‐runoff modeling Khan, Mohammad Sajjad; Coulibaly, Paulin Water Resources Research, Vol. 42, Issue 7 https://doi.org/10.1029/2005WR003971	journal	July 2006
Estimating uncertainty of streamflow simulation using Bayesian neural networks Zhang, Xuesong; Liang, Faming; Srinivasan, Raghavan Water Resources Research, Vol. 45, Issue 2 https://doi.org/10.1029/2008WR007030	journal	February 2009
Physics‐Informed Deep Neural Networks for Learning Parameters and Constitutive Relationships in Subsurface Flow Problems Tartakovsky, A. M.; Marrero, C. Ortiz; Perdikaris, Paris Water Resources Research, Vol. 56, Issue 5 https://doi.org/10.1029/2019WR026731	journal	May 2020
Physics‐Informed Neural Network Method for Forward and Backward Advection‐Dispersion Equations He, QiZhi; Tartakovsky, Alexandre M. Water Resources Research, Vol. 57, Issue 7 https://doi.org/10.1029/2020WR029479	journal	July 2021
Variational Inference: A Review for Statisticians Blei, David M.; Kucukelbir, Alp; McAuliffe, Jon D. Journal of the American Statistical Association, Vol. 112, Issue 518 https://doi.org/10.1080/01621459.2017.1285773	journal	July 2016
Bayesian Physics-Informed Neural Networks for the Subsurface Tomography Based on the Eikonal Equation Gou, Rongxi; Zhang, Yijie; Zhu, Xueyu IEEE Transactions on Geoscience and Remote Sensing, Vol. 61 https://doi.org/10.1109/TGRS.2023.3286438	journal	January 2023
Data‐driven model for river flood forecasting based on a Bayesian network approach Boutkhamouine, Brahim; Roux, Hélène; Pérés, François Journal of Contingencies and Crisis Management, Vol. 28, Issue 3 https://doi.org/10.1111/1468-5973.12316	journal	September 2020
Randomize-Then-Optimize: A Method for Sampling from Posterior Distributions in Nonlinear Inverse Problems Bardsley, Johnathan M.; Solonen, Antti; Haario, Heikki SIAM Journal on Scientific Computing, Vol. 36, Issue 4 https://doi.org/10.1137/140964023	journal	January 2014
A Randomized Maximum A Posteriori Method for Posterior Sampling of High Dimensional Nonlinear Bayesian Inverse Problems Wang, Kainan; Bui-Thanh, Tan; Ghattas, Omar SIAM Journal on Scientific Computing, Vol. 40, Issue 1 https://doi.org/10.1137/16M1060625	journal	January 2018
Understanding and Mitigating Gradient Flow Pathologies in Physics-Informed Neural Networks Wang, Sifan; Teng, Yujun; Perdikaris, Paris SIAM Journal on Scientific Computing, Vol. 43, Issue 5 https://doi.org/10.1137/20M1318043	journal	January 2021
Integrating Scientific Knowledge with Machine Learning for Engineering and Environmental Systems Willard, Jared; Jia, Xiaowei; Xu, Shaoming ACM Computing Surveys, Vol. 55, Issue 4 https://doi.org/10.1145/3514228	journal	November 2022
Initializing Bayesian Hyperparameter Optimization via Meta-Learning Feurer, Matthias; Springenberg, Jost; Hutter, Frank Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 29, Issue 1 https://doi.org/10.1609/aaai.v29i1.9354	journal	February 2015

Similar Records

Randomized physics-informed machine learning for uncertainty quantification in high-dimensional inverse problems

Journal Article · Wed Sep 04 20:00:00 EDT 2024 · Journal of Computational Physics · OSTI ID:2440115

Related Subjects

42 ENGINEERING
Approximate Bayesian inference
physics-informed machine learning
Physics-informed neural networks
Bayesian physics-informed neural networks
Data assimilation
Inverse uncertainty quantification
Partial differential equations

Randomized physics-informed neural networks for Bayesian data assimilation

Citation Formats

References (25)

Similar Records

Related Subjects