B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data

Yang, Liu; Meng, Xuhui; Karniadakis, George Em

doi:10.1016/j.jcp.2020.109913

B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data

Journal Article · Thu Oct 15 00:00:00 UTC 2020 · Journal of Computational Physics

DOI:https://doi.org/10.1016/j.jcp.2020.109913· OSTI ID:2282008

Yang, Liu ^[1]; Meng, Xuhui ^[2]; Karniadakis, George Em ^[3]

Brown Univ., Providence, RI (United States); Brown University
Brown Univ., Providence, RI (United States)
Brown Univ., Providence, RI (United States); Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)

Here we propose a Bayesian physics-informed neural network (B-PINN) to solve both forward and inverse nonlinear problems described by partial differential equations (PDEs) and noisy data. In this Bayesian framework, the Bayesian neural network (BNN) combined with a PINN for PDEs serves as the prior while the Hamiltonian Monte Carlo (HMC) or the variational inference (VI) could serve as an estimator of the posterior. B-PINNs make use of both physical laws and scattered noisy measurements to provide predictions and quantify the aleatoric uncertainty arising from the noisy data in the Bayesian framework. Compared with PINNs, in addition to uncertainty quantification, B-PINNs obtain more accurate predictions in scenarios with large noise due to their capability of avoiding overfitting. We conduct a systematic comparison between the two different approaches for the B-PINNs posterior estimation (i.e., HMC or VI), along with dropout used for quantifying uncertainty in deep neural networks. Our experiments show that HMC is more suitable than VI with mean field Gaussian approximation for the B-PINNs posterior estimation, while dropout employed in PINNs can hardly provide accurate predictions with reasonable uncertainty. Finally, we replace the BNN in the prior with a truncated Karhunen-Loève (KL) expansion combined with HMC or a deep normalizing flow (DNF) model as posterior estimators. The KL is as accurate as BNN and much faster but this framework cannot be easily extended to high-dimensional problems unlike the BNN based framework.

Research Organization:: Brown Univ., Providence, RI (United States)

Sponsoring Organization:: National Institutes of Health (NIH); US Air Force Office of Scientific Research (AFOSR); US Army Research Office (ARO); USDOE

Grant/Contract Number:: SC0019453

OSTI ID:: 2282008

Alternate ID(s):: OSTI ID: 1763962
OSTI ID: 1775918
OSTI ID: 23203555

Journal Information:: Journal of Computational Physics, Journal Name: Journal of Computational Physics Vol. 425; ISSN 0021-9991

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

References (14)

Weight Uncertainty in Neural Networks Blundell, Charles; Cornebise, Julien; Kavukcuoglu, Koray arXiv https://doi.org/10.48550/arxiv.1505.05424	preprint	January 2015
Inferring solutions of differential equations using noisy multi-fidelity data Raissi, Maziar; Perdikaris, Paris; Karniadakis, George Em Journal of Computational Physics, Vol. 335 https://doi.org/10.1016/j.jcp.2017.01.060	journal	April 2017
Machine learning of linear differential equations using Gaussian processes Raissi, Maziar; Perdikaris, Paris; Karniadakis, George Em Journal of Computational Physics, Vol. 348 https://doi.org/10.1016/j.jcp.2017.07.050	journal	November 2017
Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations Raissi, M.; Perdikaris, P.; Karniadakis, G. E. Journal of Computational Physics, Vol. 378 https://doi.org/10.1016/j.jcp.2018.10.045	journal	February 2019
Adaptive multi-fidelity polynomial chaos approach to Bayesian inference in inverse problems Yan, Liang; Zhou, Tao Journal of Computational Physics, Vol. 381 https://doi.org/10.1016/j.jcp.2018.12.025	journal	March 2019
Data-driven discovery of PDEs in complex datasets Berg, Jens; Nyström, Kaj Journal of Computational Physics, Vol. 384 https://doi.org/10.1016/j.jcp.2019.01.036	journal	May 2019
Neural-net-induced Gaussian process regression for function approximation and PDE solution Pang, Guofei; Yang, Liu; Karniadakis, George Em Journal of Computational Physics, Vol. 384 https://doi.org/10.1016/j.jcp.2019.01.045	journal	May 2019
Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems Zhang, Dongkun; Lu, Lu; Guo, Ling Journal of Computational Physics, Vol. 397 https://doi.org/10.1016/j.jcp.2019.07.048	journal	November 2019
A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems Meng, Xuhui; Karniadakis, George Em Journal of Computational Physics, Vol. 401 https://doi.org/10.1016/j.jcp.2019.109020	journal	January 2020
Bayesian deep learning with hierarchical prior: Predictions from limited and noisy data Luo, Xihaier; Kareem, Ahsan Structural Safety, Vol. 84 https://doi.org/10.1016/j.strusafe.2019.101918	journal	May 2020
Deep learning LeCun, Yann; Bengio, Yoshua; Hinton, Geoffrey Nature, Vol. 521, Issue 7553 https://doi.org/10.1038/nature14539	journal	May 2015
Data-driven discovery of partial differential equations Rudy, Samuel H.; Brunton, Steven L.; Proctor, Joshua L. Science Advances, Vol. 3, Issue 4 https://doi.org/10.1126/sciadv.1602614	journal	April 2017
Adaptive Construction of Surrogates for the Bayesian Solution of Inverse Problems Li, Jinglai; Marzouk, Youssef M. SIAM Journal on Scientific Computing, Vol. 36, Issue 3 https://doi.org/10.1137/130938189	journal	January 2014
Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations Yang, Liu; Zhang, Dongkun; Karniadakis, George Em SIAM Journal on Scientific Computing, Vol. 42, Issue 1 https://doi.org/10.1137/18M1225409	journal	January 2020

Cited By (4)

A Machine-Learning Method for Time-Dependent Wave Equations over Unbounded Domains Xie, Changjian; Chen, Jingrun; Li, Xiantao arXiv https://doi.org/10.48550/arxiv.2101.05807	preprint	January 2021
Novel Deep neural networks for solving Bayesian statistical inverse Antil, Harbir; Elman, Howard C.; Onwunta, Akwum arXiv https://doi.org/10.48550/arxiv.2102.03974	preprint	January 2021
Physics-aware deep neural networks for surrogate modeling of turbulent natural convection Lucor, Didier; Agrawal, Atul; Sergent, Anne arXiv https://doi.org/10.48550/arxiv.2103.03565	preprint	January 2021
An extended physics informed neural network for preliminary analysis of parametric optimal control problems Demo, Nicola; Strazzullo, Maria; Rozza, Gianluigi arXiv https://doi.org/10.48550/arxiv.2110.13530	preprint	January 2021

Similar Records

B-PINNs: Bayesian Physics-informal Neural Networks for Forward and Inverse PDE Problems with Noisy Data

Journal Article · Fri Jan 15 04:00:00 UTC 2021 · Journal of Computational Physics · OSTI ID:1763962

Multi-fidelity Bayesian neural networks: Algorithms and applications

Journal Article · Fri Apr 16 00:00:00 UTC 2021 · Journal of Computational Physics · OSTI ID:2281996

Learning functional priors and posteriors from data and physics

Journal Article · Thu Feb 17 00:00:00 UTC 2022 · Journal of Computational Physics · OSTI ID:1897198

Related Subjects

97 MATHEMATICS AND COMPUTING
Bayesian physics-informed neural networks
Hamiltonian Monte Carlo
noisy data
nonlinear PDEs
variational inference

B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data

Citation Formats

References (14)

Cited By (4)

Similar Records

Related Subjects