Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems

Yu, Jeremy; Lu, Lu; Meng, Xuhui; Karniadakis, George Em

doi:10.1016/j.cma.2022.114823

Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems

Journal Article · Fri Mar 18 00:00:00 EDT 2022 · Computer Methods in Applied Mechanics and Engineering

DOI:https://doi.org/10.1016/j.cma.2022.114823· OSTI ID:1976976

Yu, Jeremy ^[1]; ^[2]; Meng, Xuhui ^[3]; Karniadakis, George Em ^[3]

St. Mark's School of Texas, Dallas, TX (United States); OSTI
University of Pennsylvania, Philadelphia, PA (United States)
Brown University, Providence, RI (United States)

Deep learning has been shown to be an effective tool in solving partial differential equations (PDEs) through physics-informed neural networks (PINNs). PINNs embed the PDE residual into the loss function of the neural network, and have been successfully employed to solve diverse forward and inverse PDE problems. However, one disadvantage of the first generation of PINNs is that they usually have limited accuracy even with many training points. Here, we propose a new method, gradient-enhanced physics-informed neural networks (gPINNs), for improving the accuracy of PINNs. gPINNs leverage gradient information of the PDE residual and embed the gradient into the loss function. We tested gPINNs extensively and demonstrated the effectiveness of gPINNs in both forward and inverse PDE problems. Our numerical results show that gPINN performs better than PINN with fewer training points. Additionally, we combined gPINN with the method of residual-based adaptive refinement (RAR), a method for improving the distribution of training points adaptively during training, to further improve the performance of gPINN, especially in PDEs with solutions that have steep gradients.

View Accepted Manuscript (DOE)

Research Organization:: Brown University, Providence, RI (United States)

Sponsoring Organization:: USDOE Office of Science (SC); Air Force Office of Scientific Research (AFOSR)

Grant/Contract Number:: SC0019453

OSTI ID:: 1976976

Alternate ID(s):: OSTI ID: 2324706

Journal Information:: Computer Methods in Applied Mechanics and Engineering, Journal Name: Computer Methods in Applied Mechanics and Engineering Journal Issue: C Vol. 393; ISSN 0045-7825

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

References (24)

Neural-network-based approximations for solving partial differential equations Dissanayake, M. W. M. G.; Phan-Thien, N. Communications in Numerical Methods in Engineering, Vol. 10, Issue 3 https://doi.org/10.1002/cnm.1640100303	journal	March 1994
An Overview of Gradient-Enhanced Metamodels with Applications Laurent, Luc; Le Riche, Rodolphe; Soulier, Bruno Archives of Computational Methods in Engineering, Vol. 26, Issue 1 https://doi.org/10.1007/s11831-017-9226-3	journal	July 2017
PPINN: Parareal physics-informed neural network for time-dependent PDEs Meng, Xuhui; Li, Zhen; Zhang, Dongkun Computer Methods in Applied Mechanics and Engineering, Vol. 370 https://doi.org/10.1016/j.cma.2020.113250	journal	October 2020
Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations Raissi, M.; Perdikaris, P.; Karniadakis, G. E. Journal of Computational Physics, Vol. 378 https://doi.org/10.1016/j.jcp.2018.10.045	journal	February 2019
Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems Zhang, Dongkun; Lu, Lu; Guo, Ling Journal of Computational Physics, Vol. 397 https://doi.org/10.1016/j.jcp.2019.07.048	journal	November 2019
Solving initial-boundary value problems for systems of partial differential equations using neural networks and optimization techniques Shekari Beidokhti, R.; Malek, A. Journal of the Franklin Institute, Vol. 346, Issue 9 https://doi.org/10.1016/j.jfranklin.2009.05.003	journal	November 2009
Physics Informed Extreme Learning Machine (PIELM)–A rapid method for the numerical solution of partial differential equations Dwivedi, Vikas; Srinivasan, Balaji Neurocomputing, Vol. 391 https://doi.org/10.1016/j.neucom.2019.12.099	journal	May 2020
Physics-informed machine learning Karniadakis, George Em; Kevrekidis, Ioannis G.; Lu, Lu Nature Reviews Physics, Vol. 3, Issue 6 https://doi.org/10.1038/s42254-021-00314-5	journal	May 2021
Improving generalization performance using double backpropagation Drucker, H.; Le Cun, Y. IEEE Transactions on Neural Networks, Vol. 3, Issue 6 https://doi.org/10.1109/72.165600	journal	January 1992
Artificial neural networks for solving ordinary and partial differential equations Lagaris, I. E.; Likas, A.; Fotiadis, D. I. IEEE Transactions on Neural Networks, Vol. 9, Issue 5 https://doi.org/10.1109/72.712178	journal	January 1998
Artificial Neural Network Method for Solution of Boundary Value Problems With Exact Satisfaction of Arbitrary Boundary Conditions McFall, K. S.; Mahan, J. R. IEEE Transactions on Neural Networks, Vol. 20, Issue 8 https://doi.org/10.1109/TNN.2009.2020735	journal	August 2009
Efficient training of physics‐informed neural networks via importance sampling Nabian, Mohammad Amin; Gladstone, Rini Jasmine; Meidani, Hadi Computer-Aided Civil and Infrastructure Engineering, Vol. 36, Issue 8 https://doi.org/10.1111/mice.12685	journal	April 2021
Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations Raissi, Maziar; Yazdani, Alireza; Karniadakis, George Em Science, Vol. 367, Issue 6481 https://doi.org/10.1126/science.aaw4741	journal	January 2020
fPINNs: Fractional Physics-Informed Neural Networks Pang, Guofei; Lu, Lu; Karniadakis, George Em SIAM Journal on Scientific Computing, Vol. 41, Issue 4 https://doi.org/10.1137/18M1229845	journal	January 2019
Learning in Modal Space: Solving Time-Dependent Stochastic PDEs Using Physics-Informed Neural Networks Zhang, Dongkun; Guo, Ling; Karniadakis, George Em SIAM Journal on Scientific Computing, Vol. 42, Issue 2 https://doi.org/10.1137/19M1260141	journal	January 2020
DeepXDE: A Deep Learning Library for Solving Differential Equations Lu, Lu; Meng, Xuhui; Mao, Zhiping SIAM Review, Vol. 63, Issue 1 https://doi.org/10.1137/19M1274067	journal	January 2021
A Phase Shift Deep Neural Network for High Frequency Approximation and Wave Problems Cai, Wei; Li, Xiaoguang; Liu, Lizuo SIAM Journal on Scientific Computing, Vol. 42, Issue 5 https://doi.org/10.1137/19M1310050	journal	January 2020
Physics-Informed Neural Networks with Hard Constraints for Inverse Design Lu, Lu; Pestourie, Raphaël; Yao, Wenjie SIAM Journal on Scientific Computing, Vol. 43, Issue 6 https://doi.org/10.1137/21M1397908	journal	January 2021
Systematic Construction of Neural Forms for Solving Partial Differential Equations Inside Rectangular Domains, Subject to Initial, Boundary and Interface Conditions Lagari, Pola Lydia; Tsoukalas, Lefteri H.; Safarkhani, Salar International Journal on Artificial Intelligence Tools, Vol. 29, Issue 05 https://doi.org/10.1142/S0218213020500098	journal	August 2020
Unifying Adversarial Training Algorithms with Data Gradient Regularization Ororbia II, Alexander G.; Kifer, Daniel; Giles, C. Lee Neural Computation, Vol. 29, Issue 4 https://doi.org/10.1162/NECO_a_00928	journal	April 2017
Physics-informed neural networks for inverse problems in nano-optics and metamaterials Chen, Yuyao; Lu, Lu; Karniadakis, George Em Optics Express, Vol. 28, Issue 8 https://doi.org/10.1364/OE.384875	journal	January 2020
Systems biology informed deep learning for inferring parameters and hidden dynamics Yazdani, Alireza; Lu, Lu; Raissi, Maziar PLOS Computational Biology, Vol. 16, Issue 11 https://doi.org/10.1371/journal.pcbi.1007575	journal	November 2020
Physics-Informed Neural Networks for Cardiac Activation Mapping Sahli Costabal, Francisco; Yang, Yibo; Perdikaris, Paris Frontiers in Physics, Vol. 8 https://doi.org/10.3389/fphy.2020.00042	journal	February 2020
Extended Physics-Informed Neural Networks (XPINNs): A Generalized Space-Time Domain Decomposition Based Deep Learning Framework for Nonlinear Partial Differential Equations Karniadakis, Ameya D. Jagtap & George Em Communications in Computational Physics, Vol. 28, Issue 5 https://doi.org/10.4208/cicp.OA-2020-0164	journal	June 2020

Similar Records

A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks

Journal Article · Sat Oct 22 00:00:00 EDT 2022 · Computer Methods in Applied Mechanics and Engineering · OSTI ID:1903149

B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data

Journal Article · Thu Oct 15 00:00:00 EDT 2020 · Journal of Computational Physics · OSTI ID:2282008

On the Convergence of Physics Informed Neural Networks for Linear Second-Order Elliptic and Parabolic Type PDEs

Journal Article · Sun Nov 01 00:00:00 EDT 2020 · Communications in Computational Physics · OSTI ID:2281993

Related Subjects

42 ENGINEERING
97 MATHEMATICS AND COMPUTING
deep learning
gradient-enhanced
partial differential equations
physics-informed neural networks
residual-based adaptive refinement

Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems

Citation Formats

References (24)

Similar Records

Related Subjects