PPINN: Parareal physics-informed neural network for time-dependent PDEs
Abstract
Physics-informed neural networks (PINNs) encode physical conservation laws and prior physical knowledge into the neural networks, ensuring the correct physics is represented accurately while alleviating the need for supervised learning to a great degree (Raissi et al., 2019). While effective for relatively short-term time integration, when long time integration of the time-dependent PDEs is sought, the time–space domain may become arbitrarily large and hence training of the neural network may become prohibitively expensive. To this end, we develop a parareal physics-informed neural network (PPINN), hence decomposing a long-time problem into many independent short-time problems supervised by an inexpensive/fast coarse-grained (CG) solver. In particular, the serial CG solver is designed to provide approximate predictions of the solution at discrete times, while initiate many fine PINNs simultaneously to correct the solution iteratively. There is a two-fold benefit from training PINNs with small-data sets rather than working on a large-data set directly, i.e., training of individual PINNs with small-data is much faster, while training the fine PINNs can be readily parallelized. Consequently, compared to the original PINN approach, the proposed PPINN approach may achieve a significant speed-up for long-time integration of PDEs, assuming that the CG solver is fast and can provide reasonablemore »
- Authors:
-
- Brown Univ., Providence, RI (United States)
- Clemson Univ., SC (United States)
- Publication Date:
- Research Org.:
- Brown Univ., Providence, RI (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC)
- OSTI Identifier:
- 1853246
- Alternate Identifier(s):
- OSTI ID: 1637715; OSTI ID: 2281998
- Grant/Contract Number:
- SC0019434; SC0019453
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Computer Methods in Applied Mechanics and Engineering
- Additional Journal Information:
- Journal Volume: 370; Journal Issue: C; Journal ID: ISSN 0045-7825
- Publisher:
- Elsevier
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 42 ENGINEERING; 97 MATHEMATICS AND COMPUTING; engineering; mathematics; mechanics; deep neural network; machine learning; parallel-in-time; long-time integration; multiscale; PINN
Citation Formats
Meng, Xuhui, Li, Zhen, Zhang, Dongkun, and Karniadakis, George Em. PPINN: Parareal physics-informed neural network for time-dependent PDEs. United States: N. p., 2020.
Web. doi:10.1016/j.cma.2020.113250.
Meng, Xuhui, Li, Zhen, Zhang, Dongkun, & Karniadakis, George Em. PPINN: Parareal physics-informed neural network for time-dependent PDEs. United States. https://doi.org/10.1016/j.cma.2020.113250
Meng, Xuhui, Li, Zhen, Zhang, Dongkun, and Karniadakis, George Em. Wed .
"PPINN: Parareal physics-informed neural network for time-dependent PDEs". United States. https://doi.org/10.1016/j.cma.2020.113250. https://www.osti.gov/servlets/purl/1853246.
@article{osti_1853246,
title = {PPINN: Parareal physics-informed neural network for time-dependent PDEs},
author = {Meng, Xuhui and Li, Zhen and Zhang, Dongkun and Karniadakis, George Em},
abstractNote = {Physics-informed neural networks (PINNs) encode physical conservation laws and prior physical knowledge into the neural networks, ensuring the correct physics is represented accurately while alleviating the need for supervised learning to a great degree (Raissi et al., 2019). While effective for relatively short-term time integration, when long time integration of the time-dependent PDEs is sought, the time–space domain may become arbitrarily large and hence training of the neural network may become prohibitively expensive. To this end, we develop a parareal physics-informed neural network (PPINN), hence decomposing a long-time problem into many independent short-time problems supervised by an inexpensive/fast coarse-grained (CG) solver. In particular, the serial CG solver is designed to provide approximate predictions of the solution at discrete times, while initiate many fine PINNs simultaneously to correct the solution iteratively. There is a two-fold benefit from training PINNs with small-data sets rather than working on a large-data set directly, i.e., training of individual PINNs with small-data is much faster, while training the fine PINNs can be readily parallelized. Consequently, compared to the original PINN approach, the proposed PPINN approach may achieve a significant speed-up for long-time integration of PDEs, assuming that the CG solver is fast and can provide reasonable predictions of the solution, hence aiding the PPINN solution to converge in just a few iterations. To investigate the PPINN performance on solving time-dependent PDEs, we first apply the PPINN to solve the Burgers equation, and subsequently we apply the PPINN to solve a two-dimensional nonlinear diffusion–reaction equation. Our results demonstrate that PPINNs converge in a few iterations with significant speed-ups proportional to the number of time-subdomains employed.},
doi = {10.1016/j.cma.2020.113250},
journal = {Computer Methods in Applied Mechanics and Engineering},
number = C,
volume = 370,
place = {United States},
year = {Wed Jul 08 00:00:00 EDT 2020},
month = {Wed Jul 08 00:00:00 EDT 2020}
}
Free Publicly Available Full Text
Publisher's Version of Record
Other availability
Search WorldCat to find libraries that may hold this journal
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
A Multiresolution Strategy for Reduction of Elliptic PDEs and Eigenvalue Problems
journal, April 1998
- Beylkin, Gregory; Coult, Nicholas
- Applied and Computational Harmonic Analysis, Vol. 5, Issue 2
Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification
journal, December 2018
- Tripathy, Rohit K.; Bilionis, Ilias
- Journal of Computational Physics, Vol. 375
ConvPDE-UQ: Convolutional neural networks with quantified uncertainty for heterogeneous elliptic partial differential equations on varied domains
journal, October 2019
- Winovich, Nick; Ramani, Karthik; Lin, Guang
- Journal of Computational Physics, Vol. 394
Finite difference methods for two-dimensional fractional dispersion equation
journal, January 2006
- Meerschaert, Mark M.; Scheffler, Hans-Peter; Tadjeran, Charles
- Journal of Computational Physics, Vol. 211, Issue 1
Localized lattice Boltzmann equation model for simulating miscible viscous displacement in porous media
journal, September 2016
- Meng, Xuhui; Guo, Zhaoli
- International Journal of Heat and Mass Transfer, Vol. 100
Optimization Methods for Large-Scale Machine Learning
journal, January 2018
- Bottou, Léon; Curtis, Frank E.; Nocedal, Jorge
- SIAM Review, Vol. 60, Issue 2
Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
journal, February 2019
- Raissi, M.; Perdikaris, P.; Karniadakis, G. E.
- Journal of Computational Physics, Vol. 378
Time-decomposed parallel time-integrators: theory and feasibility studies for fluid, structure, and fluid-structure applications
journal, January 2003
- Farhat, Charbel; Chandesris, Marion
- International Journal for Numerical Methods in Engineering, Vol. 58, Issue 9
Implicit Parallel Time Integrators
journal, December 2010
- Christlieb, Andrew; Ong, Benjamin
- Journal of Scientific Computing, Vol. 49, Issue 2
A Parallel Space-Time Algorithm
journal, January 2012
- Christlieb, Andrew J.; Haynes, Ronald D.; Ong, Benjamin W.
- SIAM Journal on Scientific Computing, Vol. 34, Issue 5
Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems
journal, November 2019
- Zhang, Dongkun; Lu, Lu; Guo, Ling
- Journal of Computational Physics, Vol. 397
Adversarial uncertainty quantification in physics-informed neural networks
journal, October 2019
- Yang, Yibo; Perdikaris, Paris
- Journal of Computational Physics, Vol. 394
Supervised parallel-in-time algorithm for long-time Lagrangian simulations of stochastic dynamics: Application to hydrodynamics
journal, September 2019
- Blumers, Ansel L.; Li, Zhen; Karniadakis, George Em
- Journal of Computational Physics, Vol. 393
Parareal in time 3D numerical solver for the LWR Benchmark neutron diffusion transient model
journal, December 2014
- Baudron, Anne-Marie; Lautard, Jean-Jacques; Maday, Yvon
- Journal of Computational Physics, Vol. 279
A Micro-Macro Parareal Algorithm: Application to Singularly Perturbed Ordinary Differential Equations
journal, January 2013
- Legoll, Frédéric; Lelièvre, Tony; Samaey, Giovanni
- SIAM Journal on Scientific Computing, Vol. 35, Issue 4
Neural-network-based approximations for solving partial differential equations
journal, March 1994
- Dissanayake, M. W. M. G.; Phan-Thien, N.
- Communications in Numerical Methods in Engineering, Vol. 10, Issue 3
An Asymptotic Parallel-in-Time Method for Highly Oscillatory PDEs
journal, January 2014
- Haut, Terry; Wingate, Beth
- SIAM Journal on Scientific Computing, Vol. 36, Issue 2
Supervised parallel-in-time algorithm for long-time Lagrangian simulations of stochastic dynamics: Application to hydrodynamics
journal, September 2019
- Blumers, Ansel L.; Li, Zhen; Karniadakis, George Em
- Journal of Computational Physics, Vol. 393
A parareal in time procedure for the control of partial differential equations
journal, January 2002
- Maday, Yvon; Turinici, Gabriel
- Comptes Rendus Mathematique, Vol. 335, Issue 4
Doing the Impossible: Why Neural Networks Can Be Trained at All
journal, July 2018
- Hodas, Nathan O.; Stinis, Panos
- Frontiers in Psychology, Vol. 9
On the Computational Efficiency of Training Neural Networks
preprint, January 2014
- Livni, Roi; Shalev-Shwartz, Shai; Shamir, Ohad
- arXiv
Physical Symmetries Embedded in Neural Networks
preprint, January 2019
- Mattheakis, M.; Protopapas, P.; Sondak, D.
- arXiv
Solving Irregular and Data-enriched Differential Equations using Deep Neural Networks
preprint, January 2019
- Michoski, Craig; Milosavljevic, Milos; Oliver, Todd
- arXiv