Scalable algorithms for physics-informed neural and graph networks

Shukla, Khemraj; Xu, Mengjia; Trask, Nathaniel; Karniadakis, George E.

doi:10.1017/dce.2022.24

Scalable algorithms for physics-informed neural and graph networks

Journal Article · 29 June 2022 · Data-Centric Engineering

DOI:https://doi.org/10.1017/dce.2022.24· OSTI ID:1872036

Shukla, Khemraj ^[1]; Xu, Mengjia ^[2]; Trask, Nathaniel ^[3]; ^[1]

Brown University, Providence, RI (United States)
Brown University, Providence, RI (United States); Massachusetts Institute of Technology (MIT), Cambridge, MA (United States)
Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)

Physics-informed machine learning (PIML) has emerged as a promising new approach for simulating complex physical and biological systems that are governed by complex multiscale processes for which some data are also available. In some instances, the objective is to discover part of the hidden physics from the available data, and PIML has been shown to be particularly effective for such problems for which conventional methods may fail. Unlike commercial machine learning where training of deep neural networks requires big data, in PIML big data are not available. Instead, we can train such networks from additional information obtained by employing the physical laws and evaluating them at random points in the space–time domain. Such PIML integrates multimodality and multifidelity data with mathematical models, and implements them using neural networks or graph networks. Here, we review some of the prevailing trends in embedding physics into machine learning, using physics-informed neural networks (PINNs) based primarily on feed-forward neural networks and automatic differentiation. For more complex systems or systems of systems and unstructured data, graph neural networks (GNNs) present some distinct advantages, and here we review how physics-informed learning can be accomplished with GNNs based on graph exterior calculus to construct differential operators; we refer to these architectures as physics-informed graph networks (PIGNs). We present representative examples for both forward and inverse problems and discuss what advances are needed to scale up PINNs, PIGNs and more broadly GNNs for large-scale engineering problems.

View Accepted Manuscript (DOE)

Research Organization:: Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)

Grant/Contract Number:: NA0003525; SC0019453; FA9550-20-1-0358

OSTI ID:: 1872036

Report Number(s):: SAND2022-6955J; 706667

Journal Information:: Data-Centric Engineering, Vol. 3; ISSN 2632-6736

Publisher:: Cambridge University PressCopyright Statement

Country of Publication:: United States

Language:: English

References (57)

Protein protein interaction network analysis of differentially expressed genes to understand involved biological processes in coronary artery disease and its different severity Kashyap, Shiridhar; Kumar, Sudeep; Agarwal, Vikas Gene Reports, Vol. 12 https://doi.org/10.1016/j.genrep.2018.05.008	journal	September 2018
EvolveGCN: Evolving Graph Convolutional Networks for Dynamic Graphs Pareja, Aldo; Domeniconi, Giacomo; Chen, Jie Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34, Issue 04 https://doi.org/10.1609/aaai.v34i04.5984	journal	April 2020
Mapping higher-order relations between brain structure and function with embedded vector representations of connectomes Rosenthal, Gideon; Váša, František; Griffa, Alessandra Nature Communications, Vol. 9, Issue 1 https://doi.org/10.1038/s41467-018-04614-w	journal	June 2018
Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations Raissi, Maziar; Yazdani, Alireza; Karniadakis, George Em Science, Vol. 367, Issue 6481 https://doi.org/10.1126/science.aaw4741	journal	January 2020
The general inefficiency of batch training for gradient descent learning Wilson, D. Randall; Martinez, Tony R. Neural Networks, Vol. 16, Issue 10 https://doi.org/10.1016/S0893-6080(03)00138-2	journal	December 2003
Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems Zhang, Dongkun; Lu, Lu; Guo, Ling Journal of Computational Physics, Vol. 397 https://doi.org/10.1016/j.jcp.2019.07.048	journal	November 2019
Physics-informed machine learning Karniadakis, George Em; Kevrekidis, Ioannis G.; Lu, Lu Nature Reviews Physics, Vol. 3, Issue 6 https://doi.org/10.1038/s42254-021-00314-5	journal	May 2021
GINNs: Graph-Informed Neural Networks for multiscale physics Hall, Eric J.; Taverniers, Søren; Katsoulakis, Markos A. Journal of Computational Physics, Vol. 433 https://doi.org/10.1016/j.jcp.2021.110192	journal	May 2021
NVIDIA SimNet™: An AI-Accelerated Multi-Physics Simulation Framework Hennigh, Oliver; Narasimhan, Susheela; Nabian, Mohammad Amin Computational Science – ICCS 2021 https://doi.org/10.1007/978-3-030-77977-1_36	book	January 2021
A Physics-Informed Neural Network for Quantifying the Microstructural Properties of Polycrystalline Nickel Using Ultrasound Data: A promising approach for solving inverse problems Shukla, Khemraj; Jagtap, Ameya D.; Blackshire, James L. IEEE Signal Processing Magazine, Vol. 39, Issue 1 https://doi.org/10.1109/MSP.2021.3118904	journal	January 2022
DeepXDE: A Deep Learning Library for Solving Differential Equations Lu, Lu; Meng, Xuhui; Mao, Zhiping SIAM Review, Vol. 63, Issue 1 https://doi.org/10.1137/19M1274067	journal	January 2021
Systems biology informed deep learning for inferring parameters and hidden dynamics Yazdani, Alireza; Lu, Lu; Raissi, Maziar PLOS Computational Biology, Vol. 16, Issue 11 https://doi.org/10.1371/journal.pcbi.1007575	journal	November 2020
Chemi-Net: A Molecular Graph Convolutional Network for Accurate Drug Property Prediction Liu, Ke; Sun, Xiangyan; Jia, Lei International Journal of Molecular Sciences, Vol. 20, Issue 14 https://doi.org/10.3390/ijms20143389	journal	July 2019
Conservative physics-informed neural networks on discrete domains for conservation laws: Applications to forward and inverse problems Jagtap, Ameya D.; Kharazmi, Ehsan; Karniadakis, George Em Computer Methods in Applied Mechanics and Engineering, Vol. 365 https://doi.org/10.1016/j.cma.2020.113028	journal	June 2020
Physics-constrained deep learning of multi-zone building thermal dynamics Drgoňa, Ján; Tuor, Aaron R.; Chandan, Vikas Energy and Buildings, Vol. 243 https://doi.org/10.1016/j.enbuild.2021.110992	journal	July 2021
fPINNs: Fractional Physics-Informed Neural Networks Pang, Guofei; Lu, Lu; Karniadakis, George Em SIAM Journal on Scientific Computing, Vol. 41, Issue 4 https://doi.org/10.1137/18M1229845	journal	January 2019
Physics-Informed Neural Networks with Hard Constraints for Inverse Design Lu, Lu; Pestourie, Raphaël; Yao, Wenjie SIAM Journal on Scientific Computing, Vol. 43, Issue 6 https://doi.org/10.1137/21M1397908	journal	January 2021
hp-VPINNs: Variational physics-informed neural networks with domain decomposition Kharazmi, Ehsan; Zhang, Zhongqiang; Karniadakis, George E. M. Computer Methods in Applied Mechanics and Engineering, Vol. 374 https://doi.org/10.1016/j.cma.2020.113547	journal	February 2021
Physics-informed neural networks for high-speed flows Mao, Zhiping; Jagtap, Ameya D.; Karniadakis, George Em Computer Methods in Applied Mechanics and Engineering, Vol. 360 https://doi.org/10.1016/j.cma.2019.112789	journal	March 2020
On generalized moving least squares and diffuse derivatives Mirzaei, D.; Schaback, R.; Dehghan, M. IMA Journal of Numerical Analysis, Vol. 32, Issue 3 https://doi.org/10.1093/imanum/drr030	journal	September 2011
Highly accurate protein structure prediction with AlphaFold Jumper, John; Evans, Richard; Pritzel, Alexander Nature https://doi.org/10.1038/s41586-021-03819-2	journal	July 2021
Efficient BackProp LeCun, Yann A.; Bottou, Léon; Orr, Genevieve B. Lecture Notes in Computer Science https://doi.org/10.1007/978-3-642-35289-8_3	book	January 2012
Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems Yu, Jeremy; Lu, Lu; Meng, Xuhui Computer Methods in Applied Mechanics and Engineering, Vol. 393 https://doi.org/10.1016/j.cma.2022.114823	journal	April 2022
Accurate, Efficient and Scalable Graph Embedding Zeng, Hanqing; Zhou, Hongkuan; Srivastava, Ajitesh 2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS) https://doi.org/10.1109/IPDPS.2019.00056	conference	May 2019
Understanding Graph Embedding Methods and Their Applications Xu, Mengjia SIAM Review, Vol. 63, Issue 4 https://doi.org/10.1137/20M1386062	journal	January 2021
Enforcing exact physics in scientific machine learning: A data-driven exterior calculus on graphs Trask, Nathaniel; Huang, Andy; Hu, Xiaozhe Journal of Computational Physics, Vol. 456 https://doi.org/10.1016/j.jcp.2022.110969	journal	May 2022
Non-invasive inference of thrombus material properties with physics-informed neural networks Yin, Minglang; Zheng, Xiaoning; Humphrey, Jay D. Computer Methods in Applied Mechanics and Engineering, Vol. 375 https://doi.org/10.1016/j.cma.2020.113603	journal	March 2021
Advancing mathematics by guiding human intuition with AI Davies, Alex; Veličković, Petar; Buesing, Lars Nature, Vol. 600, Issue 7887 https://doi.org/10.1038/s41586-021-04086-x	journal	December 2021
Numerical solution of initial boundary value problems involving maxwell's equations in isotropic media No authors listed IEEE Transactions on Antennas and Propagation, Vol. 14, Issue 3 https://doi.org/10.1109/TAP.1966.1138693	journal	May 1966
A Recommendation System Based on Hierarchical Clustering of an Article-Level Citation Network West, Jevin D.; Wesley-Smith, Ian; Bergstrom, Carl T. IEEE Transactions on Big Data, Vol. 2, Issue 2 https://doi.org/10.1109/TBDATA.2016.2541167	journal	June 2016
EXAGRAPH: Graph and combinatorial methods for enabling exascale applications Acer, Seher; Azad, Ariful; Boman, Erik G. The International Journal of High Performance Computing Applications, Vol. 35, Issue 6 https://doi.org/10.1177/10943420211029299	journal	September 2021
Topological Data Analysis Wasserman, Larry Annual Review of Statistics and Its Application, Vol. 5, Issue 1 https://doi.org/10.1146/annurev-statistics-031017-100045	journal	March 2018
User identity linkage across social networks via linked heterogeneous network embedding Wang, Yaqing; Feng, Chunyan; Chen, Ling World Wide Web, Vol. 22, Issue 6 https://doi.org/10.1007/s11280-018-0572-3	journal	April 2018
Physics-Informed Graph Neural Network for Circuit Compact Model Development Gao, Xujiao; Huang, Andy; Trask, Nathaniel 2020 International Conference on Simulation of Semiconductor Processes and Devices (SISPAD) https://doi.org/10.23919/SISPAD49475.2020.9241634	conference	September 2020
Spectral/hp Element Methods for Computational Fluid Dynamics Karniadakis, George; Sherwin, Spencer Oxford University Press https://doi.org/10.1093/acprof:oso/9780198528692.001.0001	book	January 2005
DynG2G: An Efficient Stochastic Graph Embedding Method for Temporal Graphs Xu, Mengjia; Singh, Apoorva Vikram; Karniadakis, George Em IEEE Transactions on Neural Networks and Learning Systems https://doi.org/10.1109/TNNLS.2022.3178706	journal	January 2022
Artificial neural networks for solving ordinary and partial differential equations Lagaris, I. E.; Likas, A.; Fotiadis, D. I. IEEE Transactions on Neural Networks, Vol. 9, Issue 5 https://doi.org/10.1109/72.712178	journal	January 1998
Topology and data Carlsson, Gunnar Bulletin of the American Mathematical Society, Vol. 46, Issue 2 https://doi.org/10.1090/S0273-0979-09-01249-X	journal	January 2009
Statistical ranking and combinatorial Hodge theory Jiang, Xiaoye; Lim, Lek-Heng; Yao, Yuan Mathematical Programming, Vol. 127, Issue 1 https://doi.org/10.1007/s10107-010-0419-x	journal	November 2010
Physics-Informed Neural Network for Ultrasound Nondestructive Quantification of Surface Breaking Cracks Shukla, Khemraj; Di Leoni, Patricio Clark; Blackshire, James Journal of Nondestructive Evaluation, Vol. 39, Issue 3 https://doi.org/10.1007/s10921-020-00705-1	journal	August 2020
Asset diversification and systemic risk in the financial system Zhou, Yichen; Li, Honggang Journal of Economic Interaction and Coordination, Vol. 14, Issue 2 https://doi.org/10.1007/s11403-017-0205-4	journal	November 2017
Systematic Construction of Neural Forms for Solving Partial Differential Equations Inside Rectangular Domains, Subject to Initial, Boundary and Interface Conditions Lagari, Pola Lydia; Tsoukalas, Lefteri H.; Safarkhani, Salar International Journal on Artificial Intelligence Tools, Vol. 29, Issue 05 https://doi.org/10.1142/S0218213020500098	journal	August 2020
Multisymplectic Geometry, Variational Integrators, and Nonlinear PDEs Marsden, Jerrold E.; Patrick, George W.; Shkoller, Steve Communications in Mathematical Physics, Vol. 199, Issue 2 https://doi.org/10.1007/s002200050505	journal	December 1998
Learning in Modal Space: Solving Time-Dependent Stochastic PDEs Using Physics-Informed Neural Networks Zhang, Dongkun; Guo, Ling; Karniadakis, George Em SIAM Journal on Scientific Computing, Vol. 42, Issue 2 https://doi.org/10.1137/19M1260141	journal	January 2020
Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations Raissi, M.; Perdikaris, P.; Karniadakis, G. E. Journal of Computational Physics, Vol. 378 https://doi.org/10.1016/j.jcp.2018.10.045	journal	February 2019
Extended Physics-Informed Neural Networks (XPINNs): A Generalized Space-Time Domain Decomposition Based Deep Learning Framework for Nonlinear Partial Differential Equations Karniadakis, Ameya D. Jagtap & George Em Communications in Computational Physics, Vol. 28, Issue 5 https://doi.org/10.4208/cicp.OA-2020-0164	journal	June 2020
GMLS-Nets: A Framework for Learning from Unstructured Data Trask, Nathaniel; Patel, Ravi; Gross, Ben https://doi.org/10.2172/1763256	report	September 2019
A new Graph Gaussian embedding method for analyzing the effects of cognitive training Xu, Mengjia; Wang, Zhijiang; Zhang, Haifeng PLOS Computational Biology, Vol. 16, Issue 9 https://doi.org/10.1371/journal.pcbi.1008186	journal	September 2020
Learning data-driven discretizations for partial differential equations Bar-Sinai, Yohai; Hoyer, Stephan; Hickey, Jason Proceedings of the National Academy of Sciences, Vol. 116, Issue 31 https://doi.org/10.1073/pnas.1814058116	journal	July 2019
Physics-Informed Neural Networks for Cardiac Activation Mapping Sahli Costabal, Francisco; Yang, Yibo; Perdikaris, Paris Frontiers in Physics, Vol. 8 https://doi.org/10.3389/fphy.2020.00042	journal	February 2020
DeepSpeed Rasley, Jeff; Rajbhandari, Samyam; Ruwase, Olatunji Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining https://doi.org/10.1145/3394486.3406703	conference	August 2020
A Graph Gaussian Embedding Method for Predicting Alzheimer's Disease Progression With MEG Brain Networks Xu, Mengjia; Sanz, David Lopez; Garces, Pilar IEEE Transactions on Biomedical Engineering, Vol. 68, Issue 5 https://doi.org/10.1109/TBME.2021.3049199	journal	May 2021
Principles of Mimetic Discretizations of Differential Operators Bochev, Pavel B.; Hyman, James M. Compatible Spatial Discretizations https://doi.org/10.1007/0-387-38034-5_5	book
Numerical Calculation of Time-Dependent Viscous Incompressible Flow of Fluid with Free Surface Harlow, Francis H.; Welch, J. Eddie Physics of Fluids, Vol. 8, Issue 12 https://doi.org/10.1063/1.1761178	journal	January 1965
Physics-informed graph neural Galerkin networks: A unified framework for solving PDE-governed forward and inverse problems Gao, Han; Zahr, Matthew J.; Wang, Jian-Xun Computer Methods in Applied Mechanics and Engineering, Vol. 390 https://doi.org/10.1016/j.cma.2021.114502	journal	February 2022
Thermodynamics-informed Graph Neural Networks Hernandez, Quercus; Badias, Alberto; Chinesta, Francisco IEEE Transactions on Artificial Intelligence https://doi.org/10.1109/TAI.2022.3179681	journal	January 2022
Parallel physics-informed neural networks via domain decomposition Shukla, Khemraj; Jagtap, Ameya D.; Karniadakis, George Em Journal of Computational Physics, Vol. 447 https://doi.org/10.1016/j.jcp.2021.110683	journal	December 2021

Cited By (1)

Multimodal learning with graphs Ektefaie, Yasha; Dasoulas, George; Noori, Ayush Nature Machine Intelligence, Vol. 5, Issue 4 https://doi.org/10.1038/s42256-023-00624-6	journal	April 2023

Similar Records

Learning functional priors and posteriors from data and physics

Journal Article · 2022 · Journal of Computational Physics · OSTI ID:1897198

Meng, Xuhui; Yang, Liu; Mao, Zhiping; +2 more

Physics-informed machine learning

Journal Article · 2021 · Nature Reviews Physics · OSTI ID:2282016

Karniadakis, George Em; Kevrekidis, Ioannis G.; Lu, Lu; +3 more

Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems

Journal Article · 2022 · Computer Methods in Applied Mechanics and Engineering · OSTI ID:1976976

Yu, Jeremy; Lu, Lu; Meng, Xuhui; +1 more

Related Subjects

42 ENGINEERING
graph calculus
graph neural networks
PINNS
scalability
scientific machine learning

Scalable algorithms for physics-informed neural and graph networks

Citation Formats

References (57)

Cited By (1)

Similar Records

Related Subjects