Search Menu
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information
Search Scholarly Publications

Title: Scalable algorithms for physics-informed neural and graph networks

Abstract

Physics-informed machine learning (PIML) has emerged as a promising new approach for simulating complex physical and biological systems that are governed by complex multiscale processes for which some data are also available. In some instances, the objective is to discover part of the hidden physics from the available data, and PIML has been shown to be particularly effective for such problems for which conventional methods may fail. Unlike commercial machine learning where training of deep neural networks requires big data, in PIML big data are not available. Instead, we can train such networks from additional information obtained by employing the physical laws and evaluating them at random points in the space–time domain. Such PIML integrates multimodality and multifidelity data with mathematical models, and implements them using neural networks or graph networks. Here, we review some of the prevailing trends in embedding physics into machine learning, using physics-informed neural networks (PINNs) based primarily on feed-forward neural networks and automatic differentiation. For more complex systems or systems of systems and unstructured data, graph neural networks (GNNs) present some distinct advantages, and here we review how physics-informed learning can be accomplished with GNNs based on graph exterior calculus to construct differential operators; wemore » refer to these architectures as physics-informed graph networks (PIGNs). We present representative examples for both forward and inverse problems and discuss what advances are needed to scale up PINNs, PIGNs and more broadly GNNs for large-scale engineering problems.« less

Authors:
 [1];  [2];  [3]; ORCiD logo [1]
+ Show Author Affiliations
  1. Brown University, Providence, RI (United States)
  2. Brown University, Providence, RI (United States); Massachusetts Institute of Technology (MIT), Cambridge, MA (United States)
  3. Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
Publication Date:
Research Org.:
Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
OSTI Identifier:
1872036
Report Number(s):
SAND2022-6955J
Journal ID: ISSN 2632-6736; 706667
Grant/Contract Number:  
NA0003525; SC0019453; FA9550-20-1-0358
Resource Type:
Accepted Manuscript
Journal Name:
Data-Centric Engineering
Additional Journal Information:
Journal Volume: 3; Journal ID: ISSN 2632-6736
Publisher:
Cambridge University Press
Country of Publication:
United States
Language:
English
Subject:
42 ENGINEERING; graph calculus; graph neural networks; PINNS; scalability; scientific machine learning

Citation Formats

Shukla, Khemraj, Xu, Mengjia, Trask, Nathaniel, and Karniadakis, George E. Scalable algorithms for physics-informed neural and graph networks. United States: N. p., 2022. Web. doi:10.1017/dce.2022.24.
Copy to clipboard
Shukla, Khemraj, Xu, Mengjia, Trask, Nathaniel, & Karniadakis, George E. Scalable algorithms for physics-informed neural and graph networks. United States. https://doi.org/10.1017/dce.2022.24
Copy to clipboard
Shukla, Khemraj, Xu, Mengjia, Trask, Nathaniel, and Karniadakis, George E. Wed . "Scalable algorithms for physics-informed neural and graph networks". United States. https://doi.org/10.1017/dce.2022.24. https://www.osti.gov/servlets/purl/1872036.
Copy to clipboard
@article{osti_1872036,
title = {Scalable algorithms for physics-informed neural and graph networks},
author = {Shukla, Khemraj and Xu, Mengjia and Trask, Nathaniel and Karniadakis, George E.},
abstractNote = {Physics-informed machine learning (PIML) has emerged as a promising new approach for simulating complex physical and biological systems that are governed by complex multiscale processes for which some data are also available. In some instances, the objective is to discover part of the hidden physics from the available data, and PIML has been shown to be particularly effective for such problems for which conventional methods may fail. Unlike commercial machine learning where training of deep neural networks requires big data, in PIML big data are not available. Instead, we can train such networks from additional information obtained by employing the physical laws and evaluating them at random points in the space–time domain. Such PIML integrates multimodality and multifidelity data with mathematical models, and implements them using neural networks or graph networks. Here, we review some of the prevailing trends in embedding physics into machine learning, using physics-informed neural networks (PINNs) based primarily on feed-forward neural networks and automatic differentiation. For more complex systems or systems of systems and unstructured data, graph neural networks (GNNs) present some distinct advantages, and here we review how physics-informed learning can be accomplished with GNNs based on graph exterior calculus to construct differential operators; we refer to these architectures as physics-informed graph networks (PIGNs). We present representative examples for both forward and inverse problems and discuss what advances are needed to scale up PINNs, PIGNs and more broadly GNNs for large-scale engineering problems.},
doi = {10.1017/dce.2022.24},
journal = {Data-Centric Engineering},
number = ,
volume = 3,
place = {United States},
year = {Wed Jun 29 00:00:00 EDT 2022},
month = {Wed Jun 29 00:00:00 EDT 2022}
}
Copy to clipboard
Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
https://doi.org/10.1017/dce.2022.24
Copyright Statement
Other availability
Search WorldCat Search WorldCat to find libraries that may hold this journal
Save / Share:
Export Metadata
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.

Works referenced in this record:

EvolveGCN: Evolving Graph Convolutional Networks for Dynamic Graphs
journal, April 2020

  • Pareja, Aldo; Domeniconi, Giacomo; Chen, Jie
  • Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34, Issue 04
  • DOI: 10.1609/aaai.v34i04.5984

Mapping higher-order relations between brain structure and function with embedded vector representations of connectomes
journal, June 2018

  • Rosenthal, Gideon; Váša, František; Griffa, Alessandra
  • Nature Communications, Vol. 9, Issue 1
  • DOI: 10.1038/s41467-018-04614-w

Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations
journal, January 2020

  • Raissi, Maziar; Yazdani, Alireza; Karniadakis, George Em
  • Science, Vol. 367, Issue 6481
  • DOI: 10.1126/science.aaw4741

The general inefficiency of batch training for gradient descent learning
journal, December 2003

Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems
journal, November 2019

Physics-informed machine learning
journal, May 2021

GINNs: Graph-Informed Neural Networks for multiscale physics
journal, May 2021

  • Hall, Eric J.; Taverniers, Søren; Katsoulakis, Markos A.
  • Journal of Computational Physics, Vol. 433
  • DOI: 10.1016/j.jcp.2021.110192

A Physics-Informed Neural Network for Quantifying the Microstructural Properties of Polycrystalline Nickel Using Ultrasound Data: A promising approach for solving inverse problems
journal, January 2022

  • Shukla, Khemraj; Jagtap, Ameya D.; Blackshire, James L.
  • IEEE Signal Processing Magazine, Vol. 39, Issue 1
  • DOI: 10.1109/MSP.2021.3118904

DeepXDE: A Deep Learning Library for Solving Differential Equations
journal, January 2021

Systems biology informed deep learning for inferring parameters and hidden dynamics
journal, November 2020

Chemi-Net: A Molecular Graph Convolutional Network for Accurate Drug Property Prediction
journal, July 2019

  • Liu, Ke; Sun, Xiangyan; Jia, Lei
  • International Journal of Molecular Sciences, Vol. 20, Issue 14
  • DOI: 10.3390/ijms20143389

Conservative physics-informed neural networks on discrete domains for conservation laws: Applications to forward and inverse problems
journal, June 2020

  • Jagtap, Ameya D.; Kharazmi, Ehsan; Karniadakis, George Em
  • Computer Methods in Applied Mechanics and Engineering, Vol. 365
  • DOI: 10.1016/j.cma.2020.113028

Physics-constrained deep learning of multi-zone building thermal dynamics
journal, July 2021

fPINNs: Fractional Physics-Informed Neural Networks
journal, January 2019

  • Pang, Guofei; Lu, Lu; Karniadakis, George Em
  • SIAM Journal on Scientific Computing, Vol. 41, Issue 4
  • DOI: 10.1137/18M1229845

Physics-Informed Neural Networks with Hard Constraints for Inverse Design
journal, January 2021

  • Lu, Lu; Pestourie, Raphaël; Yao, Wenjie
  • SIAM Journal on Scientific Computing, Vol. 43, Issue 6
  • DOI: 10.1137/21M1397908

hp-VPINNs: Variational physics-informed neural networks with domain decomposition
journal, February 2021

  • Kharazmi, Ehsan; Zhang, Zhongqiang; Karniadakis, George E. M.
  • Computer Methods in Applied Mechanics and Engineering, Vol. 374
  • DOI: 10.1016/j.cma.2020.113547

Physics-informed neural networks for high-speed flows
journal, March 2020

  • Mao, Zhiping; Jagtap, Ameya D.; Karniadakis, George Em
  • Computer Methods in Applied Mechanics and Engineering, Vol. 360
  • DOI: 10.1016/j.cma.2019.112789

On generalized moving least squares and diffuse derivatives
journal, September 2011

  • Mirzaei, D.; Schaback, R.; Dehghan, M.
  • IMA Journal of Numerical Analysis, Vol. 32, Issue 3
  • DOI: 10.1093/imanum/drr030

Highly accurate protein structure prediction with AlphaFold
journal, July 2021

Efficient BackProp
book, January 2012

Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems
journal, April 2022

  • Yu, Jeremy; Lu, Lu; Meng, Xuhui
  • Computer Methods in Applied Mechanics and Engineering, Vol. 393
  • DOI: 10.1016/j.cma.2022.114823

Accurate, Efficient and Scalable Graph Embedding
conference, May 2019

  • Zeng, Hanqing; Zhou, Hongkuan; Srivastava, Ajitesh
  • 2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS)
  • DOI: 10.1109/IPDPS.2019.00056

Understanding Graph Embedding Methods and Their Applications
journal, January 2021

Enforcing exact physics in scientific machine learning: A data-driven exterior calculus on graphs
journal, May 2022

Non-invasive inference of thrombus material properties with physics-informed neural networks
journal, March 2021

  • Yin, Minglang; Zheng, Xiaoning; Humphrey, Jay D.
  • Computer Methods in Applied Mechanics and Engineering, Vol. 375
  • DOI: 10.1016/j.cma.2020.113603

Advancing mathematics by guiding human intuition with AI
journal, December 2021

Numerical solution of initial boundary value problems involving maxwell's equations in isotropic media
journal, May 1966

A Recommendation System Based on Hierarchical Clustering of an Article-Level Citation Network
journal, June 2016

  • West, Jevin D.; Wesley-Smith, Ian; Bergstrom, Carl T.
  • IEEE Transactions on Big Data, Vol. 2, Issue 2
  • DOI: 10.1109/TBDATA.2016.2541167

EXAGRAPH: Graph and combinatorial methods for enabling exascale applications
journal, September 2021

  • Acer, Seher; Azad, Ariful; Boman, Erik G.
  • The International Journal of High Performance Computing Applications, Vol. 35, Issue 6
  • DOI: 10.1177/10943420211029299

Topological Data Analysis
journal, March 2018

User identity linkage across social networks via linked heterogeneous network embedding
journal, April 2018

Physics-Informed Graph Neural Network for Circuit Compact Model Development
conference, September 2020

  • Gao, Xujiao; Huang, Andy; Trask, Nathaniel
  • 2020 International Conference on Simulation of Semiconductor Processes and Devices (SISPAD)
  • DOI: 10.23919/SISPAD49475.2020.9241634

Spectral/hp Element Methods for Computational Fluid Dynamics
book, January 2005

DynG2G: An Efficient Stochastic Graph Embedding Method for Temporal Graphs
journal, January 2022

  • Xu, Mengjia; Singh, Apoorva Vikram; Karniadakis, George Em
  • IEEE Transactions on Neural Networks and Learning Systems
  • DOI: 10.1109/TNNLS.2022.3178706

Artificial neural networks for solving ordinary and partial differential equations
journal, January 1998

  • Lagaris, I. E.; Likas, A.; Fotiadis, D. I.
  • IEEE Transactions on Neural Networks, Vol. 9, Issue 5
  • DOI: 10.1109/72.712178

Topology and data
journal, January 2009

Statistical ranking and combinatorial Hodge theory
journal, November 2010

Physics-Informed Neural Network for Ultrasound Nondestructive Quantification of Surface Breaking Cracks
journal, August 2020

  • Shukla, Khemraj; Di Leoni, Patricio Clark; Blackshire, James
  • Journal of Nondestructive Evaluation, Vol. 39, Issue 3
  • DOI: 10.1007/s10921-020-00705-1

Asset diversification and systemic risk in the financial system
journal, November 2017

  • Zhou, Yichen; Li, Honggang
  • Journal of Economic Interaction and Coordination, Vol. 14, Issue 2
  • DOI: 10.1007/s11403-017-0205-4

Systematic Construction of Neural Forms for Solving Partial Differential Equations Inside Rectangular Domains, Subject to Initial, Boundary and Interface Conditions
journal, August 2020

  • Lagari, Pola Lydia; Tsoukalas, Lefteri H.; Safarkhani, Salar
  • International Journal on Artificial Intelligence Tools, Vol. 29, Issue 05
  • DOI: 10.1142/S0218213020500098

Multisymplectic Geometry, Variational Integrators, and Nonlinear PDEs
journal, December 1998

  • Marsden, Jerrold E.; Patrick, George W.; Shkoller, Steve
  • Communications in Mathematical Physics, Vol. 199, Issue 2
  • DOI: 10.1007/s002200050505

Learning in Modal Space: Solving Time-Dependent Stochastic PDEs Using Physics-Informed Neural Networks
journal, January 2020

  • Zhang, Dongkun; Guo, Ling; Karniadakis, George Em
  • SIAM Journal on Scientific Computing, Vol. 42, Issue 2
  • DOI: 10.1137/19M1260141

Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
journal, February 2019

A new Graph Gaussian embedding method for analyzing the effects of cognitive training
journal, September 2020

Learning data-driven discretizations for partial differential equations
journal, July 2019

  • Bar-Sinai, Yohai; Hoyer, Stephan; Hickey, Jason
  • Proceedings of the National Academy of Sciences, Vol. 116, Issue 31
  • DOI: 10.1073/pnas.1814058116

Physics-Informed Neural Networks for Cardiac Activation Mapping
journal, February 2020

  • Sahli Costabal, Francisco; Yang, Yibo; Perdikaris, Paris
  • Frontiers in Physics, Vol. 8
  • DOI: 10.3389/fphy.2020.00042

DeepSpeed
conference, August 2020

  • Rasley, Jeff; Rajbhandari, Samyam; Ruwase, Olatunji
  • Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining
  • DOI: 10.1145/3394486.3406703

A Graph Gaussian Embedding Method for Predicting Alzheimer's Disease Progression With MEG Brain Networks
journal, May 2021

  • Xu, Mengjia; Sanz, David Lopez; Garces, Pilar
  • IEEE Transactions on Biomedical Engineering, Vol. 68, Issue 5
  • DOI: 10.1109/TBME.2021.3049199

Principles of Mimetic Discretizations of Differential Operators
book,  

Numerical Gaussian Processes for Time-Dependent and Nonlinear Partial Differential Equations
journal, January 2018

  • Raissi, Maziar; Perdikaris, Paris; Karniadakis, George Em
  • SIAM Journal on Scientific Computing, Vol. 40, Issue 1
  • DOI: 10.1137/17M1120762

Numerical Calculation of Time-Dependent Viscous Incompressible Flow of Fluid with Free Surface
journal, January 1965

  • Harlow, Francis H.; Welch, J. Eddie
  • Physics of Fluids, Vol. 8, Issue 12
  • DOI: 10.1063/1.1761178

Physics-informed graph neural Galerkin networks: A unified framework for solving PDE-governed forward and inverse problems
journal, February 2022

  • Gao, Han; Zahr, Matthew J.; Wang, Jian-Xun
  • Computer Methods in Applied Mechanics and Engineering, Vol. 390
  • DOI: 10.1016/j.cma.2021.114502

Thermodynamics-informed Graph Neural Networks
journal, January 2022

  • Hernandez, Quercus; Badias, Alberto; Chinesta, Francisco
  • IEEE Transactions on Artificial Intelligence
  • DOI: 10.1109/TAI.2022.3179681

Parallel physics-informed neural networks via domain decomposition
journal, December 2021

  • Shukla, Khemraj; Jagtap, Ameya D.; Karniadakis, George Em
  • Journal of Computational Physics, Vol. 447
  • DOI: 10.1016/j.jcp.2021.110683
    Sort by:
    [ × clear filter / sort ]

    Works referencing / citing this record:

    Multimodal learning with graphs
    journal, April 2023

      Sort by:
      [ × clear filter / sort ]

      Similar Records in DOE PAGES and OSTI.GOV collections: