Search Menu
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information
Search Scholarly Publications

Title: Solving differential equations using deep neural networks

Abstract

Recent work on solving partial differential equations (PDEs) with deep neural networks (DNNs) is presented. The paper reviews and extends some of these methods while carefully analyzing a fundamental feature in numerical PDEs and nonlinear analysis: irregular solutions. First, the Sod shock tube solution to the compressible Euler equations is discussed and analyzed. This analysis includes a comparison of a DNN-based approach with conventional finite element and finite volume methods, and demonstrates that the DNN is competitive in terms of degrees of freedom required for a given accuracy. Further, the DNN-based approach is extended to consider performance improvements and simultaneous parameter space exploration. Next, a shock solution to compressible magnetohydrodynamics (MHD) is solved for, and used in a scenario where experimental data is utilized to enhance a PDE system that is a priori insufficient to validate against the observed/experimental data. This is accomplished by enriching the model PDE system with source terms that are then inferred via supervised training with synthetic experimental data. The resulting DNN framework for PDEs enables straightforward system prototyping and natural integration of large data sets (be they synthetic or experimental), all while simultaneously enabling single-pass exploration of an entire parameter space.

Authors:
ORCiD logo [1];  [1];  [1];  [1]
+ Show Author Affiliations
  1. Univ. of Texas, Austin, TX (United States)
Publication Date:
Research Org.:
Univ. of Texas, Austin, TX (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Fusion Energy Sciences (FES); National Science Foundation (NSF)
OSTI Identifier:
1785492
Alternate Identifier(s):
OSTI ID: 1630182
Grant/Contract Number:  
FG02-04ER54742; SC0018429; AST-1413501
Resource Type:
Accepted Manuscript
Journal Name:
Neurocomputing
Additional Journal Information:
Journal Volume: 399; Journal ID: ISSN 0925-2312
Publisher:
Elsevier
Country of Publication:
United States
Language:
English
Subject:
70 PLASMA PHYSICS AND FUSION TECHNOLOGY; Deep neural networks; Differential equations; Partial differential equations; Nonlinear; Shocks; Data analytics; Optimization

Citation Formats

Michoski, Craig, Milosavljević, Miloš, Oliver, Todd, and Hatch, David R. Solving differential equations using deep neural networks. United States: N. p., 2020. Web. doi:10.1016/j.neucom.2020.02.015.
Copy to clipboard
Michoski, Craig, Milosavljević, Miloš, Oliver, Todd, & Hatch, David R. Solving differential equations using deep neural networks. United States. https://doi.org/10.1016/j.neucom.2020.02.015
Copy to clipboard
Michoski, Craig, Milosavljević, Miloš, Oliver, Todd, and Hatch, David R. Wed . "Solving differential equations using deep neural networks". United States. https://doi.org/10.1016/j.neucom.2020.02.015. https://www.osti.gov/servlets/purl/1785492.
Copy to clipboard
@article{osti_1785492,
title = {Solving differential equations using deep neural networks},
author = {Michoski, Craig and Milosavljević, Miloš and Oliver, Todd and Hatch, David R.},
abstractNote = {Recent work on solving partial differential equations (PDEs) with deep neural networks (DNNs) is presented. The paper reviews and extends some of these methods while carefully analyzing a fundamental feature in numerical PDEs and nonlinear analysis: irregular solutions. First, the Sod shock tube solution to the compressible Euler equations is discussed and analyzed. This analysis includes a comparison of a DNN-based approach with conventional finite element and finite volume methods, and demonstrates that the DNN is competitive in terms of degrees of freedom required for a given accuracy. Further, the DNN-based approach is extended to consider performance improvements and simultaneous parameter space exploration. Next, a shock solution to compressible magnetohydrodynamics (MHD) is solved for, and used in a scenario where experimental data is utilized to enhance a PDE system that is a priori insufficient to validate against the observed/experimental data. This is accomplished by enriching the model PDE system with source terms that are then inferred via supervised training with synthetic experimental data. The resulting DNN framework for PDEs enables straightforward system prototyping and natural integration of large data sets (be they synthetic or experimental), all while simultaneously enabling single-pass exploration of an entire parameter space.},
doi = {10.1016/j.neucom.2020.02.015},
journal = {Neurocomputing},
number = ,
volume = 399,
place = {United States},
year = {Wed Feb 19 00:00:00 EST 2020},
month = {Wed Feb 19 00:00:00 EST 2020}
}
Copy to clipboard
Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
https://doi.org/10.1016/j.neucom.2020.02.015
Copyright Statement
Other availability
Search WorldCat Search WorldCat to find libraries that may hold this journal
Citation Metrics:
Cited by: 27 works
Citation information provided by
Web of Science
Save / Share:
Export Metadata
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.

Works referenced in this record:

Local bounds preserving stabilization for continuous Galerkin discretization of hyperbolic systems
journal, May 2018

  • Mabuza, Sibusiso; Shadid, John N.; Kuzmin, Dmitri
  • Journal of Computational Physics, Vol. 361
  • DOI: 10.1016/j.jcp.2018.01.048

Data-driven discovery of coordinates and governing equations
journal, October 2019

  • Champion, Kathleen; Lusch, Bethany; Kutz, J. Nathan
  • Proceedings of the National Academy of Sciences, Vol. 116, Issue 45
  • DOI: 10.1073/pnas.1906995116

Adaptive hierarchic transformations for dynamically p-enriched slope-limiting over discontinuous Galerkin systems of generalized equations
journal, September 2011

Local adaptive mesh refinement for shock hydrodynamics
journal, May 1989

Survey of Multifidelity Methods in Uncertainty Propagation, Inference, and Optimization
journal, January 2018

  • Peherstorfer, Benjamin; Willcox, Karen; Gunzburger, Max
  • SIAM Review, Vol. 60, Issue 3
  • DOI: 10.1137/16M1082469

Hierarchical reconstruction for spectral volume method on unstructured grids
journal, September 2009

  • Xu, Zhiliang; Liu, Yingjie; Shu, Chi-Wang
  • Journal of Computational Physics, Vol. 228, Issue 16
  • DOI: 10.1016/j.jcp.2009.05.001

Discrete equations for physical and numerical compressible multiphase mixtures
journal, April 2003

Physics-Driven Regularization of Deep Neural Networks for Enhanced Engineering Design and Analysis
journal, September 2019

  • Nabian, Mohammad Amin; Meidani, Hadi
  • Journal of Computing and Information Science in Engineering, Vol. 20, Issue 1
  • DOI: 10.1115/1.4044507

Hydrodynamic instabilities in supernova remnants - Self-similar driven waves
journal, June 1992

  • Chevalier, Roger A.; Blondin, John M.; Emmering, Robert T.
  • The Astrophysical Journal, Vol. 392
  • DOI: 10.1086/171411

Performance Comparison of HPX Versus Traditional Parallelization Strategies for the Discontinuous Galerkin Method
journal, May 2019

  • Bremer, Maximilian; Kazhyken, Kazbek; Kaiser, Hartmut
  • Journal of Scientific Computing, Vol. 80, Issue 2
  • DOI: 10.1007/s10915-019-00960-z

Management of discontinuous reconstruction in kinetic schemes
journal, June 2004

Artificial neural networks for solving ordinary and partial differential equations
journal, January 1998

  • Lagaris, I. E.; Likas, A.; Fotiadis, D. I.
  • IEEE Transactions on Neural Networks, Vol. 9, Issue 5
  • DOI: 10.1109/72.712178

Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
journal, February 2019

Optimal regularity for the Poisson equation
journal, January 2009

  • Wang, Lihe; Yao, Fengping; Zhou, Shulin
  • Proceedings of the American Mathematical Society, Vol. 137, Issue 06
  • DOI: 10.1090/S0002-9939-09-09805-0

A Centre Manifold Description of Contaminant Dispersion in Channels with Varying Flow Properties
journal, December 1990

  • Mercer, G. N.; Roberts, A. J.
  • SIAM Journal on Applied Mathematics, Vol. 50, Issue 6
  • DOI: 10.1137/0150091

Foundations of the blended isogeometric discontinuous Galerkin (BIDG) method
journal, June 2016

  • Michoski, C.; Chan, J.; Engvall, L.
  • Computer Methods in Applied Mechanics and Engineering, Vol. 305
  • DOI: 10.1016/j.cma.2016.02.015

Validating predictions of unobserved quantities
journal, January 2015

  • Oliver, Todd A.; Terejanu, Gabriel; Simmons, Christopher S.
  • Computer Methods in Applied Mechanics and Engineering, Vol. 283
  • DOI: 10.1016/j.cma.2014.08.023

Data-driven discovery of partial differential equations
journal, April 2017

  • Rudy, Samuel H.; Brunton, Steven L.; Proctor, Joshua L.
  • Science Advances, Vol. 3, Issue 4
  • DOI: 10.1126/sciadv.1602614

Solving high-dimensional partial differential equations using deep learning
journal, August 2018

  • Han, Jiequn; Jentzen, Arnulf; E., Weinan
  • Proceedings of the National Academy of Sciences, Vol. 115, Issue 34
  • DOI: 10.1073/pnas.1718942115

Data-driven discovery of PDEs in complex datasets
journal, May 2019

AD Model Builder: using automatic differentiation for statistical inference of highly parameterized complex nonlinear models
journal, April 2012

  • Fournier, David A.; Skaug, Hans J.; Ancheta, Johnoel
  • Optimization Methods and Software, Vol. 27, Issue 2
  • DOI: 10.1080/10556788.2011.597854

A Comparison of Artificial Viscosity, Limiters, and Filters, for High Order Discontinuous Galerkin Solutions in Nonlinear Settings
journal, April 2015

Theory of shock wave propagation during laser ablation
journal, June 2004

DGM: A deep learning algorithm for solving partial differential equations
journal, December 2018

Wave Propagation Algorithms for Multidimensional Hyperbolic Systems
journal, March 1997

Survey of meshless and generalized finite element methods: A unified approach
journal, May 2003

Novel approach to nonlinear/non-Gaussian Bayesian state estimation
journal, January 1993

  • Gordon, N. J.; Salmond, D. J.; Smith, A. F. M.
  • IEE Proceedings F Radar and Signal Processing, Vol. 140, Issue 2
  • DOI: 10.1049/ip-f-2.1993.0015

Element-free Galerkin methods
journal, January 1994

  • Belytschko, T.; Lu, Y. Y.; Gu, L.
  • International Journal for Numerical Methods in Engineering, Vol. 37, Issue 2
  • DOI: 10.1002/nme.1620370205

Optimization by Simulated Annealing
journal, May 1983

Robust Estimation of a Location Parameter
journal, March 1964

A survey of several finite difference methods for systems of nonlinear hyperbolic conservation laws
journal, April 1978

Spectral element-FCT method for scalar hyperbolic conservation laws
journal, March 1992

  • Giannakouros, John; Karniadakis, George Em
  • International Journal for Numerical Methods in Fluids, Vol. 14, Issue 6
  • DOI: 10.1002/fld.1650140605

Least-squares finite element methods for compressible Euler equations
journal, April 1990

  • Jiang, Bo-Nan; Carey, G. F.
  • International Journal for Numerical Methods in Fluids, Vol. 10, Issue 5
  • DOI: 10.1002/fld.1650100504

Learning partial differential equations via data discovery and sparse optimization
journal, January 2017

  • Schaeffer, Hayden
  • Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 473, Issue 2197
  • DOI: 10.1098/rspa.2016.0446

Quadrature-free non-oscillatory finite volume schemes on unstructured meshes for nonlinear hyperbolic systems
journal, September 2007

  • Dumbser, Michael; Käser, Martin; Titarev, Vladimir A.
  • Journal of Computational Physics, Vol. 226, Issue 1
  • DOI: 10.1016/j.jcp.2007.04.004
    Sort by:
    [ × clear filter / sort ]

    Works referencing / citing this record:

    An overview on deep learning-based approximation methods for partial differential equations
    text, January 2020

    Using analog computers in today's largest computational challenges
    journal, December 2021

      Sort by:
      [ × clear filter / sort ]

      Similar Records in DOE PAGES and OSTI.GOV collections: