skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Data-driven discovery of coordinates and governing equations

Abstract

The discovery of governing equations from scientific data has the potential to transform data-rich fields that lack well-characterized quantitative descriptions. Advances in sparse regression are currently enabling the tractable identification of both the structure and parameters of a nonlinear dynamical system from data. The resulting models have the fewest terms necessary to describe the dynamics, balancing model complexity with descriptive ability, and thus promoting interpretability and generalizability. This provides an algorithmic approach to Occam’s razor for model discovery. However, this approach fundamentally relies on an effective coordinate system in which the dynamics have a simple representation. In this work, we design a custom deep autoencoder network to discover a coordinate transformation into a reduced space where the dynamics may be sparsely represented. Thus, we simultaneously learn the governing equations and the associated coordinate system. We demonstrate this approach on several example high-dimensional systems with low-dimensional behavior. The resulting modeling framework combines the strengths of deep neural networks for flexible representation and sparse identification of nonlinear dynamics (SINDy) for parsimonious models. This method places the discovery of coordinates and models on an equal footing.

Authors:
; ; ;
Publication Date:
Sponsoring Org.:
USDOE
OSTI Identifier:
1571320
Grant/Contract Number:  
AC02-06CH11357
Resource Type:
Published Article
Journal Name:
Proceedings of the National Academy of Sciences of the United States of America
Additional Journal Information:
Journal Name: Proceedings of the National Academy of Sciences of the United States of America Journal Volume: 116 Journal Issue: 45; Journal ID: ISSN 0027-8424
Publisher:
Proceedings of the National Academy of Sciences
Country of Publication:
United States
Language:
English

Citation Formats

Champion, Kathleen, Lusch, Bethany, Kutz, J. Nathan, and Brunton, Steven L. Data-driven discovery of coordinates and governing equations. United States: N. p., 2019. Web. doi:10.1073/pnas.1906995116.
Champion, Kathleen, Lusch, Bethany, Kutz, J. Nathan, & Brunton, Steven L. Data-driven discovery of coordinates and governing equations. United States. doi:10.1073/pnas.1906995116.
Champion, Kathleen, Lusch, Bethany, Kutz, J. Nathan, and Brunton, Steven L. Mon . "Data-driven discovery of coordinates and governing equations". United States. doi:10.1073/pnas.1906995116.
@article{osti_1571320,
title = {Data-driven discovery of coordinates and governing equations},
author = {Champion, Kathleen and Lusch, Bethany and Kutz, J. Nathan and Brunton, Steven L.},
abstractNote = {The discovery of governing equations from scientific data has the potential to transform data-rich fields that lack well-characterized quantitative descriptions. Advances in sparse regression are currently enabling the tractable identification of both the structure and parameters of a nonlinear dynamical system from data. The resulting models have the fewest terms necessary to describe the dynamics, balancing model complexity with descriptive ability, and thus promoting interpretability and generalizability. This provides an algorithmic approach to Occam’s razor for model discovery. However, this approach fundamentally relies on an effective coordinate system in which the dynamics have a simple representation. In this work, we design a custom deep autoencoder network to discover a coordinate transformation into a reduced space where the dynamics may be sparsely represented. Thus, we simultaneously learn the governing equations and the associated coordinate system. We demonstrate this approach on several example high-dimensional systems with low-dimensional behavior. The resulting modeling framework combines the strengths of deep neural networks for flexible representation and sparse identification of nonlinear dynamics (SINDy) for parsimonious models. This method places the discovery of coordinates and models on an equal footing.},
doi = {10.1073/pnas.1906995116},
journal = {Proceedings of the National Academy of Sciences of the United States of America},
number = 45,
volume = 116,
place = {United States},
year = {2019},
month = {10}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
DOI: 10.1073/pnas.1906995116

Save / Share:

Works referenced in this record:

Recovering missing CFD data for high-order discretizations using deep neural networks and dynamics learning
journal, October 2019

  • Carlberg, Kevin T.; Jameson, Antony; Kochenderfer, Mykel J.
  • Journal of Computational Physics, Vol. 395
  • DOI: 10.1016/j.jcp.2019.05.041

Reactive SINDy: Discovering governing reactions from concentration data
journal, January 2019

  • Hoffmann, Moritz; Fröhner, Christoph; Noé, Frank
  • The Journal of Chemical Physics, Vol. 150, Issue 2
  • DOI: 10.1063/1.5066099

Dynamic mode decomposition of numerical and experimental data
journal, July 2010


Sparse structural system identification method for nonlinear dynamic systems with hysteresis/inelastic behavior
journal, February 2019


A Data–Driven Approximation of the Koopman Operator: Extending Dynamic Mode Decomposition
journal, June 2015

  • Williams, Matthew O.; Kevrekidis, Ioannis G.; Rowley, Clarence W.
  • Journal of Nonlinear Science, Vol. 25, Issue 6
  • DOI: 10.1007/s00332-015-9258-5

Distilling Free-Form Natural Laws from Experimental Data
journal, April 2009


On the Convergence of the SINDy Algorithm
journal, January 2019

  • Zhang, Linan; Schaeffer, Hayden
  • Multiscale Modeling & Simulation, Vol. 17, Issue 3
  • DOI: 10.1137/18M1189828

Numerical Differentiation of Noisy, Nonsmooth Data
journal, January 2011


Data-driven operator inference for nonintrusive projection-based model reduction
journal, July 2016

  • Peherstorfer, Benjamin; Willcox, Karen
  • Computer Methods in Applied Mechanics and Engineering, Vol. 306
  • DOI: 10.1016/j.cma.2016.03.025

Model selection for dynamical systems via sparse regression and information criteria
journal, August 2017

  • Mangan, N. M.; Kutz, J. N.; Brunton, S. L.
  • Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 473, Issue 2204
  • DOI: 10.1098/rspa.2017.0009

Multilayer feedforward networks are universal approximators
journal, January 1989


Automated reverse engineering of nonlinear dynamical systems
journal, June 2007

  • Bongard, J.; Lipson, H.
  • Proceedings of the National Academy of Sciences, Vol. 104, Issue 24
  • DOI: 10.1073/pnas.0609476104

Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks
journal, May 2018

  • Vlachas, Pantelis R.; Byeon, Wonmin; Wan, Zhong Y.
  • Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 474, Issue 2213
  • DOI: 10.1098/rspa.2017.0844

Model-Free Prediction of Large Spatiotemporally Chaotic Systems from Data: A Reservoir Computing Approach
journal, January 2018


Extracting Sparse High-Dimensional Dynamics from Limited Data
journal, January 2018

  • Schaeffer, Hayden; Tran, Giang; Ward, Rachel
  • SIAM Journal on Applied Mathematics, Vol. 78, Issue 6
  • DOI: 10.1137/18M116798X

VAMPnets for deep learning of molecular kinetics
journal, January 2018


A Unified Framework for Sparse Relaxed Regularized Regression: SR3
journal, January 2019


Turbulence Modeling in the Age of Data
journal, January 2019


Time-lagged autoencoders: Deep learning of slow collective variables for molecular kinetics
journal, June 2018

  • Wehmeyer, Christoph; Noé, Frank
  • The Journal of Chemical Physics, Vol. 148, Issue 24
  • DOI: 10.1063/1.5011399

Chaos as an intermittently forced linear system
journal, May 2017

  • Brunton, Steven L.; Brunton, Bingni W.; Proctor, Joshua L.
  • Nature Communications, Vol. 8, Issue 1
  • DOI: 10.1038/s41467-017-00030-8

Discovering governing equations from data by sparse identification of nonlinear dynamical systems
journal, March 2016

  • Brunton, Steven L.; Proctor, Joshua L.; Kutz, J. Nathan
  • Proceedings of the National Academy of Sciences, Vol. 113, Issue 15
  • DOI: 10.1073/pnas.1517384113

Identification of distributed parameter systems: A neural net based approach
journal, March 1998


Neural Network Modeling for Near Wall Turbulent Flow
journal, October 2002

  • Milano, Michele; Koumoutsakos, Petros
  • Journal of Computational Physics, Vol. 182, Issue 1
  • DOI: 10.1006/jcph.2002.7146

Data-driven discovery of partial differential equations
journal, April 2017

  • Rudy, Samuel H.; Brunton, Steven L.; Proctor, Joshua L.
  • Science Advances, Vol. 3, Issue 4
  • DOI: 10.1126/sciadv.1602614

Spectral analysis of nonlinear flows
journal, November 2009


Sparse identification for nonlinear optical communication systems: SINO method
journal, January 2016

  • Sorokina, Mariia; Sygletos, Stylianos; Turitsyn, Sergei
  • Optics Express, Vol. 24, Issue 26
  • DOI: 10.1364/OE.24.030433

Sparse model selection via integral terms
journal, August 2017


Constrained sparse Galerkin regression
journal, January 2018

  • Loiseau, Jean-Christophe; Brunton, Steven L.
  • Journal of Fluid Mechanics, Vol. 838
  • DOI: 10.1017/jfm.2017.823

Exact Recovery of Chaotic Systems from Highly Corrupted Data
journal, January 2017

  • Tran, Giang; Ward, Rachel
  • Multiscale Modeling & Simulation, Vol. 15, Issue 3
  • DOI: 10.1137/16M1086637

Sparse dynamics for partial differential equations
journal, March 2013

  • Schaeffer, H.; Caflisch, R.; Hauck, C. D.
  • Proceedings of the National Academy of Sciences, Vol. 110, Issue 17
  • DOI: 10.1073/pnas.1302752110

Neural networks and principal component analysis: Learning from examples without local minima
journal, January 1989


Deep learning for universal linear embeddings of nonlinear dynamics
journal, November 2018


Inferring Biological Networks by Sparse Identification of Nonlinear Dynamics
journal, June 2016

  • Mangan, Niall M.; Brunton, Steven L.; Proctor, Joshua L.
  • IEEE Transactions on Molecular, Biological and Multi-Scale Communications, Vol. 2, Issue 1
  • DOI: 10.1109/TMBMC.2016.2633265

Learning data-driven discretizations for partial differential equations
journal, July 2019

  • Bar-Sinai, Yohai; Hoyer, Stephan; Hickey, Jason
  • Proceedings of the National Academy of Sciences, Vol. 116, Issue 31
  • DOI: 10.1073/pnas.1814058116

Long Short-Term Memory
journal, November 1997


A Survey of Projection-Based Model Reduction Methods for Parametric Dynamical Systems
journal, January 2015

  • Benner, Peter; Gugercin, Serkan; Willcox, Karen
  • SIAM Review, Vol. 57, Issue 4
  • DOI: 10.1137/130932715

Reconstruction of normal forms by learning informed observation geometries from data
journal, August 2017

  • Yair, Or; Talmon, Ronen; Coifman, Ronald R.
  • Proceedings of the National Academy of Sciences, Vol. 114, Issue 38
  • DOI: 10.1073/pnas.1620045114

Spectral Properties of Dynamical Systems, Model Reduction and Decompositions
journal, August 2005


Sparse identification of a predator-prey system from simulation data of a convection model
journal, February 2017

  • Dam, Magnus; Brøns, Morten; Juul Rasmussen, Jens
  • Physics of Plasmas, Vol. 24, Issue 2
  • DOI: 10.1063/1.4977057

Double Sparsity: Learning Sparse Dictionaries for Sparse Signal Approximation
journal, March 2010

  • Rubinstein, R.; Zibulevsky, M.; Elad, M.
  • IEEE Transactions on Signal Processing, Vol. 58, Issue 3
  • DOI: 10.1109/TSP.2009.2036477

Learning partial differential equations via data discovery and sparse optimization
journal, January 2017

  • Schaeffer, Hayden
  • Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 473, Issue 2197
  • DOI: 10.1098/rspa.2016.0446

Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders
journal, November 2019


Deep learning of dynamics and signal-noise decomposition with time-stepping constraints
journal, November 2019

  • Rudy, Samuel H.; Nathan Kutz, J.; Brunton, Steven L.
  • Journal of Computational Physics, Vol. 396
  • DOI: 10.1016/j.jcp.2019.06.056

Sparse identification of nonlinear dynamics for model predictive control in the low-data limit
journal, November 2018

  • Kaiser, E.; Kutz, J. N.; Brunton, S. L.
  • Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 474, Issue 2219
  • DOI: 10.1098/rspa.2018.0335