Data-driven discovery of coordinates and governing equations
Abstract
The discovery of governing equations from scientific data has the potential to transform data-rich fields that lack well-characterized quantitative descriptions. Advances in sparse regression are currently enabling the tractable identification of both the structure and parameters of a nonlinear dynamical system from data. The resulting models have the fewest terms necessary to describe the dynamics, balancing model complexity with descriptive ability, and thus promoting interpretability and generalizability. This provides an algorithmic approach to Occam’s razor for model discovery. However, this approach fundamentally relies on an effective coordinate system in which the dynamics have a simple representation. In this work, we design a custom deep autoencoder network to discover a coordinate transformation into a reduced space where the dynamics may be sparsely represented. Thus, we simultaneously learn the governing equations and the associated coordinate system. We demonstrate this approach on several example high-dimensional systems with low-dimensional behavior. The resulting modeling framework combines the strengths of deep neural networks for flexible representation and sparse identification of nonlinear dynamics (SINDy) for parsimonious models. This method places the discovery of coordinates and models on an equal footing.
- Authors:
- Publication Date:
- Research Org.:
- Argonne National Lab. (ANL), Argonne, IL (United States)
- Sponsoring Org.:
- USDOD; Defense Advanced Research Projects Agency (DARPA); National Science Foundation (NSF); US Army Research Office (ARO); USDOE Office of Science (SC)
- OSTI Identifier:
- 1571320
- Alternate Identifier(s):
- OSTI ID: 1596679
- Grant/Contract Number:
- AC02-06CH11357; DGE-1256082
- Resource Type:
- Published Article
- Journal Name:
- Proceedings of the National Academy of Sciences of the United States of America
- Additional Journal Information:
- Journal Name: Proceedings of the National Academy of Sciences of the United States of America Journal Volume: 116 Journal Issue: 45; Journal ID: ISSN 0027-8424
- Publisher:
- National Academy of Sciences
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 97 MATHEMATICS AND COMPUTING; dynamical systems; deep learning; machine learning; model discovery
Citation Formats
Champion, Kathleen, Lusch, Bethany, Kutz, J. Nathan, and Brunton, Steven L. Data-driven discovery of coordinates and governing equations. United States: N. p., 2019.
Web. doi:10.1073/pnas.1906995116.
Champion, Kathleen, Lusch, Bethany, Kutz, J. Nathan, & Brunton, Steven L. Data-driven discovery of coordinates and governing equations. United States. doi:10.1073/pnas.1906995116.
Champion, Kathleen, Lusch, Bethany, Kutz, J. Nathan, and Brunton, Steven L. Mon .
"Data-driven discovery of coordinates and governing equations". United States. doi:10.1073/pnas.1906995116.
@article{osti_1571320,
title = {Data-driven discovery of coordinates and governing equations},
author = {Champion, Kathleen and Lusch, Bethany and Kutz, J. Nathan and Brunton, Steven L.},
abstractNote = {The discovery of governing equations from scientific data has the potential to transform data-rich fields that lack well-characterized quantitative descriptions. Advances in sparse regression are currently enabling the tractable identification of both the structure and parameters of a nonlinear dynamical system from data. The resulting models have the fewest terms necessary to describe the dynamics, balancing model complexity with descriptive ability, and thus promoting interpretability and generalizability. This provides an algorithmic approach to Occam’s razor for model discovery. However, this approach fundamentally relies on an effective coordinate system in which the dynamics have a simple representation. In this work, we design a custom deep autoencoder network to discover a coordinate transformation into a reduced space where the dynamics may be sparsely represented. Thus, we simultaneously learn the governing equations and the associated coordinate system. We demonstrate this approach on several example high-dimensional systems with low-dimensional behavior. The resulting modeling framework combines the strengths of deep neural networks for flexible representation and sparse identification of nonlinear dynamics (SINDy) for parsimonious models. This method places the discovery of coordinates and models on an equal footing.},
doi = {10.1073/pnas.1906995116},
journal = {Proceedings of the National Academy of Sciences of the United States of America},
number = 45,
volume = 116,
place = {United States},
year = {2019},
month = {10}
}
Free Publicly Available Full Text
Publisher's Version of Record
DOI: 10.1073/pnas.1906995116
DOI: 10.1073/pnas.1906995116
Other availability
Search WorldCat to find libraries that may hold this journal
Cited by: 4 works
Citation information provided by
Web of Science
Web of Science
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
Recovering missing CFD data for high-order discretizations using deep neural networks and dynamics learning
journal, October 2019
- Carlberg, Kevin T.; Jameson, Antony; Kochenderfer, Mykel J.
- Journal of Computational Physics, Vol. 395
Reactive SINDy: Discovering governing reactions from concentration data
journal, January 2019
- Hoffmann, Moritz; Fröhner, Christoph; Noé, Frank
- The Journal of Chemical Physics, Vol. 150, Issue 2
Dynamic mode decomposition of numerical and experimental data
journal, July 2010
- Schmid, Peter J.
- Journal of Fluid Mechanics, Vol. 656
Sparse structural system identification method for nonlinear dynamic systems with hysteresis/inelastic behavior
journal, February 2019
- Lai, Zhilu; Nagarajaiah, Satish
- Mechanical Systems and Signal Processing, Vol. 117
A Data–Driven Approximation of the Koopman Operator: Extending Dynamic Mode Decomposition
journal, June 2015
- Williams, Matthew O.; Kevrekidis, Ioannis G.; Rowley, Clarence W.
- Journal of Nonlinear Science, Vol. 25, Issue 6
Kernel dictionary learning
conference, March 2012
- Nguyen, Hien Van; Patel, Vishal M.; Nasrabadi, Nasser M.
- ICASSP 2012 - 2012 IEEE International Conference on Acoustics, Speech and Signal Processing, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Distilling Free-Form Natural Laws from Experimental Data
journal, April 2009
- Schmidt, Michael; Lipson, Hod
- Science, Vol. 324, Issue 5923
On the Convergence of the SINDy Algorithm
journal, January 2019
- Zhang, Linan; Schaeffer, Hayden
- Multiscale Modeling & Simulation, Vol. 17, Issue 3
Numerical Differentiation of Noisy, Nonsmooth Data
journal, January 2011
- Chartrand, Rick
- ISRN Applied Mathematics, Vol. 2011
Data-driven operator inference for nonintrusive projection-based model reduction
journal, July 2016
- Peherstorfer, Benjamin; Willcox, Karen
- Computer Methods in Applied Mechanics and Engineering, Vol. 306
Model selection for dynamical systems via sparse regression and information criteria
journal, August 2017
- Mangan, N. M.; Kutz, J. N.; Brunton, S. L.
- Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 473, Issue 2204
Multilayer feedforward networks are universal approximators
journal, January 1989
- Hornik, Kurt; Stinchcombe, Maxwell; White, Halbert
- Neural Networks, Vol. 2, Issue 5
Automated reverse engineering of nonlinear dynamical systems
journal, June 2007
- Bongard, J.; Lipson, H.
- Proceedings of the National Academy of Sciences, Vol. 104, Issue 24
Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks
journal, May 2018
- Vlachas, Pantelis R.; Byeon, Wonmin; Wan, Zhong Y.
- Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 474, Issue 2213
Model-Free Prediction of Large Spatiotemporally Chaotic Systems from Data: A Reservoir Computing Approach
journal, January 2018
- Pathak, Jaideep; Hunt, Brian; Girvan, Michelle
- Physical Review Letters, Vol. 120, Issue 2
Extracting Sparse High-Dimensional Dynamics from Limited Data
journal, January 2018
- Schaeffer, Hayden; Tran, Giang; Ward, Rachel
- SIAM Journal on Applied Mathematics, Vol. 78, Issue 6
VAMPnets for deep learning of molecular kinetics
journal, January 2018
- Mardt, Andreas; Pasquali, Luca; Wu, Hao
- Nature Communications, Vol. 9, Issue 1
A Unified Framework for Sparse Relaxed Regularized Regression: SR3
journal, January 2019
- Zheng, Peng; Askham, Travis; Brunton, Steven L.
- IEEE Access, Vol. 7
Turbulence Modeling in the Age of Data
journal, January 2019
- Duraisamy, Karthik; Iaccarino, Gianluca; Xiao, Heng
- Annual Review of Fluid Mechanics, Vol. 51, Issue 1
Time-lagged autoencoders: Deep learning of slow collective variables for molecular kinetics
journal, June 2018
- Wehmeyer, Christoph; Noé, Frank
- The Journal of Chemical Physics, Vol. 148, Issue 24
Chaos as an intermittently forced linear system
journal, May 2017
- Brunton, Steven L.; Brunton, Bingni W.; Proctor, Joshua L.
- Nature Communications, Vol. 8, Issue 1
Discovering governing equations from data by sparse identification of nonlinear dynamical systems
journal, March 2016
- Brunton, Steven L.; Proctor, Joshua L.; Kutz, J. Nathan
- Proceedings of the National Academy of Sciences, Vol. 113, Issue 15
Learning Deep Neural Network Representations for Koopman Operators of Nonlinear Dynamical Systems
conference, July 2019
- Yeung, Enoch; Kundu, Soumya; Hodas, Nathan
- 2019 American Control Conference (ACC)
Identification of distributed parameter systems: A neural net based approach
journal, March 1998
- González-García, R.; Rico-Martínez, R.; Kevrekidis, I. G.
- Computers & Chemical Engineering, Vol. 22
Neural Network Modeling for Near Wall Turbulent Flow
journal, October 2002
- Milano, Michele; Koumoutsakos, Petros
- Journal of Computational Physics, Vol. 182, Issue 1
Data-driven discovery of partial differential equations
journal, April 2017
- Rudy, Samuel H.; Brunton, Steven L.; Proctor, Joshua L.
- Science Advances, Vol. 3, Issue 4
Spectral analysis of nonlinear flows
journal, November 2009
- Rowley, Clarence W.; MeziĆ, Igor; Bagheri, Shervin
- Journal of Fluid Mechanics, Vol. 641
Sparse identification for nonlinear optical communication systems: SINO method
journal, January 2016
- Sorokina, Mariia; Sygletos, Stylianos; Turitsyn, Sergei
- Optics Express, Vol. 24, Issue 26
Sparse model selection via integral terms
journal, August 2017
- Schaeffer, Hayden; McCalla, Scott G.
- Physical Review E, Vol. 96, Issue 2
Constrained sparse Galerkin regression
journal, January 2018
- Loiseau, Jean-Christophe; Brunton, Steven L.
- Journal of Fluid Mechanics, Vol. 838
Exact Recovery of Chaotic Systems from Highly Corrupted Data
journal, January 2017
- Tran, Giang; Ward, Rachel
- Multiscale Modeling & Simulation, Vol. 15, Issue 3
Sparse dynamics for partial differential equations
journal, March 2013
- Schaeffer, H.; Caflisch, R.; Hauck, C. D.
- Proceedings of the National Academy of Sciences, Vol. 110, Issue 17
Neural networks and principal component analysis: Learning from examples without local minima
journal, January 1989
- Baldi, Pierre; Hornik, Kurt
- Neural Networks, Vol. 2, Issue 1
Deep learning for universal linear embeddings of nonlinear dynamics
journal, November 2018
- Lusch, Bethany; Kutz, J. Nathan; Brunton, Steven L.
- Nature Communications, Vol. 9, Issue 1
Inferring Biological Networks by Sparse Identification of Nonlinear Dynamics
journal, June 2016
- Mangan, Niall M.; Brunton, Steven L.; Proctor, Joshua L.
- IEEE Transactions on Molecular, Biological and Multi-Scale Communications, Vol. 2, Issue 1
Learning data-driven discretizations for partial differential equations
journal, July 2019
- Bar-Sinai, Yohai; Hoyer, Stephan; Hickey, Jason
- Proceedings of the National Academy of Sciences, Vol. 116, Issue 31
Long Short-Term Memory
journal, November 1997
- Hochreiter, Sepp; Schmidhuber, Jürgen
- Neural Computation, Vol. 9, Issue 8
Modeling and nonlinear parameter estimation with Kronecker product representation for coupled oscillators and spatiotemporal systems
journal, March 2007
- Yao, Chen; Bollt, Erik M.
- Physica D: Nonlinear Phenomena, Vol. 227, Issue 1
A Survey of Projection-Based Model Reduction Methods for Parametric Dynamical Systems
journal, January 2015
- Benner, Peter; Gugercin, Serkan; Willcox, Karen
- SIAM Review, Vol. 57, Issue 4
Reconstruction of normal forms by learning informed observation geometries from data
journal, August 2017
- Yair, Or; Talmon, Ronen; Coifman, Ronald R.
- Proceedings of the National Academy of Sciences, Vol. 114, Issue 38
Spectral Properties of Dynamical Systems, Model Reduction and Decompositions
journal, August 2005
- Mezić, Igor
- Nonlinear Dynamics, Vol. 41, Issue 1-3
Sparse identification of a predator-prey system from simulation data of a convection model
journal, February 2017
- Dam, Magnus; Brøns, Morten; Juul Rasmussen, Jens
- Physics of Plasmas, Vol. 24, Issue 2
Double Sparsity: Learning Sparse Dictionaries for Sparse Signal Approximation
journal, March 2010
- Rubinstein, R.; Zibulevsky, M.; Elad, M.
- IEEE Transactions on Signal Processing, Vol. 58, Issue 3
Learning partial differential equations via data discovery and sparse optimization
journal, January 2017
- Schaeffer, Hayden
- Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 473, Issue 2197
Deep learning of dynamics and signal-noise decomposition with time-stepping constraints
journal, November 2019
- Rudy, Samuel H.; Nathan Kutz, J.; Brunton, Steven L.
- Journal of Computational Physics, Vol. 396
Sparse identification of nonlinear dynamics for model predictive control in the low-data limit
journal, November 2018
- Kaiser, E.; Kutz, J. N.; Brunton, S. L.
- Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 474, Issue 2219