DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Learning continuous models for continuous physics

Journal Article · · Communications Physics
ORCiD logo [1];  [2]; ORCiD logo [3];  [4]
  1. Univ. of California, Berkeley, CA (United States); Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)
  2. Google Research, Mountain View, CA (United States)
  3. Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States); International Computer Sciences Institute (ICSI), Berkeley, CA (United States)
  4. Univ. of California, Berkeley, CA (United States); Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States); International Computer Sciences Institute (ICSI), Berkeley, CA (United States)

Dynamical systems that evolve continuously over time are ubiquitous throughout science and engineering. Machine learning (ML) provides data-driven approaches to model and predict the dynamics of such systems. A core issue with this approach is that ML models are typically trained on discrete data, using ML methodologies that are not aware of underlying continuity properties. This results in models that often do not capture any underlying continuous dynamics—either of the system of interest, or indeed of any related system. To address this challenge, we develop a convergence test based on numerical analysis theory. Our test verifies whether a model has learned a function that accurately approximates an underlying continuous dynamics. Models that fail this test fail to capture relevant dynamics, rendering them of limited utility for many scientific prediction tasks; while models that pass this test enable both better interpolation and better extrapolation in multiple ways. Our results illustrate how principled numerical analysis methods can be coupled with existing ML training/testing methodologies to validate models for science and engineering applications.

Research Organization:
Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)
Sponsoring Organization:
USDOE Laboratory Directed Research and Development (LDRD) Program; National Science Foundation (NSF)
Grant/Contract Number:
AC02-05CH11231; DMS-2319621
OSTI ID:
2204781
Alternate ID(s):
OSTI ID: 2367186
Journal Information:
Communications Physics, Vol. 6, Issue 1; ISSN 2399-3650
Publisher:
Springer NatureCopyright Statement
Country of Publication:
United States
Language:
English

References (29)

Discrete Dynamical Systems book March 2012
DISCRETE- vs. CONTINUOUS-TIME NONLINEAR SIGNAL PROCESSING OF Cu ELECTRODISSOLUTION DATA journal November 1992
Physics-informed machine learning journal May 2021
Heavy-Tailed Universality Predicts Trends in Test Accuracies for Very Large Pre-Trained Deep Neural Networks book January 2020
Data-Driven Science and Engineering book February 2019
Convergence and stability in the numerical integration of ordinary differential equations journal December 1956
Fundamentals of Engineering Numerical Analysis book January 2010
Discovering governing equations from data by sparse identification of nonlinear dynamical systems journal March 2016
Statistical dynamical systems for skills acquisition in humanoids conference November 2012
SympNets: Intrinsic structure-preserving symplectic networks for identifying Hamiltonian systems journal December 2020
Stability, Consistency, and Convergence of Numerical Discretizations book January 2015
Data-driven predictions of the Lorenz system journal July 2020
The Need for Verifiable Visualization journal September 2008
Knowledge-based learning of nonlinear dynamics and chaos journal November 2021
PySINDy: A comprehensive Python package for robust sparse system identification journal January 2022
PySINDy: A Python package for the sparse identification of nonlinear dynamical systems from data journal May 2020
A review on data-driven linear parameter-varying modeling approaches: A high-purity distillation column case study journal April 2014
Predicting trends in the quality of state-of-the-art neural networks without access to training or testing data journal July 2021
Deep Neural Networks Motivated by Partial Differential Equations journal September 2019
Numerical Methods for Conservation Laws book January 1992
Deep Variational Koopman Models: Inferring Koopman Observations for Uncertainty-Aware Dynamics Modeling and Control conference August 2019
Learning data-driven discretizations for partial differential equations journal July 2019
Towards Machine Learning of Motor Skills book January 2007
Discovery of Dynamics Using Linear Multistep Methods journal January 2021
QRnet: Optimal Regulator Design With LQR-Augmented Neural Networks journal October 2021
Lectures on Finite Precision Computations book January 1996
Linearly Recurrent Autoencoder Networks for Learning Dynamics journal January 2019
Overcoming the curse of dimensionality for some Hamilton–Jacobi partial differential equations via neural network architectures journal July 2020
A Proposal on Machine Learning via Dynamical Systems journal March 2017