skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Equation Discovery Using Fast Function Extraction: a Deterministic Symbolic Regression Approach

Abstract

Advances in machine learning (ML) coupled with increased computational power have enabled identification of patterns in data extracted from complex systems. ML algorithms are actively being sought in recovering physical models or mathematical equations from data. This is a highly valuable technique where models cannot be built using physical reasoning alone. In this paper, we investigate the application of fast function extraction (FFX), a fast, scalable, deterministic symbolic regression algorithm to recover partial differential equations (PDEs). FFX identifies active bases among a huge set of candidate basis functions and their corresponding coefficients from recorded snapshot data. This approach uses a sparsity-promoting technique from compressive sensing and sparse optimization called pathwise regularized learning to perform feature selection and parameter estimation. Furthermore, it recovers several models of varying complexity (number of basis terms). FFX finally filters out many identified models using non-dominated sorting and forms a Pareto front consisting of optimal models with respect to minimizing complexity and test accuracy. Numerical experiments are carried out to recover several ubiquitous PDEs such as wave and heat equations among linear PDEs and Burgers, Korteweg–de Vries (KdV), and Kawahara equations among higher-order nonlinear PDEs. Additional simulations are conducted on the same PDEs under noisy conditionsmore » to test the robustness of the proposed approach.« less

Authors:
 [1]; ORCiD logo [1]
  1. Oklahoma State Univ., Stillwater, OK (United States)
Publication Date:
Research Org.:
Oklahoma State Univ., Stillwater, OK (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)
OSTI Identifier:
1593572
Grant/Contract Number:  
SC0019290
Resource Type:
Accepted Manuscript
Journal Name:
Fluids
Additional Journal Information:
Journal Volume: 4; Journal Issue: 2; Journal ID: ISSN 2311-5521
Country of Publication:
United States
Language:
English
Subject:
deterministic symbolic regression; fast function extraction; compressive sensing; pathwise regularized learning; non-dominated sorting

Citation Formats

Vaddireddy, Harsha, and San, Omer. Equation Discovery Using Fast Function Extraction: a Deterministic Symbolic Regression Approach. United States: N. p., 2019. Web. doi:10.3390/fluids4020111.
Vaddireddy, Harsha, & San, Omer. Equation Discovery Using Fast Function Extraction: a Deterministic Symbolic Regression Approach. United States. doi:10.3390/fluids4020111.
Vaddireddy, Harsha, and San, Omer. Sat . "Equation Discovery Using Fast Function Extraction: a Deterministic Symbolic Regression Approach". United States. doi:10.3390/fluids4020111. https://www.osti.gov/servlets/purl/1593572.
@article{osti_1593572,
title = {Equation Discovery Using Fast Function Extraction: a Deterministic Symbolic Regression Approach},
author = {Vaddireddy, Harsha and San, Omer},
abstractNote = {Advances in machine learning (ML) coupled with increased computational power have enabled identification of patterns in data extracted from complex systems. ML algorithms are actively being sought in recovering physical models or mathematical equations from data. This is a highly valuable technique where models cannot be built using physical reasoning alone. In this paper, we investigate the application of fast function extraction (FFX), a fast, scalable, deterministic symbolic regression algorithm to recover partial differential equations (PDEs). FFX identifies active bases among a huge set of candidate basis functions and their corresponding coefficients from recorded snapshot data. This approach uses a sparsity-promoting technique from compressive sensing and sparse optimization called pathwise regularized learning to perform feature selection and parameter estimation. Furthermore, it recovers several models of varying complexity (number of basis terms). FFX finally filters out many identified models using non-dominated sorting and forms a Pareto front consisting of optimal models with respect to minimizing complexity and test accuracy. Numerical experiments are carried out to recover several ubiquitous PDEs such as wave and heat equations among linear PDEs and Burgers, Korteweg–de Vries (KdV), and Kawahara equations among higher-order nonlinear PDEs. Additional simulations are conducted on the same PDEs under noisy conditions to test the robustness of the proposed approach.},
doi = {10.3390/fluids4020111},
journal = {Fluids},
number = 2,
volume = 4,
place = {United States},
year = {2019},
month = {6}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Save / Share:

Works referenced in this record:

Stable signal recovery from incomplete and inaccurate measurements
journal, January 2006

  • Candès, Emmanuel J.; Romberg, Justin K.; Tao, Terence
  • Communications on Pure and Applied Mathematics, Vol. 59, Issue 8, p. 1207-1223
  • DOI: 10.1002/cpa.20124

Application of an evolutionary algorithm to LES modelling of turbulent transport in premixed flames
journal, December 2018

  • Schoepplein, Matthias; Weatheritt, Jack; Sandberg, Richard
  • Journal of Computational Physics, Vol. 374
  • DOI: 10.1016/j.jcp.2018.08.016

Deep learning
journal, May 2015

  • LeCun, Yann; Bengio, Yoshua; Hinton, Geoffrey
  • Nature, Vol. 521, Issue 7553
  • DOI: 10.1038/nature14539

Distilling Free-Form Natural Laws from Experimental Data
journal, April 2009


Force identification of dynamic systems using genetic programming
journal, January 2005

  • Yang, Y. W.; Wang, C.; Soh, C. K.
  • International Journal for Numerical Methods in Engineering, Vol. 63, Issue 9
  • DOI: 10.1002/nme.1323

Closed-Loop Turbulence Control: Progress and Challenges
journal, August 2015

  • Brunton, Steven L.; Noack, Bernd R.
  • Applied Mechanics Reviews, Vol. 67, Issue 5
  • DOI: 10.1115/1.4031175

Model selection for dynamical systems via sparse regression and information criteria
journal, August 2017

  • Mangan, N. M.; Kutz, J. N.; Brunton, S. L.
  • Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 473, Issue 2204
  • DOI: 10.1098/rspa.2017.0009

The application of robust Bayesian analysis to hypothesis testing and Occam's Razor
journal, February 1992

  • Berger, James O.; Jefferys, William H.
  • Journal of the Italian Statistical Society, Vol. 1, Issue 1
  • DOI: 10.1007/BF02589047

Extracting Sparse High-Dimensional Dynamics from Limited Data
journal, January 2018

  • Schaeffer, Hayden; Tran, Giang; Ward, Rachel
  • SIAM Journal on Applied Mathematics, Vol. 78, Issue 6
  • DOI: 10.1137/18M116798X

The perceptron: A probabilistic model for information storage and organization in the brain.
journal, January 1958


An Introduction To Compressive Sampling
journal, March 2008


Discovering governing equations from data by sparse identification of nonlinear dynamical systems
journal, March 2016

  • Brunton, Steven L.; Proctor, Joshua L.; Kutz, J. Nathan
  • Proceedings of the National Academy of Sciences, Vol. 113, Issue 15
  • DOI: 10.1073/pnas.1517384113

Compressed sensing
journal, April 2006


A novel evolutionary algorithm applied to algebraic modifications of the RANS stress–strain relationship
journal, November 2016


Regularization and variable selection via the elastic net
journal, April 2005


The big challenges of big data
journal, June 2013


Data-driven discovery of partial differential equations
journal, April 2017

  • Rudy, Samuel H.; Brunton, Steven L.; Proctor, Joshua L.
  • Science Advances, Vol. 3, Issue 4
  • DOI: 10.1126/sciadv.1602614

Parse-matrix evolution for symbolic regression
journal, September 2012


Extreme Learning Machines as Encoders for Sparse Reconstruction
journal, November 2018


Exact Recovery of Chaotic Systems from Highly Corrupted Data
journal, January 2017

  • Tran, Giang; Ward, Rachel
  • Multiscale Modeling & Simulation, Vol. 15, Issue 3
  • DOI: 10.1137/16M1086637

Regularization Paths for Generalized Linear Models via Coordinate Descent
journal, January 2010

  • Friedman, Jerome; Hastie, Trevor; Tibshirani, Robert
  • Journal of Statistical Software, Vol. 33, Issue 1
  • DOI: 10.18637/jss.v033.i01

Sparse dynamics for partial differential equations
journal, March 2013

  • Schaeffer, H.; Caflisch, R.; Hauck, C. D.
  • Proceedings of the National Academy of Sciences, Vol. 110, Issue 17
  • DOI: 10.1073/pnas.1302752110

Multi-objective genetic programming for nonlinear system identification
journal, January 1998

  • Rodríguez-Vázquez, K.; Fleming, P. J.
  • Electronics Letters, Vol. 34, Issue 9
  • DOI: 10.1049/el:19980632

Machine learning: Trends, perspectives, and prospects
journal, July 2015


Occam's Razor
journal, April 1987


Generalized Linear Models
journal, January 1972

  • Nelder, J. A.; Wedderburn, R. W. M.
  • Journal of the Royal Statistical Society. Series A (General), Vol. 135, Issue 3
  • DOI: 10.2307/2344614

Closed-loop separation control using machine learning
journal, April 2015

  • Gautier, N.; Aider, J. -L.; Duriez, T.
  • Journal of Fluid Mechanics, Vol. 770
  • DOI: 10.1017/jfm.2015.95

Learning partial differential equations via data discovery and sparse optimization
journal, January 2017

  • Schaeffer, Hayden
  • Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 473, Issue 2197
  • DOI: 10.1098/rspa.2016.0446

A fast and elitist multiobjective genetic algorithm: NSGA-II
journal, April 2002

  • Deb, K.; Pratap, A.; Agarwal, S.
  • IEEE Transactions on Evolutionary Computation, Vol. 6, Issue 2
  • DOI: 10.1109/4235.996017

Regression Shrinkage and Selection Via the Lasso
journal, January 1996


Existence of perturbed solitary wave solutions to a model equation for water waves
journal, September 1988