Equation Discovery Using Fast Function Extraction: a Deterministic Symbolic Regression Approach
Abstract
Advances in machine learning (ML) coupled with increased computational power have enabled identification of patterns in data extracted from complex systems. ML algorithms are actively being sought in recovering physical models or mathematical equations from data. This is a highly valuable technique where models cannot be built using physical reasoning alone. In this paper, we investigate the application of fast function extraction (FFX), a fast, scalable, deterministic symbolic regression algorithm to recover partial differential equations (PDEs). FFX identifies active bases among a huge set of candidate basis functions and their corresponding coefficients from recorded snapshot data. This approach uses a sparsity-promoting technique from compressive sensing and sparse optimization called pathwise regularized learning to perform feature selection and parameter estimation. Furthermore, it recovers several models of varying complexity (number of basis terms). FFX finally filters out many identified models using non-dominated sorting and forms a Pareto front consisting of optimal models with respect to minimizing complexity and test accuracy. Numerical experiments are carried out to recover several ubiquitous PDEs such as wave and heat equations among linear PDEs and Burgers, Korteweg–de Vries (KdV), and Kawahara equations among higher-order nonlinear PDEs. Additional simulations are conducted on the same PDEs under noisy conditionsmore »
- Authors:
-
- Oklahoma State Univ., Stillwater, OK (United States)
- Publication Date:
- Research Org.:
- Oklahoma State Univ., Stillwater, OK (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
- OSTI Identifier:
- 1593572
- Grant/Contract Number:
- SC0019290
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Fluids
- Additional Journal Information:
- Journal Volume: 4; Journal Issue: 2; Journal ID: ISSN 2311-5521
- Publisher:
- MDPI
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 42 ENGINEERING; deterministic symbolic regression; fast function extraction; compressive sensing; pathwise regularized learning; non-dominated sorting
Citation Formats
Vaddireddy, Harsha, and San, Omer. Equation Discovery Using Fast Function Extraction: a Deterministic Symbolic Regression Approach. United States: N. p., 2019.
Web. doi:10.3390/fluids4020111.
Vaddireddy, Harsha, & San, Omer. Equation Discovery Using Fast Function Extraction: a Deterministic Symbolic Regression Approach. United States. https://doi.org/10.3390/fluids4020111
Vaddireddy, Harsha, and San, Omer. Sat .
"Equation Discovery Using Fast Function Extraction: a Deterministic Symbolic Regression Approach". United States. https://doi.org/10.3390/fluids4020111. https://www.osti.gov/servlets/purl/1593572.
@article{osti_1593572,
title = {Equation Discovery Using Fast Function Extraction: a Deterministic Symbolic Regression Approach},
author = {Vaddireddy, Harsha and San, Omer},
abstractNote = {Advances in machine learning (ML) coupled with increased computational power have enabled identification of patterns in data extracted from complex systems. ML algorithms are actively being sought in recovering physical models or mathematical equations from data. This is a highly valuable technique where models cannot be built using physical reasoning alone. In this paper, we investigate the application of fast function extraction (FFX), a fast, scalable, deterministic symbolic regression algorithm to recover partial differential equations (PDEs). FFX identifies active bases among a huge set of candidate basis functions and their corresponding coefficients from recorded snapshot data. This approach uses a sparsity-promoting technique from compressive sensing and sparse optimization called pathwise regularized learning to perform feature selection and parameter estimation. Furthermore, it recovers several models of varying complexity (number of basis terms). FFX finally filters out many identified models using non-dominated sorting and forms a Pareto front consisting of optimal models with respect to minimizing complexity and test accuracy. Numerical experiments are carried out to recover several ubiquitous PDEs such as wave and heat equations among linear PDEs and Burgers, Korteweg–de Vries (KdV), and Kawahara equations among higher-order nonlinear PDEs. Additional simulations are conducted on the same PDEs under noisy conditions to test the robustness of the proposed approach.},
doi = {10.3390/fluids4020111},
journal = {Fluids},
number = 2,
volume = 4,
place = {United States},
year = {2019},
month = {6}
}
Works referenced in this record:
Stable signal recovery from incomplete and inaccurate measurements
journal, January 2006
- Candès, Emmanuel J.; Romberg, Justin K.; Tao, Terence
- Communications on Pure and Applied Mathematics, Vol. 59, Issue 8, p. 1207-1223
Application of an evolutionary algorithm to LES modelling of turbulent transport in premixed flames
journal, December 2018
- Schoepplein, Matthias; Weatheritt, Jack; Sandberg, Richard
- Journal of Computational Physics, Vol. 374
Deep learning
journal, May 2015
- LeCun, Yann; Bengio, Yoshua; Hinton, Geoffrey
- Nature, Vol. 521, Issue 7553
Distilling Free-Form Natural Laws from Experimental Data
journal, April 2009
- Schmidt, Michael; Lipson, Hod
- Science, Vol. 324, Issue 5923
Force identification of dynamic systems using genetic programming
journal, January 2005
- Yang, Y. W.; Wang, C.; Soh, C. K.
- International Journal for Numerical Methods in Engineering, Vol. 63, Issue 9
Closed-Loop Turbulence Control: Progress and Challenges
journal, August 2015
- Brunton, Steven L.; Noack, Bernd R.
- Applied Mechanics Reviews, Vol. 67, Issue 5
Model selection for dynamical systems via sparse regression and information criteria
journal, August 2017
- Mangan, N. M.; Kutz, J. N.; Brunton, S. L.
- Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 473, Issue 2204
Oscillatory Solitary Waves in Dispersive Media
journal, July 1972
- Kawahara, Takuji
- Journal of the Physical Society of Japan, Vol. 33, Issue 1
The application of robust Bayesian analysis to hypothesis testing and Occam's Razor
journal, February 1992
- Berger, James O.; Jefferys, William H.
- Journal of the Italian Statistical Society, Vol. 1, Issue 1
Extracting Sparse High-Dimensional Dynamics from Limited Data
journal, January 2018
- Schaeffer, Hayden; Tran, Giang; Ward, Rachel
- SIAM Journal on Applied Mathematics, Vol. 78, Issue 6
The perceptron: A probabilistic model for information storage and organization in the brain.
journal, January 1958
- Rosenblatt, F.
- Psychological Review, Vol. 65, Issue 6
An Introduction To Compressive Sampling
journal, March 2008
- Candes, E. J.; Wakin, M. B.
- IEEE Signal Processing Magazine, Vol. 25, Issue 2
Discovering governing equations from data by sparse identification of nonlinear dynamical systems
journal, March 2016
- Brunton, Steven L.; Proctor, Joshua L.; Kutz, J. Nathan
- Proceedings of the National Academy of Sciences, Vol. 113, Issue 15
Compressed sensing
journal, April 2006
- Donoho, D. L.
- IEEE Transactions on Information Theory, Vol. 52, Issue 4
A novel evolutionary algorithm applied to algebraic modifications of the RANS stress–strain relationship
journal, November 2016
- Weatheritt, Jack; Sandberg, Richard
- Journal of Computational Physics, Vol. 325
Deep Reinforcement Learning framework for Autonomous Driving
journal, January 2017
- Sallab, AhmadEL; Abdou, Mohammed; Perot, Etienne
- Electronic Imaging, Vol. 2017, Issue 19
Regularization and variable selection via the elastic net
journal, April 2005
- Zou, Hui; Hastie, Trevor
- Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 67, Issue 2
Data-driven discovery of partial differential equations
journal, April 2017
- Rudy, Samuel H.; Brunton, Steven L.; Proctor, Joshua L.
- Science Advances, Vol. 3, Issue 4
Prediction of dynamical systems by symbolic regression
journal, July 2016
- Quade, Markus; Abel, Markus; Shafi, Kamran
- Physical Review E, Vol. 94, Issue 1
New exact travelling wave solutions for the Kawahara and modified Kawahara equations
journal, January 2004
- Sirendaoreji,
- Chaos, Solitons & Fractals, Vol. 19, Issue 1
Parse-matrix evolution for symbolic regression
journal, September 2012
- Luo, Changtong; Zhang, Shao-Liang
- Engineering Applications of Artificial Intelligence, Vol. 25, Issue 6
Extreme Learning Machines as Encoders for Sparse Reconstruction
journal, November 2018
- Al Mamun, S.; Lu, Chen; Jayaraman, Balaji
- Fluids, Vol. 3, Issue 4
Exact Recovery of Chaotic Systems from Highly Corrupted Data
journal, January 2017
- Tran, Giang; Ward, Rachel
- Multiscale Modeling & Simulation, Vol. 15, Issue 3
Regularization Paths for Generalized Linear Models via Coordinate Descent
journal, January 2010
- Friedman, Jerome; Hastie, Trevor; Tibshirani, Robert
- Journal of Statistical Software, Vol. 33, Issue 1
Sparse dynamics for partial differential equations
journal, March 2013
- Schaeffer, H.; Caflisch, R.; Hauck, C. D.
- Proceedings of the National Academy of Sciences, Vol. 110, Issue 17
Nonlinear Interaction between Short and Long Capillary-Gravity Waves
journal, November 1975
- Kawahara, Takuji; Sugimoto, Nobumasa; Kakutani, Tsunehiko
- Journal of the Physical Society of Japan, Vol. 39, Issue 5
Multi-objective genetic programming for nonlinear system identification
journal, January 1998
- Rodríguez-Vázquez, K.; Fleming, P. J.
- Electronics Letters, Vol. 34, Issue 9
Machine learning: Trends, perspectives, and prospects
journal, July 2015
- Jordan, M. I.; Mitchell, T. M.
- Science, Vol. 349, Issue 6245
Occam's Razor
journal, April 1987
- Blumer, Anselm; Ehrenfeucht, Andrzej; Haussler, David
- Information Processing Letters, Vol. 24, Issue 6
Generalized Linear Models
journal, January 1972
- Nelder, J. A.; Wedderburn, R. W. M.
- Journal of the Royal Statistical Society. Series A (General), Vol. 135, Issue 3
Closed-loop separation control using machine learning
journal, April 2015
- Gautier, N.; Aider, J. -L.; Duriez, T.
- Journal of Fluid Mechanics, Vol. 770
Learning partial differential equations via data discovery and sparse optimization
journal, January 2017
- Schaeffer, Hayden
- Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 473, Issue 2197
A fast and elitist multiobjective genetic algorithm: NSGA-II
journal, April 2002
- Deb, K.; Pratap, A.; Agarwal, S.
- IEEE Transactions on Evolutionary Computation, Vol. 6, Issue 2
Regression Shrinkage and Selection Via the Lasso
journal, January 1996
- Tibshirani, Robert
- Journal of the Royal Statistical Society: Series B (Methodological), Vol. 58, Issue 1
Existence of perturbed solitary wave solutions to a model equation for water waves
journal, September 1988
- Hunter, John K.; Scheurle, Jurgen
- Physica D: Nonlinear Phenomena, Vol. 32, Issue 2
The Roles of Plastic Surgeons in Advancing Artificial Intelligence in Plastic Surgery
journal, April 2021
- Luan, Fang; Gao, Xuejun; Zhao, Shanbaga
- Aesthetic Plastic Surgery
Deep Representational Similarity Learning for Analyzing Neural Signatures in Task-based fMRI Dataset
journal, October 2020
- Yousefnezhad, Muhammad; Sawalha, Jeffrey; Selvitella, Alessandro
- Neuroinformatics, Vol. 19, Issue 3
Machine learning identifies a strong association between warming and reduced primary productivity in an oligotrophic ocean gyre
journal, February 2020
- D’Alelio, Domenico; Rampone, Salvatore; Cusano, Luigi Maria
- Scientific Reports, Vol. 10, Issue 1
Circulating Proteomic Signature of Early Death in Heart Failure Patients with Reduced Ejection Fraction
journal, January 2019
- Cuvelliez, Marie; Vandewalle, Vincent; Brunin, Maxime
- SSRN Electronic Journal
Generalized Linear Models
journal, December 2000
- McCulloch, Charles E.
- Journal of the American Statistical Association, Vol. 95, Issue 452
Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
text, January 2015
- Brunton, Steven L.; Proctor, Joshua L.; Kutz, J. Nathan
- arXiv
Prediction of Dynamical Systems by Symbolic Regression
text, January 2016
- Quade, Markus; Abel, Markus; Shafi, Kamran
- arXiv
Data-driven discovery of partial differential equations
preprint, January 2016
- Rudy, Samuel H.; Brunton, Steven L.; Proctor, Joshua L.
- arXiv
Works referencing / citing this record:
Feature engineering and symbolic regression methods for detecting hidden physics from sparse sensor observation data
journal, January 2020
- Vaddireddy, Harsha; Rasheed, Adil; Staples, Anne E.
- Physics of Fluids, Vol. 32, Issue 1