Feature engineering and symbolic regression methods for detecting hidden physics from sparse sensor observation data
Abstract
Here we put forth a modular approach for distilling hidden flow physics from discrete and sparse observations. To address functional expressiblity, a key limitation of the black-box machine learning methods, we have exploited the use of symbolic regression as a principle for identifying relations and operators that are related to the underlying processes. This approach combines evolutionary computation with feature engineering to provide a tool for discovering hidden parameterizations embedded in the trajectory of fluid flows in the Eulerian frame of reference. Our approach in this study mainly involves gene expression programming (GEP) and sequential threshold ridge regression (STRidge) algorithms. We demonstrate our results in three different applications: (i) equation discovery, (ii) truncation error analysis, and (iii) hidden physics discovery, for which we include both predicting unknown source terms from a set of sparse observations and discovering subgrid scale closure models. We illustrate that both GEP and STRidge algorithms are able to distill the Smagorinsky model from an array of tailored features in solving the Kraichnan turbulence problem. Our results demonstrate the huge potential of these techniques in complex physics problems, and reveal the importance of feature selection and feature engineering in model discovery approaches.
- Authors:
-
[2];
[3];
[1]
- Oklahoma State Univ., Stillwater, OK (United States)
- Norwegian Univ. of Science and Technology, Trondheim (Norway)
- Virginia Polytechnic Inst. and State Univ. (Virginia Tech), Blacksburg, VA (United States)
- Publication Date:
- Research Org.:
- Oklahoma State Univ., Stillwater, OK (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
- OSTI Identifier:
- 1593556
- Alternate Identifier(s):
- OSTI ID: 1591974
- Grant/Contract Number:
- SC0019290
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Physics of Fluids
- Additional Journal Information:
- Journal Volume: 32; Journal Issue: 1; Journal ID: ISSN 1070-6631
- Publisher:
- American Institute of Physics (AIP)
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 42 ENGINEERING
Citation Formats
Vaddireddy, Harsha, Rasheed, Adil, Staples, Anne E., and San, Omer. Feature engineering and symbolic regression methods for detecting hidden physics from sparse sensor observation data. United States: N. p., 2020.
Web. doi:10.1063/1.5136351.
Vaddireddy, Harsha, Rasheed, Adil, Staples, Anne E., & San, Omer. Feature engineering and symbolic regression methods for detecting hidden physics from sparse sensor observation data. United States. doi:10.1063/1.5136351.
Vaddireddy, Harsha, Rasheed, Adil, Staples, Anne E., and San, Omer. Thu .
"Feature engineering and symbolic regression methods for detecting hidden physics from sparse sensor observation data". United States. doi:10.1063/1.5136351. https://www.osti.gov/servlets/purl/1593556.
@article{osti_1593556,
title = {Feature engineering and symbolic regression methods for detecting hidden physics from sparse sensor observation data},
author = {Vaddireddy, Harsha and Rasheed, Adil and Staples, Anne E. and San, Omer},
abstractNote = {Here we put forth a modular approach for distilling hidden flow physics from discrete and sparse observations. To address functional expressiblity, a key limitation of the black-box machine learning methods, we have exploited the use of symbolic regression as a principle for identifying relations and operators that are related to the underlying processes. This approach combines evolutionary computation with feature engineering to provide a tool for discovering hidden parameterizations embedded in the trajectory of fluid flows in the Eulerian frame of reference. Our approach in this study mainly involves gene expression programming (GEP) and sequential threshold ridge regression (STRidge) algorithms. We demonstrate our results in three different applications: (i) equation discovery, (ii) truncation error analysis, and (iii) hidden physics discovery, for which we include both predicting unknown source terms from a set of sparse observations and discovering subgrid scale closure models. We illustrate that both GEP and STRidge algorithms are able to distill the Smagorinsky model from an array of tailored features in solving the Kraichnan turbulence problem. Our results demonstrate the huge potential of these techniques in complex physics problems, and reveal the importance of feature selection and feature engineering in model discovery approaches.},
doi = {10.1063/1.5136351},
journal = {Physics of Fluids},
number = 1,
volume = 32,
place = {United States},
year = {2020},
month = {1}
}
Other availability
Search WorldCat to find libraries that may hold this journal
Cited by: 4 works
Citation information provided by
Web of Science
Web of Science
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
Stable signal recovery from incomplete and inaccurate measurements
journal, January 2006
- Candès, Emmanuel J.; Romberg, Justin K.; Tao, Terence
- Communications on Pure and Applied Mathematics, Vol. 59, Issue 8, p. 1207-1223
Diffusion Approximation for Two-Dimensional Turbulence
journal, January 1968
- Leith, C. E.
- Physics of Fluids, Vol. 11, Issue 3
XLI. On the change of form of long waves advancing in a rectangular canal, and on a new type of long stationary waves
journal, May 1895
- Korteweg, D. J.; de Vries, G.
- The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, Vol. 39, Issue 240
Deep learning
journal, May 2015
- LeCun, Yann; Bengio, Yoshua; Hinton, Geoffrey
- Nature, Vol. 521, Issue 7553
Two-Dimensional Turbulence
journal, January 2012
- Boffetta, Guido; Ecke, Robert E.
- Annual Review of Fluid Mechanics, Vol. 44, Issue 1
Distilling Free-Form Natural Laws from Experimental Data
journal, April 2009
- Schmidt, Michael; Lipson, Hod
- Science, Vol. 324, Issue 5923
Model selection for dynamical systems via sparse regression and information criteria
journal, August 2017
- Mangan, N. M.; Kutz, J. N.; Brunton, S. L.
- Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 473, Issue 2204
Automated reverse engineering of nonlinear dynamical systems
journal, June 2007
- Bongard, J.; Lipson, H.
- Proceedings of the National Academy of Sciences, Vol. 104, Issue 24
Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
journal, February 2019
- Raissi, M.; Perdikaris, P.; Karniadakis, G. E.
- Journal of Computational Physics, Vol. 378
Prediction and minimization of blast-induced ground vibration using two robust meta-heuristic algorithms
journal, February 2017
- Faradonbeh, Roohollah Shirani; Monjezi, Masoud
- Engineering with Computers, Vol. 33, Issue 4
Evidence for the double cascade scenario in two-dimensional turbulence
journal, July 2010
- Boffetta, G.; Musacchio, S.
- Physical Review E, Vol. 82, Issue 1
A new synergetic paradigm in environmental numerical modeling: Hybrid models combining deterministic and machine learning components
journal, January 2006
- Krasnopolsky, Vladimir M.; Fox-Rabinovitz, Michael S.
- Ecological Modelling, Vol. 191, Issue 1
Analytic solutions of the Nagumo equation
journal, January 1992
- Zhi-Xiong, Chen; Ben-Yu, Guo
- IMA Journal of Applied Mathematics, Vol. 48, Issue 2
Pseudospectral methods for Nagumo equation
journal, March 2011
- Dehghan, Mehdi; Fakhar-Izadi, Farhad
- International Journal for Numerical Methods in Biomedical Engineering, Vol. 27, Issue 4
Regularization and variable selection via the elastic net
journal, April 2005
- Zou, Hui; Hastie, Trevor
- Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 67, Issue 2
Roadheader performance prediction using genetic programming (GP) and gene expression programming (GEP) techniques
journal, August 2017
- Shirani Faradonbeh, Roohollah; Salimi, Alireza; Monjezi, Masoud
- Environmental Earth Sciences, Vol. 76, Issue 16
Closed-loop separation control over a sharp edge ramp using genetic programming
journal, February 2016
- Debien, Antoine; von Krbek, Kai A. F. F.; Mazellier, Nicolas
- Experiments in Fluids, Vol. 57, Issue 3
Multidimensional nonlinear diffusion arising in population genetics
journal, October 1978
- Aronson, D. G.; Weinberger, H. F.
- Advances in Mathematics, Vol. 30, Issue 1
An Introduction To Compressive Sampling
journal, March 2008
- Candes, E. J.; Wakin, M. B.
- IEEE Signal Processing Magazine, Vol. 25, Issue 2
The critical merger distance between two co-rotating quasi-geostrophic vortices
journal, January 2005
- Reinaud, Jean N.; Dritschel, David G.
- Journal of Fluid Mechanics, Vol. 522
Sparse identification of truncation errors
journal, November 2019
- Thaler, Stephan; Paehler, Ludger; Adams, Nikolaus A.
- Journal of Computational Physics, Vol. 397
A model unified field equation
journal, March 1962
- Perring, J. K.; Skyrme, T. H. R.
- Nuclear Physics, Vol. 31
Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence
book, January 1992
- Holland, John H.
- The MIT Press
Application of an evolutionary algorithm to LES modelling of turbulent transport in premixed flames
journal, December 2018
- Schoepplein, Matthias; Weatheritt, Jack; Sandberg, Richard
- Journal of Computational Physics, Vol. 374
Hidden physics models: Machine learning of nonlinear partial differential equations
journal, March 2018
- Raissi, Maziar; Karniadakis, George Em
- Journal of Computational Physics, Vol. 357
Prediction of compressive and tensile strength of Gaziantep basalts via neural networks and gene expression programming
journal, November 2008
- Çanakcı, Hanifi; Baykasoğlu, Adil; Güllü, Hamza
- Neural Computing and Applications, Vol. 18, Issue 8
Hybrid Reynolds-Averaged/Large-Eddy Simulation Methodology from Symbolic Regression: Formulation and Application
journal, November 2017
- Weatheritt, Jack; Sandberg, Richard D.
- AIAA Journal, Vol. 55, Issue 11
Data-driven deconvolution for large eddy simulations of Kraichnan turbulence
journal, December 2018
- Maulik, R.; San, O.; Rasheed, A.
- Physics of Fluids, Vol. 30, Issue 12
Physics of vortex merging
journal, May 2005
- Meunier, Patrice; Le Dizès, Stéphane; Leweke, Thomas
- Comptes Rendus Physique, Vol. 6, Issue 4-5
Inertial Ranges in Two-Dimensional Turbulence
journal, January 1967
- Kraichnan, Robert H.
- Physics of Fluids, Vol. 10, Issue 7
Heuristic stability theory for finite-difference equations
journal, June 1968
- Hirt, C. W.
- Journal of Computational Physics, Vol. 2, Issue 4
Data-driven discovery of partial differential equations
journal, April 2017
- Rudy, Samuel H.; Brunton, Steven L.; Proctor, Joshua L.
- Science Advances, Vol. 3, Issue 4
Dynamic systems identification with Gaussian processes
journal, December 2005
- Kocijan, Juš; Girard, Agathe; Banko, Blaž
- Mathematical and Computer Modelling of Dynamical Systems, Vol. 11, Issue 4
A coarse-grid projection method for accelerating incompressible flow computations
journal, January 2013
- San, Omer; Staples, Anne E.
- Journal of Computational Physics, Vol. 233
Parse-matrix evolution for symbolic regression
journal, September 2012
- Luo, Changtong; Zhang, Shao-Liang
- Engineering Applications of Artificial Intelligence, Vol. 25, Issue 6
An Active Pulse Transmission Line Simulating Nerve Axon
journal, October 1962
- Nagumo, J.; Arimoto, S.; Yoshizawa, S.
- Proceedings of the IRE, Vol. 50, Issue 10
Regularization Paths for Generalized Linear Models via Coordinate Descent
journal, January 2010
- Friedman, Jerome; Hastie, Trevor; Tibshirani, Robert
- Journal of Statistical Software, Vol. 33, Issue 1
Prediction of dynamical systems by symbolic regression
journal, July 2016
- Quade, Markus; Abel, Markus; Shafi, Kamran
- Physical Review E, Vol. 94, Issue 1
Inferring Biological Networks by Sparse Identification of Nonlinear Dynamics
journal, June 2016
- Mangan, Niall M.; Brunton, Steven L.; Proctor, Joshua L.
- IEEE Transactions on Molecular, Biological and Multi-Scale Communications, Vol. 2, Issue 1
Model identification of reduced order fluid dynamics systems using deep learning: Model identification in fluid dynamics using deep learning
journal, August 2017
- Wang, Z.; Xiao, D.; Fang, F.
- International Journal for Numerical Methods in Fluids, Vol. 86, Issue 4
Extracting Sparse High-Dimensional Dynamics from Limited Data
journal, January 2018
- Schaeffer, Hayden; Tran, Giang; Ward, Rachel
- SIAM Journal on Applied Mathematics, Vol. 78, Issue 6
Enhancing Sparsity by Reweighted ℓ 1 Minimization
journal, October 2008
- Candès, Emmanuel J.; Wakin, Michael B.; Boyd, Stephen P.
- Journal of Fourier Analysis and Applications, Vol. 14, Issue 5-6
Closed-loop separation control using machine learning
journal, April 2015
- Gautier, N.; Aider, J. -L.; Duriez, T.
- Journal of Fluid Mechanics, Vol. 770
Large-eddy simulation: achievements and challenges
journal, May 1999
- Piomelli, U.
- Progress in Aerospace Sciences, Vol. 35, Issue 4
Elite bases regression: A real-time algorithm for symbolic regression
conference, July 2017
- Chen, Chen; Luo, Changtong; Jiang, Zonglin
- 2017 13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD)
Oscillatory Solitary Waves in Dispersive Media
journal, July 1972
- Kawahara, Takuji
- Journal of the Physical Society of Japan, Vol. 33, Issue 1
A systematic approach for correcting nonlinear instabilities: The Lax-Wendroff scheme for scalar conservation laws
journal, December 1978
- Majda, Andrew; Osher, Stanley
- Numerische Mathematik, Vol. 30, Issue 4
Nonlinear Interaction between Short and Long Capillary-Gravity Waves
journal, November 1975
- Kawahara, Takuji; Sugimoto, Nobumasa; Kakutani, Tsunehiko
- Journal of the Physical Society of Japan, Vol. 39, Issue 5
Numerical solutions for solute transport in unconfined aquifers
journal, March 1983
- Guvanasen, V.; Volker, R. E.
- International Journal for Numerical Methods in Fluids, Vol. 3, Issue 2
The perceptron: A probabilistic model for information storage and organization in the brain.
journal, January 1958
- Rosenblatt, F.
- Psychological Review, Vol. 65, Issue 6
Identification strategies for model-based control
journal, July 2013
- Cordier, Laurent; Noack, Bernd R.; Tissot, Gilles
- Experiments in Fluids, Vol. 54, Issue 8
Neuristor propagation on a tunnel diode loaded transmission line
journal, January 1963
- Scott, A. C.
- Proceedings of the IEEE, Vol. 51, Issue 1
High-order methods for decaying two-dimensional homogeneous isotropic turbulence
journal, June 2012
- San, Omer; Staples, Anne E.
- Computers & Fluids, Vol. 63
Compressive Sensing [Lecture Notes]
journal, August 2007
- Baraniuk, Richard G.
- IEEE Signal Processing Magazine, Vol. 24, Issue 4, p. 118-121
A modified tanh–coth method for solving the KdV and the KdV–Burgers’ equations
journal, February 2009
- Wazzan, Luwai
- Communications in Nonlinear Science and Numerical Simulation, Vol. 14, Issue 2
Semi-autogenous mill power model development using gene expression programming
journal, February 2017
- Hoseinian, Fatemeh Sadat; Faradonbeh, Roohollah Shirani; Abdollahzadeh, Aliakbar
- Powder Technology, Vol. 308
Complex hybrid models combining deterministic and machine learning components for numerical climate modeling and weather prediction
journal, March 2006
- Krasnopolsky, Vladimir M.; Fox-Rabinovitz, Michael S.
- Neural Networks, Vol. 19, Issue 2
Sparse reduced-order modelling: sensor-based dynamics to full-state estimation
journal, April 2018
- Loiseau, Jean-Christophe; Noack, Bernd R.; Brunton, Steven L.
- Journal of Fluid Mechanics, Vol. 844
Atmospheric Predictability and Two-Dimensional Turbulence
journal, March 1971
- Leith, C. E.
- Journal of the Atmospheric Sciences, Vol. 28, Issue 2
Numerical Gaussian Processes for Time-Dependent and Nonlinear Partial Differential Equations
journal, January 2018
- Raissi, Maziar; Perdikaris, Paris; Karniadakis, George Em
- SIAM Journal on Scientific Computing, Vol. 40, Issue 1
A rationale for implicit turbulence modelling
journal, January 2002
- Margolin, Len G.; Rider, William J.
- International Journal for Numerical Methods in Fluids, Vol. 39, Issue 9
Exact Recovery of Chaotic Systems from Highly Corrupted Data
journal, January 2017
- Tran, Giang; Ward, Rachel
- Multiscale Modeling & Simulation, Vol. 15, Issue 3
Image Restoration: Wavelet Frame Shrinkage, Nonlinear Evolution PDEs, and Beyond
journal, January 2017
- Dong, Bin; Jiang, Qingtang; Shen, Zuowei
- Multiscale Modeling & Simulation, Vol. 15, Issue 1
Computation of the Energy Spectrum in Homogeneous Two-Dimensional Turbulence
journal, January 1969
- Batchelor, G. K.
- Physics of Fluids, Vol. 12, Issue 12
A Unified Framework for Sparse Relaxed Regularized Regression: SR3
journal, January 2019
- Zheng, Peng; Askham, Travis; Brunton, Steven L.
- IEEE Access, Vol. 7
Sub-grid scale model classification and blending through deep learning
journal, May 2019
- Maulik, Romit; San, Omer; Jacob, Jamey D.
- Journal of Fluid Mechanics, Vol. 870
Learning partial differential equations via data discovery and sparse optimization
journal, January 2017
- Schaeffer, Hayden
- Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 473, Issue 2197
Scale-Invariance and Turbulence Models for Large-Eddy Simulation
journal, January 2000
- Meneveau, Charles; Katz, Joseph
- Annual Review of Fluid Mechanics, Vol. 32, Issue 1
Regression Shrinkage and Selection Via the Lasso
journal, January 1996
- Tibshirani, Robert
- Journal of the Royal Statistical Society: Series B (Methodological), Vol. 58, Issue 1
Adaptive space transformation: An invariant based method for predicting aerodynamic coefficients of hypersonic vehicles
journal, November 2015
- Luo, Changtong; Hu, Zongmin; Zhang, Shao-Liang
- Engineering Applications of Artificial Intelligence, Vol. 46
Implicit subgrid-scale modeling by adaptive deconvolution
journal, November 2004
- Adams, N. A.; Hickel, S.; Franz, S.
- Journal of Computational Physics, Vol. 200, Issue 2
Force identification of dynamic systems using genetic programming
journal, January 2005
- Yang, Y. W.; Wang, C.; Soh, C. K.
- International Journal for Numerical Methods in Engineering, Vol. 63, Issue 9
Closed-Loop Turbulence Control: Progress and Challenges
journal, August 2015
- Brunton, Steven L.; Noack, Bernd R.
- Applied Mechanics Reviews, Vol. 67, Issue 5
Unsteady flow against dispersion in finite porous media
journal, June 1983
- Kumar, Naveen
- Journal of Hydrology, Vol. 63, Issue 3-4
Theory and applications of the sine-gordon equation
journal, April 1971
- Barone, A.; Esposito, F.; Magee, C. J.
- La Rivista del Nuovo Cimento, Vol. 1, Issue 2
Subgrid modelling for two-dimensional turbulence using neural networks
journal, November 2018
- Maulik, R.; San, O.; Rasheed, A.
- Journal of Fluid Mechanics, Vol. 858
Nonlinear system identification: From multiple-model networks to Gaussian processes
journal, October 2008
- Gregorčič, Gregor; Lightbody, Gordon
- Engineering Applications of Artificial Intelligence, Vol. 21, Issue 7
Numerical solution of modified differential equations based on symmetry preservation
journal, December 2017
- Ozbenli, Ersin; Vedula, Prakash
- Physical Review E, Vol. 96, Issue 6
Nonlinear truncation error analysis of finite difference schemes forthe Euler equations
journal, April 1983
- Klopfer, Goetzh; McRae, David S.
- AIAA Journal, Vol. 21, Issue 4
Discovering governing equations from data by sparse identification of nonlinear dynamical systems
journal, March 2016
- Brunton, Steven L.; Proctor, Joshua L.; Kutz, J. Nathan
- Proceedings of the National Academy of Sciences, Vol. 113, Issue 15
A novel evolutionary algorithm applied to algebraic modifications of the RANS stress–strain relationship
journal, November 2016
- Weatheritt, Jack; Sandberg, Richard
- Journal of Computational Physics, Vol. 325
PDE-Net 2.0: Learning PDEs from data with a numeric-symbolic hybrid deep network
journal, December 2019
- Long, Zichao; Lu, Yiping; Dong, Bin
- Journal of Computational Physics, Vol. 399
Deep Reinforcement Learning framework for Autonomous Driving
journal, January 2017
- Sallab, AhmadEL; Abdou, Mohammed; Perot, Etienne
- Electronic Imaging, Vol. 2017, Issue 19
Computational design for long-term numerical integration of the equations of fluid motion: Two-dimensional incompressible flow. Part I
journal, August 1966
- Arakawa, Akio
- Journal of Computational Physics, Vol. 1, Issue 1
New exact travelling wave solutions for the Kawahara and modified Kawahara equations
journal, January 2004
- Sirendaoreji,
- Chaos, Solitons & Fractals, Vol. 19, Issue 1
High order accurate finite difference schemes based on symmetry preservation
journal, November 2017
- Ozbenli, Ersin; Vedula, Prakash
- Journal of Computational Physics, Vol. 349
CFD Julia: A Learning Module Structuring an Introductory Course on Computational Fluid Dynamics
journal, August 2019
- Pawar, Suraj; San, Omer
- Fluids, Vol. 4, Issue 3
Sparse dynamics for partial differential equations
journal, March 2013
- Schaeffer, H.; Caflisch, R.; Hauck, C. D.
- Proceedings of the National Academy of Sciences, Vol. 110, Issue 17
Internet of Things Mobile–Air Pollution Monitoring System (IoT-Mobair)
journal, June 2019
- Dhingra, Swati; Madda, Rajasekhara Babu; Gandomi, Amir H.
- IEEE Internet of Things Journal, Vol. 6, Issue 3
A simple similarity-transformation-iterative scheme applied to Korteweg–de Vries equation
journal, February 2006
- Öziş, T.; Özer, S.
- Applied Mathematics and Computation, Vol. 173, Issue 1
Equation Discovery Using Fast Function Extraction: a Deterministic Symbolic Regression Approach
journal, June 2019
- Vaddireddy, Harsha; San, Omer
- Fluids, Vol. 4, Issue 2
Heat transfer to a draining film
journal, February 1973
- Isenberg, J.; Gutfinger, C.
- International Journal of Heat and Mass Transfer, Vol. 16, Issue 2
Machine learning: Trends, perspectives, and prospects
journal, July 2015
- Jordan, M. I.; Mitchell, T. M.
- Science, Vol. 349, Issue 6245
Some Recent Researches on the Motion of Fluids
journal, April 1915
- Bateman, Harry
- Monthly Weather Review, Vol. 43, Issue 4
Existence of perturbed solitary wave solutions to a model equation for water waves
journal, September 1988
- Hunter, John K.; Scheurle, Jurgen
- Physica D: Nonlinear Phenomena, Vol. 32, Issue 2
General Circulation Experiments with the Primitive Equations: i. the Basic Experiment*
journal, March 1963
- Smagorinsky, J.
- Monthly Weather Review, Vol. 91, Issue 3
Two-dimensional turbulence
journal, May 1980
- Kraichnan, R. H.; Montgomery, D.
- Reports on Progress in Physics, Vol. 43, Issue 5