skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

This content will become publicly available on January 16, 2021

Title: Feature engineering and symbolic regression methods for detecting hidden physics from sparse sensor observation data

Abstract

Here we put forth a modular approach for distilling hidden flow physics from discrete and sparse observations. To address functional expressiblity, a key limitation of the black-box machine learning methods, we have exploited the use of symbolic regression as a principle for identifying relations and operators that are related to the underlying processes. This approach combines evolutionary computation with feature engineering to provide a tool for discovering hidden parameterizations embedded in the trajectory of fluid flows in the Eulerian frame of reference. Our approach in this study mainly involves gene expression programming (GEP) and sequential threshold ridge regression (STRidge) algorithms. We demonstrate our results in three different applications: (i) equation discovery, (ii) truncation error analysis, and (iii) hidden physics discovery, for which we include both predicting unknown source terms from a set of sparse observations and discovering subgrid scale closure models. We illustrate that both GEP and STRidge algorithms are able to distill the Smagorinsky model from an array of tailored features in solving the Kraichnan turbulence problem. Our results demonstrate the huge potential of these techniques in complex physics problems, and reveal the importance of feature selection and feature engineering in model discovery approaches.

Authors:
 [1]; ORCiD logo [2]; ORCiD logo [3]; ORCiD logo [1]
  1. Oklahoma State Univ., Stillwater, OK (United States)
  2. Norwegian Univ. of Science and Technology, Trondheim (Norway)
  3. Virginia Polytechnic Inst. and State Univ. (Virginia Tech), Blacksburg, VA (United States)
Publication Date:
Research Org.:
Oklahoma State Univ., Stillwater, OK (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)
OSTI Identifier:
1593556
Alternate Identifier(s):
OSTI ID: 1591974
Grant/Contract Number:  
SC0019290
Resource Type:
Accepted Manuscript
Journal Name:
Physics of Fluids
Additional Journal Information:
Journal Volume: 32; Journal Issue: 1; Journal ID: ISSN 1070-6631
Publisher:
American Institute of Physics (AIP)
Country of Publication:
United States
Language:
English
Subject:
42 ENGINEERING

Citation Formats

Vaddireddy, Harsha, Rasheed, Adil, Staples, Anne E., and San, Omer. Feature engineering and symbolic regression methods for detecting hidden physics from sparse sensor observation data. United States: N. p., 2020. Web. doi:10.1063/1.5136351.
Vaddireddy, Harsha, Rasheed, Adil, Staples, Anne E., & San, Omer. Feature engineering and symbolic regression methods for detecting hidden physics from sparse sensor observation data. United States. doi:10.1063/1.5136351.
Vaddireddy, Harsha, Rasheed, Adil, Staples, Anne E., and San, Omer. Thu . "Feature engineering and symbolic regression methods for detecting hidden physics from sparse sensor observation data". United States. doi:10.1063/1.5136351.
@article{osti_1593556,
title = {Feature engineering and symbolic regression methods for detecting hidden physics from sparse sensor observation data},
author = {Vaddireddy, Harsha and Rasheed, Adil and Staples, Anne E. and San, Omer},
abstractNote = {Here we put forth a modular approach for distilling hidden flow physics from discrete and sparse observations. To address functional expressiblity, a key limitation of the black-box machine learning methods, we have exploited the use of symbolic regression as a principle for identifying relations and operators that are related to the underlying processes. This approach combines evolutionary computation with feature engineering to provide a tool for discovering hidden parameterizations embedded in the trajectory of fluid flows in the Eulerian frame of reference. Our approach in this study mainly involves gene expression programming (GEP) and sequential threshold ridge regression (STRidge) algorithms. We demonstrate our results in three different applications: (i) equation discovery, (ii) truncation error analysis, and (iii) hidden physics discovery, for which we include both predicting unknown source terms from a set of sparse observations and discovering subgrid scale closure models. We illustrate that both GEP and STRidge algorithms are able to distill the Smagorinsky model from an array of tailored features in solving the Kraichnan turbulence problem. Our results demonstrate the huge potential of these techniques in complex physics problems, and reveal the importance of feature selection and feature engineering in model discovery approaches.},
doi = {10.1063/1.5136351},
journal = {Physics of Fluids},
number = 1,
volume = 32,
place = {United States},
year = {2020},
month = {1}
}

Journal Article:
Free Publicly Available Full Text
This content will become publicly available on January 16, 2021
Publisher's Version of Record

Save / Share:

Works referenced in this record:

Stable signal recovery from incomplete and inaccurate measurements
journal, January 2006

  • Candès, Emmanuel J.; Romberg, Justin K.; Tao, Terence
  • Communications on Pure and Applied Mathematics, Vol. 59, Issue 8, p. 1207-1223
  • DOI: 10.1002/cpa.20124

Diffusion Approximation for Two-Dimensional Turbulence
journal, January 1968


XLI. On the change of form of long waves advancing in a rectangular canal, and on a new type of long stationary waves
journal, May 1895

  • Korteweg, D. J.; de Vries, G.
  • The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, Vol. 39, Issue 240
  • DOI: 10.1080/14786449508620739

Deep learning
journal, May 2015

  • LeCun, Yann; Bengio, Yoshua; Hinton, Geoffrey
  • Nature, Vol. 521, Issue 7553
  • DOI: 10.1038/nature14539

Two-Dimensional Turbulence
journal, January 2012


Distilling Free-Form Natural Laws from Experimental Data
journal, April 2009


Model selection for dynamical systems via sparse regression and information criteria
journal, August 2017

  • Mangan, N. M.; Kutz, J. N.; Brunton, S. L.
  • Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 473, Issue 2204
  • DOI: 10.1098/rspa.2017.0009

Automated reverse engineering of nonlinear dynamical systems
journal, June 2007

  • Bongard, J.; Lipson, H.
  • Proceedings of the National Academy of Sciences, Vol. 104, Issue 24
  • DOI: 10.1073/pnas.0609476104

Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
journal, February 2019


Prediction and minimization of blast-induced ground vibration using two robust meta-heuristic algorithms
journal, February 2017


A new synergetic paradigm in environmental numerical modeling: Hybrid models combining deterministic and machine learning components
journal, January 2006


Analytic solutions of the Nagumo equation
journal, January 1992


Pseudospectral methods for Nagumo equation
journal, March 2011

  • Dehghan, Mehdi; Fakhar-Izadi, Farhad
  • International Journal for Numerical Methods in Biomedical Engineering, Vol. 27, Issue 4
  • DOI: 10.1002/cnm.1319

Regularization and variable selection via the elastic net
journal, April 2005


Roadheader performance prediction using genetic programming (GP) and gene expression programming (GEP) techniques
journal, August 2017

  • Shirani Faradonbeh, Roohollah; Salimi, Alireza; Monjezi, Masoud
  • Environmental Earth Sciences, Vol. 76, Issue 16
  • DOI: 10.1007/s12665-017-6920-2

Closed-loop separation control over a sharp edge ramp using genetic programming
journal, February 2016

  • Debien, Antoine; von Krbek, Kai A. F. F.; Mazellier, Nicolas
  • Experiments in Fluids, Vol. 57, Issue 3
  • DOI: 10.1007/s00348-016-2126-8

Multidimensional nonlinear diffusion arising in population genetics
journal, October 1978


The critical merger distance between two co-rotating quasi-geostrophic vortices
journal, January 2005


Sparse identification of truncation errors
journal, November 2019

  • Thaler, Stephan; Paehler, Ludger; Adams, Nikolaus A.
  • Journal of Computational Physics, Vol. 397
  • DOI: 10.1016/j.jcp.2019.07.049

A model unified field equation
journal, March 1962


Application of an evolutionary algorithm to LES modelling of turbulent transport in premixed flames
journal, December 2018

  • Schoepplein, Matthias; Weatheritt, Jack; Sandberg, Richard
  • Journal of Computational Physics, Vol. 374
  • DOI: 10.1016/j.jcp.2018.08.016

Hidden physics models: Machine learning of nonlinear partial differential equations
journal, March 2018


Prediction of compressive and tensile strength of Gaziantep basalts via neural networks and gene expression programming
journal, November 2008

  • Çanakcı, Hanifi; Baykasoğlu, Adil; Güllü, Hamza
  • Neural Computing and Applications, Vol. 18, Issue 8
  • DOI: 10.1007/s00521-008-0208-0

Hybrid Reynolds-Averaged/Large-Eddy Simulation Methodology from Symbolic Regression: Formulation and Application
journal, November 2017

  • Weatheritt, Jack; Sandberg, Richard D.
  • AIAA Journal, Vol. 55, Issue 11
  • DOI: 10.2514/1.j055378

Data-driven deconvolution for large eddy simulations of Kraichnan turbulence
journal, December 2018

  • Maulik, R.; San, O.; Rasheed, A.
  • Physics of Fluids, Vol. 30, Issue 12
  • DOI: 10.1063/1.5079582

Physics of vortex merging
journal, May 2005

  • Meunier, Patrice; Le Dizès, Stéphane; Leweke, Thomas
  • Comptes Rendus Physique, Vol. 6, Issue 4-5
  • DOI: 10.1016/j.crhy.2005.06.003

Inertial Ranges in Two-Dimensional Turbulence
journal, January 1967


Heuristic stability theory for finite-difference equations
journal, June 1968


Data-driven discovery of partial differential equations
journal, April 2017

  • Rudy, Samuel H.; Brunton, Steven L.; Proctor, Joshua L.
  • Science Advances, Vol. 3, Issue 4
  • DOI: 10.1126/sciadv.1602614

Dynamic systems identification with Gaussian processes
journal, December 2005

  • Kocijan, Juš; Girard, Agathe; Banko, Blaž
  • Mathematical and Computer Modelling of Dynamical Systems, Vol. 11, Issue 4
  • DOI: 10.1080/13873950500068567

A coarse-grid projection method for accelerating incompressible flow computations
journal, January 2013


Parse-matrix evolution for symbolic regression
journal, September 2012


Regularization Paths for Generalized Linear Models via Coordinate Descent
journal, January 2010

  • Friedman, Jerome; Hastie, Trevor; Tibshirani, Robert
  • Journal of Statistical Software, Vol. 33, Issue 1
  • DOI: 10.18637/jss.v033.i01

Prediction of dynamical systems by symbolic regression
journal, July 2016


Model identification of reduced order fluid dynamics systems using deep learning: Model identification in fluid dynamics using deep learning
journal, August 2017

  • Wang, Z.; Xiao, D.; Fang, F.
  • International Journal for Numerical Methods in Fluids, Vol. 86, Issue 4
  • DOI: 10.1002/fld.4416

Enhancing Sparsity by Reweighted ℓ 1 Minimization
journal, October 2008

  • Candès, Emmanuel J.; Wakin, Michael B.; Boyd, Stephen P.
  • Journal of Fourier Analysis and Applications, Vol. 14, Issue 5-6
  • DOI: 10.1007/s00041-008-9045-x

Closed-loop separation control using machine learning
journal, April 2015

  • Gautier, N.; Aider, J. -L.; Duriez, T.
  • Journal of Fluid Mechanics, Vol. 770
  • DOI: 10.1017/jfm.2015.95

Elite bases regression: A real-time algorithm for symbolic regression
conference, July 2017

  • Chen, Chen; Luo, Changtong; Jiang, Zonglin
  • 2017 13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD)
  • DOI: 10.1109/FSKD.2017.8393325

Oscillatory Solitary Waves in Dispersive Media
journal, July 1972

  • Kawahara, Takuji
  • Journal of the Physical Society of Japan, Vol. 33, Issue 1
  • DOI: 10.1143/jpsj.33.260

A systematic approach for correcting nonlinear instabilities: The Lax-Wendroff scheme for scalar conservation laws
journal, December 1978

  • Majda, Andrew; Osher, Stanley
  • Numerische Mathematik, Vol. 30, Issue 4
  • DOI: 10.1007/bf01398510

Nonlinear Interaction between Short and Long Capillary-Gravity Waves
journal, November 1975

  • Kawahara, Takuji; Sugimoto, Nobumasa; Kakutani, Tsunehiko
  • Journal of the Physical Society of Japan, Vol. 39, Issue 5
  • DOI: 10.1143/jpsj.39.1379

Numerical solutions for solute transport in unconfined aquifers
journal, March 1983

  • Guvanasen, V.; Volker, R. E.
  • International Journal for Numerical Methods in Fluids, Vol. 3, Issue 2
  • DOI: 10.1002/fld.1650030203

The perceptron: A probabilistic model for information storage and organization in the brain.
journal, January 1958


Identification strategies for model-based control
journal, July 2013


Neuristor propagation on a tunnel diode loaded transmission line
journal, January 1963


High-order methods for decaying two-dimensional homogeneous isotropic turbulence
journal, June 2012


A modified tanh–coth method for solving the KdV and the KdV–Burgers’ equations
journal, February 2009


The big challenges of big data
journal, June 2013


Semi-autogenous mill power model development using gene expression programming
journal, February 2017

  • Hoseinian, Fatemeh Sadat; Faradonbeh, Roohollah Shirani; Abdollahzadeh, Aliakbar
  • Powder Technology, Vol. 308
  • DOI: 10.1016/j.powtec.2016.11.045

Complex hybrid models combining deterministic and machine learning components for numerical climate modeling and weather prediction
journal, March 2006


Sparse reduced-order modelling: sensor-based dynamics to full-state estimation
journal, April 2018

  • Loiseau, Jean-Christophe; Noack, Bernd R.; Brunton, Steven L.
  • Journal of Fluid Mechanics, Vol. 844
  • DOI: 10.1017/jfm.2018.147

Numerical Gaussian Processes for Time-Dependent and Nonlinear Partial Differential Equations
journal, January 2018

  • Raissi, Maziar; Perdikaris, Paris; Karniadakis, George Em
  • SIAM Journal on Scientific Computing, Vol. 40, Issue 1
  • DOI: 10.1137/17m1120762

A rationale for implicit turbulence modelling
journal, January 2002

  • Margolin, Len G.; Rider, William J.
  • International Journal for Numerical Methods in Fluids, Vol. 39, Issue 9
  • DOI: 10.1002/fld.331

Image Restoration: Wavelet Frame Shrinkage, Nonlinear Evolution PDEs, and Beyond
journal, January 2017

  • Dong, Bin; Jiang, Qingtang; Shen, Zuowei
  • Multiscale Modeling & Simulation, Vol. 15, Issue 1
  • DOI: 10.1137/15m1037457

Computation of the Energy Spectrum in Homogeneous Two-Dimensional Turbulence
journal, January 1969


Sub-grid scale model classification and blending through deep learning
journal, May 2019

  • Maulik, Romit; San, Omer; Jacob, Jamey D.
  • Journal of Fluid Mechanics, Vol. 870
  • DOI: 10.1017/jfm.2019.254

Learning partial differential equations via data discovery and sparse optimization
journal, January 2017

  • Schaeffer, Hayden
  • Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 473, Issue 2197
  • DOI: 10.1098/rspa.2016.0446

Scale-Invariance and Turbulence Models for Large-Eddy Simulation
journal, January 2000


Regression Shrinkage and Selection Via the Lasso
journal, January 1996


Adaptive space transformation: An invariant based method for predicting aerodynamic coefficients of hypersonic vehicles
journal, November 2015


Implicit subgrid-scale modeling by adaptive deconvolution
journal, November 2004


Force identification of dynamic systems using genetic programming
journal, January 2005

  • Yang, Y. W.; Wang, C.; Soh, C. K.
  • International Journal for Numerical Methods in Engineering, Vol. 63, Issue 9
  • DOI: 10.1002/nme.1323

Closed-Loop Turbulence Control: Progress and Challenges
journal, August 2015

  • Brunton, Steven L.; Noack, Bernd R.
  • Applied Mechanics Reviews, Vol. 67, Issue 5
  • DOI: 10.1115/1.4031175

Unsteady flow against dispersion in finite porous media
journal, June 1983


Theory and applications of the sine-gordon equation
journal, April 1971

  • Barone, A.; Esposito, F.; Magee, C. J.
  • La Rivista del Nuovo Cimento, Vol. 1, Issue 2
  • DOI: 10.1007/bf02820622

Subgrid modelling for two-dimensional turbulence using neural networks
journal, November 2018


Nonlinear system identification: From multiple-model networks to Gaussian processes
journal, October 2008


Numerical solution of modified differential equations based on symmetry preservation
journal, December 2017


Nonlinear truncation error analysis of finite difference schemes forthe Euler equations
journal, April 1983

  • Klopfer, Goetzh; McRae, David S.
  • AIAA Journal, Vol. 21, Issue 4
  • DOI: 10.2514/3.8104

Discovering governing equations from data by sparse identification of nonlinear dynamical systems
journal, March 2016

  • Brunton, Steven L.; Proctor, Joshua L.; Kutz, J. Nathan
  • Proceedings of the National Academy of Sciences, Vol. 113, Issue 15
  • DOI: 10.1073/pnas.1517384113

A novel evolutionary algorithm applied to algebraic modifications of the RANS stress–strain relationship
journal, November 2016


PDE-Net 2.0: Learning PDEs from data with a numeric-symbolic hybrid deep network
journal, December 2019


Deep Reinforcement Learning framework for Autonomous Driving
journal, January 2017


New exact travelling wave solutions for the Kawahara and modified Kawahara equations
journal, January 2004


High order accurate finite difference schemes based on symmetry preservation
journal, November 2017


CFD Julia: A Learning Module Structuring an Introductory Course on Computational Fluid Dynamics
journal, August 2019


Sparse dynamics for partial differential equations
journal, March 2013

  • Schaeffer, H.; Caflisch, R.; Hauck, C. D.
  • Proceedings of the National Academy of Sciences, Vol. 110, Issue 17
  • DOI: 10.1073/pnas.1302752110

Internet of Things Mobile–Air Pollution Monitoring System (IoT-Mobair)
journal, June 2019

  • Dhingra, Swati; Madda, Rajasekhara Babu; Gandomi, Amir H.
  • IEEE Internet of Things Journal, Vol. 6, Issue 3
  • DOI: 10.1109/jiot.2019.2903821

A simple similarity-transformation-iterative scheme applied to Korteweg–de Vries equation
journal, February 2006


Equation Discovery Using Fast Function Extraction: a Deterministic Symbolic Regression Approach
journal, June 2019


Heat transfer to a draining film
journal, February 1973


Machine learning: Trends, perspectives, and prospects
journal, July 2015


Some Recent Researches on the Motion of Fluids
journal, April 1915


Existence of perturbed solitary wave solutions to a model equation for water waves
journal, September 1988