DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Feature engineering and symbolic regression methods for detecting hidden physics from sparse sensor observation data

Journal Article · · Physics of Fluids
DOI: https://doi.org/10.1063/1.5136351 · OSTI ID:1593556
 [1]; ORCiD logo [2]; ORCiD logo [3]; ORCiD logo [4]
  1. Oklahoma State Univ., Stillwater, OK (United States); Oklahoma State University Stillwater
  2. Norwegian Univ. of Science and Technology, Trondheim (Norway)
  3. Virginia Polytechnic Inst. and State Univ. (Virginia Tech), Blacksburg, VA (United States)
  4. Oklahoma State Univ., Stillwater, OK (United States)

Here we put forth a modular approach for distilling hidden flow physics from discrete and sparse observations. To address functional expressiblity, a key limitation of the black-box machine learning methods, we have exploited the use of symbolic regression as a principle for identifying relations and operators that are related to the underlying processes. This approach combines evolutionary computation with feature engineering to provide a tool for discovering hidden parameterizations embedded in the trajectory of fluid flows in the Eulerian frame of reference. Our approach in this study mainly involves gene expression programming (GEP) and sequential threshold ridge regression (STRidge) algorithms. We demonstrate our results in three different applications: (i) equation discovery, (ii) truncation error analysis, and (iii) hidden physics discovery, for which we include both predicting unknown source terms from a set of sparse observations and discovering subgrid scale closure models. We illustrate that both GEP and STRidge algorithms are able to distill the Smagorinsky model from an array of tailored features in solving the Kraichnan turbulence problem. Our results demonstrate the huge potential of these techniques in complex physics problems, and reveal the importance of feature selection and feature engineering in model discovery approaches.

Research Organization:
Oklahoma State Univ., Stillwater, OK (United States)
Sponsoring Organization:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)
Grant/Contract Number:
SC0019290
OSTI ID:
1593556
Journal Information:
Physics of Fluids, Journal Name: Physics of Fluids Journal Issue: 1 Vol. 32; ISSN 1070-6631
Publisher:
American Institute of Physics (AIP)Copyright Statement
Country of Publication:
United States
Language:
English

References (120)

Pseudospectral methods for Nagumo equation journal March 2011
Stable signal recovery from incomplete and inaccurate measurements
  • Candès, Emmanuel J.; Romberg, Justin K.; Tao, Terence
  • Communications on Pure and Applied Mathematics, Vol. 59, Issue 8, p. 1207-1223 https://doi.org/10.1002/cpa.20124
journal January 2006
Numerical solutions for solute transport in unconfined aquifers journal March 1983
A rationale for implicit turbulence modelling journal January 2002
Model identification of reduced order fluid dynamics systems using deep learning: Model identification in fluid dynamics using deep learning journal August 2017
Force identification of dynamic systems using genetic programming journal January 2005
Computational Design for Long-Term Numerical Integration of the Equations of Fluid Motion: Two-Dimensional Incompressible Flow. Part I journal August 1997
An Introduction to Statistical Learning book January 2021
An Introduction to Statistical Learning book January 2013
A systematic approach for correcting nonlinear instabilities: The Lax-Wendroff scheme for scalar conservation laws journal December 1978
Theory and applications of the sine-gordon equation journal April 1971
Enhancing Sparsity by Reweighted ℓ 1 Minimization journal October 2008
The Roles of Plastic Surgeons in Advancing Artificial Intelligence in Plastic Surgery journal April 2021
Identification strategies for model-based control journal July 2013
Closed-loop separation control over a sharp edge ramp using genetic programming journal February 2016
Prediction and minimization of blast-induced ground vibration using two robust meta-heuristic algorithms journal February 2017
Prediction of compressive and tensile strength of Gaziantep basalts via neural networks and gene expression programming journal November 2008
Deep Representational Similarity Learning for Analyzing Neural Signatures in Task-based fMRI Dataset journal October 2020
Roadheader performance prediction using genetic programming (GP) and gene expression programming (GEP) techniques journal August 2017
Multidimensional nonlinear diffusion arising in population genetics journal October 1978
Heat transfer to a draining film journal February 1973
Computational design for long-term numerical integration of the equations of fluid motion: Two-dimensional incompressible flow. Part I journal August 1966
Heuristic stability theory for finite-difference equations journal June 1968
Unsteady flow against dispersion in finite porous media journal June 1983
A model unified field equation journal March 1962
Existence of perturbed solitary wave solutions to a model equation for water waves journal September 1988
A simple similarity-transformation-iterative scheme applied to Korteweg–de Vries equation journal February 2006
A modified tanh–coth method for solving the KdV and the KdV–Burgers’ equations journal February 2009
High-order methods for decaying two-dimensional homogeneous isotropic turbulence journal June 2012
Physics of vortex merging journal May 2005
A new synergetic paradigm in environmental numerical modeling: Hybrid models combining deterministic and machine learning components journal January 2006
Nonlinear system identification: From multiple-model networks to Gaussian processes journal October 2008
Parse-matrix evolution for symbolic regression journal September 2012
Adaptive space transformation: An invariant based method for predicting aerodynamic coefficients of hypersonic vehicles journal November 2015
Implicit subgrid-scale modeling by adaptive deconvolution journal November 2004
A coarse-grid projection method for accelerating incompressible flow computations journal January 2013
A novel evolutionary algorithm applied to algebraic modifications of the RANS stress–strain relationship journal November 2016
High order accurate finite difference schemes based on symmetry preservation journal November 2017
Hidden physics models: Machine learning of nonlinear partial differential equations journal March 2018
Application of an evolutionary algorithm to LES modelling of turbulent transport in premixed flames journal December 2018
Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations journal February 2019
Sparse identification of truncation errors journal November 2019
PDE-Net 2.0: Learning PDEs from data with a numeric-symbolic hybrid deep network journal December 2019
Complex hybrid models combining deterministic and machine learning components for numerical climate modeling and weather prediction journal March 2006
Semi-autogenous mill power model development using gene expression programming journal February 2017
Large-eddy simulation: achievements and challenges journal May 1999
New exact travelling wave solutions for the Kawahara and modified Kawahara equations journal January 2004
Closed-loop separation control using machine learning journal April 2015
Sparse reduced-order modelling: sensor-based dynamics to full-state estimation journal April 2018
Subgrid modelling for two-dimensional turbulence using neural networks journal November 2018
Sub-grid scale model classification and blending through deep learning journal May 2019
The critical merger distance between two co-rotating quasi-geostrophic vortices journal January 2005
The perceptron: A probabilistic model for information storage and organization in the brain. journal January 1958
The big challenges of big data journal June 2013
Deep learning journal May 2015
Machine learning identifies a strong association between warming and reduced primary productivity in an oligotrophic ocean gyre journal February 2020
Diffusion Approximation for Two-Dimensional Turbulence journal January 1968
Computation of the Energy Spectrum in Homogeneous Two-Dimensional Turbulence journal January 1969
Inertial Ranges in Two-Dimensional Turbulence journal January 1967
Data-driven deconvolution for large eddy simulations of Kraichnan turbulence journal December 2018
Automated reverse engineering of nonlinear dynamical systems journal June 2007
Sparse dynamics for partial differential equations journal March 2013
Discovering governing equations from data by sparse identification of nonlinear dynamical systems journal March 2016
Dynamic systems identification with Gaussian processes journal December 2005
XLI. On the change of form of long waves advancing in a rectangular canal, and on a new type of long stationary waves journal May 1895
Two-dimensional turbulence journal May 1980
Image restoration: Total variation, wavelet frames, and beyond journal May 2012
Analytic solutions of the Nagumo equation journal January 1992
Learning partial differential equations via data discovery and sparse optimization journal January 2017
Model selection for dynamical systems via sparse regression and information criteria journal August 2017
Evidence for the double cascade scenario in two-dimensional turbulence journal July 2010
Prediction of dynamical systems by symbolic regression journal July 2016
Numerical solution of modified differential equations based on symmetry preservation journal December 2017
Elite bases regression: A real-time algorithm for symbolic regression conference July 2017
A Unified Framework for Sparse Relaxed Regularized Regression: SR3 journal January 2019
Internet of Things Mobile–Air Pollution Monitoring System (IoT-Mobair) journal June 2019
An Active Pulse Transmission Line Simulating Nerve Axon journal October 1962
Compressive Sensing [Lecture Notes] journal August 2007
An Introduction To Compressive Sampling journal March 2008
Neuristor propagation on a tunnel diode loaded transmission line journal January 1963
Inferring Biological Networks by Sparse Identification of Nonlinear Dynamics journal June 2016
Regularization and variable selection via the elastic net journal April 2005
Regression Shrinkage and Selection Via the Lasso journal January 1996
Closed-Loop Turbulence Control: Progress and Challenges journal August 2015
Data-driven discovery of partial differential equations journal April 2017
Distilling Free-Form Natural Laws from Experimental Data journal April 2009
Machine learning: Trends, perspectives, and prospects journal July 2015
Image Restoration: Wavelet Frame Shrinkage, Nonlinear Evolution PDEs, and Beyond journal January 2017
Exact Recovery of Chaotic Systems from Highly Corrupted Data journal January 2017
Numerical Gaussian Processes for Time-Dependent and Nonlinear Partial Differential Equations journal January 2018
Extracting Sparse High-Dimensional Dynamics from Limited Data journal January 2018
Oscillatory Solitary Waves in Dispersive Media journal July 1972
Nonlinear Interaction between Short and Long Capillary-Gravity Waves journal November 1975
Prioritized grammar enumeration: symbolic regression by dynamic programming conference January 2013
Two-Dimensional Turbulence journal January 2012
Scale-Invariance and Turbulence Models for Large-Eddy Simulation journal January 2000
Atmospheric Predictability and Two-Dimensional Turbulence journal March 1971
Some Recent Researches on the Motion of Fluids journal April 1915
General Circulation Experiments with the Primitive Equations: i. the Basic Experiment* journal March 1963
Regularization Paths for Generalized Linear Models via Coordinate Descent journal January 2010
Circulating Proteomic Signature of Early Death in Heart Failure Patients with Reduced Ejection Fraction journal January 2019
Deep Reinforcement Learning framework for Autonomous Driving journal January 2017
Hybrid Reynolds-Averaged/Large-Eddy Simulation Methodology from Symbolic Regression: Formulation and Application journal November 2017
Nonlinear truncation error analysis of finite difference schemes forthe Euler equations journal April 1983
Nonlinear truncation error analysis of finite difference schemes forthe Euler equations conference August 1981
Equation Discovery Using Fast Function Extraction: a Deterministic Symbolic Regression Approach journal June 2019
CFD Julia: A Learning Module Structuring an Introductory Course on Computational Fluid Dynamics journal August 2019
High-order methods for decaying two-dimensional homogeneous isotropic turbulence text January 2012
A coarse-grid projection method for accelerating incompressible flow computations text January 2012
Discovering governing equations from data: Sparse identification of nonlinear dynamical systems text January 2015
Prediction of Dynamical Systems by Symbolic Regression text January 2016
Inferring biological networks by sparse identification of nonlinear dynamics preprint January 2016
Data-driven discovery of partial differential equations preprint January 2016
Sparse reduced-order modeling : Sensor-based dynamics to full-state estimation text January 2017
Data-driven deconvolution for large eddy simulations of Kraichnan turbulence preprint January 2018
PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network text January 2018
Sub-grid scale model classification and blending through deep learning text January 2018
Deep Learning text January 2018
Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence book January 1992
An Introduction to Genetic Algorithms book January 1998

Cited By (1)

An Evolve-Then-Correct Reduced Order Model for Hidden Fluid Dynamics journal April 2020

Related Subjects