DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Model fusion with physics-guided machine learning: Projection-based reduced-order modeling

Abstract

The unprecedented amount of data generated from experiments, field observations, and large-scale numerical simulations at a wide range of spatiotemporal scales has enabled the rapid advancement of data-driven and especially deep learning models in the field of fluid mechanics. Although these methods are proven successful for many applications, there is a grand challenge of improving their generalizability. This is particularly essential when data-driven models are employed within outer-loop applications like optimization. In this work, we put forth a physics-guided machine learning (PGML) framework that leverages the interpretable physics-based model with a deep learning model. Leveraging a concatenated neural network design from multi-modal data sources, the PGML framework is capable of enhancing the generalizability of data-driven models and effectively protects against or inform about the inaccurate predictions resulting from extrapolation. We apply the PGML framework as a novel model fusion approach combining the physics-based Galerkin projection model and long- to short-term memory (LSTM) network for parametric model order reduction of fluid flows. We demonstrate the improved generalizability of the PGML framework against a purely data-driven approach through the injection of physics features into intermediate LSTM layers. Our quantitative analysis shows that the overall model uncertainty can be reduced through the PGMLmore » approach, especially for test data coming from a distribution different than the training data. Moreover, we demonstrate that our approach can be used as an inverse diagnostic tool providing a confidence score associated with models and observations. The proposed framework also allows for multi-fidelity computing by making use of low-fidelity models in the online deployment of quantified data-driven models.« less

Authors:
ORCiD logo [1]; ORCiD logo [1];  [2]; ORCiD logo [3]; ORCiD logo [3]
  1. Oklahoma State University, Stillwater, OK (United States)
  2. University of Nevada, Reno, NV (United States)
  3. Norwegian University of Science and Technology, Trondheim (Norway); SINTEF Energy Research, Trondheim (Norway)
Publication Date:
Research Org.:
Oklahoma State Univ., Stillwater, OK (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
OSTI Identifier:
1853024
Alternate Identifier(s):
OSTI ID: 1804133
Grant/Contract Number:  
SC0019290
Resource Type:
Accepted Manuscript
Journal Name:
Physics of Fluids
Additional Journal Information:
Journal Volume: 33; Journal Issue: 6; Journal ID: ISSN 1070-6631
Publisher:
American Institute of Physics (AIP)
Country of Publication:
United States
Language:
English
Subject:
71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSICS; 42 ENGINEERING; fluid mechanics; computational physics; equations of fluid dynamics; artificial neural networks; vortex dynamics; computational methods; machine learning; Navier Stokes equations

Citation Formats

Pawar, Suraj, San, Omer, Nair, Aditya, Rasheed, Adil, and Kvamsdal, Trond. Model fusion with physics-guided machine learning: Projection-based reduced-order modeling. United States: N. p., 2021. Web. doi:10.1063/5.0053349.
Pawar, Suraj, San, Omer, Nair, Aditya, Rasheed, Adil, & Kvamsdal, Trond. Model fusion with physics-guided machine learning: Projection-based reduced-order modeling. United States. https://doi.org/10.1063/5.0053349
Pawar, Suraj, San, Omer, Nair, Aditya, Rasheed, Adil, and Kvamsdal, Trond. Tue . "Model fusion with physics-guided machine learning: Projection-based reduced-order modeling". United States. https://doi.org/10.1063/5.0053349. https://www.osti.gov/servlets/purl/1853024.
@article{osti_1853024,
title = {Model fusion with physics-guided machine learning: Projection-based reduced-order modeling},
author = {Pawar, Suraj and San, Omer and Nair, Aditya and Rasheed, Adil and Kvamsdal, Trond},
abstractNote = {The unprecedented amount of data generated from experiments, field observations, and large-scale numerical simulations at a wide range of spatiotemporal scales has enabled the rapid advancement of data-driven and especially deep learning models in the field of fluid mechanics. Although these methods are proven successful for many applications, there is a grand challenge of improving their generalizability. This is particularly essential when data-driven models are employed within outer-loop applications like optimization. In this work, we put forth a physics-guided machine learning (PGML) framework that leverages the interpretable physics-based model with a deep learning model. Leveraging a concatenated neural network design from multi-modal data sources, the PGML framework is capable of enhancing the generalizability of data-driven models and effectively protects against or inform about the inaccurate predictions resulting from extrapolation. We apply the PGML framework as a novel model fusion approach combining the physics-based Galerkin projection model and long- to short-term memory (LSTM) network for parametric model order reduction of fluid flows. We demonstrate the improved generalizability of the PGML framework against a purely data-driven approach through the injection of physics features into intermediate LSTM layers. Our quantitative analysis shows that the overall model uncertainty can be reduced through the PGML approach, especially for test data coming from a distribution different than the training data. Moreover, we demonstrate that our approach can be used as an inverse diagnostic tool providing a confidence score associated with models and observations. The proposed framework also allows for multi-fidelity computing by making use of low-fidelity models in the online deployment of quantified data-driven models.},
doi = {10.1063/5.0053349},
journal = {Physics of Fluids},
number = 6,
volume = 33,
place = {United States},
year = {Tue Jun 29 00:00:00 EDT 2021},
month = {Tue Jun 29 00:00:00 EDT 2021}
}

Works referenced in this record:

AutoML: A survey of the state-of-the-art
journal, January 2021


Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks
journal, May 2018

  • Vlachas, Pantelis R.; Byeon, Wonmin; Wan, Zhong Y.
  • Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 474, Issue 2213
  • DOI: 10.1098/rspa.2017.0844

Propagating uncertainties in large-scale hemodynamics models via network uncertainty quantification and reduced-order modeling
journal, January 2020

  • Guzzetti, S.; Mansilla Alvarez, L. A.; Blanco, P. J.
  • Computer Methods in Applied Mechanics and Engineering, Vol. 358
  • DOI: 10.1016/j.cma.2019.112626

Model-Free Prediction of Large Spatiotemporally Chaotic Systems from Data: A Reservoir Computing Approach
journal, January 2018


Non-autoregressive time-series methods for stable parametric reduced-order models
journal, August 2020

  • Maulik, Romit; Lusch, Bethany; Balaprakash, Prasanna
  • Physics of Fluids, Vol. 32, Issue 8
  • DOI: 10.1063/5.0019884

Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
journal, February 2019


Digital Twin: Values, Challenges and Enablers From a Modeling Perspective
journal, January 2020


Turbulence Modeling in the Age of Data
journal, January 2019


Nonlinear information fusion algorithms for data-efficient multi-fidelity modelling
journal, February 2017

  • Perdikaris, P.; Raissi, M.; Damianou, A.
  • Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 473, Issue 2198
  • DOI: 10.1098/rspa.2016.0751

Model Reduction for Flow Analysis and Control
journal, January 2017


Data‐Driven Equation Discovery of Ocean Mesoscale Closures
journal, August 2020

  • Zanna, Laure; Bolton, Thomas
  • Geophysical Research Letters, Vol. 47, Issue 17
  • DOI: 10.1029/2020GL088376

Physics guided machine learning using simplified theories
journal, January 2021

  • Pawar, Suraj; San, Omer; Aksoylu, Burak
  • Physics of Fluids, Vol. 33, Issue 1
  • DOI: 10.1063/5.0038929

Modeling the dynamics of PDE systems with physics-constrained deep auto-regressive networks
journal, February 2020


On Low-Dimensional Galerkin Models for Fluid Flow
journal, June 2000


Complex hybrid models combining deterministic and machine learning components for numerical climate modeling and weather prediction
journal, March 2006


Reduced-order modeling of advection-dominated systems with recurrent neural networks and convolutional autoencoders
journal, March 2021

  • Maulik, Romit; Lusch, Bethany; Balaprakash, Prasanna
  • Physics of Fluids, Vol. 33, Issue 3
  • DOI: 10.1063/5.0039986

Long Short-Term Memory
journal, November 1997


A reduced-order approach to four-dimensional variational data assimilation using proper orthogonal decomposition
journal, January 2007

  • Cao, Yanhua; Zhu, Jiang; Navon, I. M.
  • International Journal for Numerical Methods in Fluids, Vol. 53, Issue 10
  • DOI: 10.1002/fld.1365

Machine-learning-based spatio-temporal super resolution reconstruction of turbulent flows
journal, December 2020

  • Fukami, Kai; Fukagata, Koji; Taira, Kunihiko
  • Journal of Fluid Mechanics, Vol. 909
  • DOI: 10.1017/jfm.2020.948

A Survey of Projection-Based Model Reduction Methods for Parametric Dynamical Systems
journal, January 2015

  • Benner, Peter; Gugercin, Serkan; Willcox, Karen
  • SIAM Review, Vol. 57, Issue 4
  • DOI: 10.1137/130932715

Modal Analysis of Fluid Flows: Applications and Outlook
journal, March 2020

  • Taira, Kunihiko; Hemati, Maziar S.; Brunton, Steven L.
  • AIAA Journal, Vol. 58, Issue 3
  • DOI: 10.2514/1.J058462

Prediction of non-stationary response functions using a Bayesian composite Gaussian process
journal, February 2021

  • Davis, Casey B.; Hans, Christopher M.; Santner, Thomas J.
  • Computational Statistics & Data Analysis, Vol. 154
  • DOI: 10.1016/j.csda.2020.107083

Neural network closures for nonlinear model order reduction
journal, January 2018


Enforcing statistical constraints in generative adversarial networks for modeling chaotic dynamical systems
journal, April 2020


Physics-informed machine learning: case studies for weather and climate modelling
journal, February 2021

  • Kashinath, K.; Mustafa, M.; Albert, A.
  • Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 379, Issue 2194
  • DOI: 10.1098/rsta.2020.0093

Explicit Model Predictive Control for Large-Scale Systems via Model Reduction
journal, July 2008

  • Hovland, Svein; Gravdahl, Jan Tommy; Willcox, Karen E.
  • Journal of Guidance, Control, and Dynamics, Vol. 31, Issue 4
  • DOI: 10.2514/1.33079

Interpolation Method for Adapting Reduced-Order Models and Application to Aeroelasticity
journal, July 2008

  • Amsallem, David; Farhat, Charbel
  • AIAA Journal, Vol. 46, Issue 7
  • DOI: 10.2514/1.35374

Survey of Multifidelity Methods in Uncertainty Propagation, Inference, and Optimization
journal, January 2018

  • Peherstorfer, Benjamin; Willcox, Karen; Gunzburger, Max
  • SIAM Review, Vol. 60, Issue 3
  • DOI: 10.1137/16M1082469

Multilayer feedforward networks are universal approximators
journal, January 1989


Deep learning and process understanding for data-driven Earth system science
journal, February 2019


Theory-Guided Data Science: A New Paradigm for Scientific Discovery from Data
journal, October 2017

  • Karpatne, Anuj; Atluri, Gowtham; Faghmous, James H.
  • IEEE Transactions on Knowledge and Data Engineering, Vol. 29, Issue 10
  • DOI: 10.1109/TKDE.2017.2720168

Reynolds averaged turbulence modelling using deep neural networks with embedded invariance
journal, October 2016

  • Ling, Julia; Kurzawski, Andrew; Templeton, Jeremy
  • Journal of Fluid Mechanics, Vol. 807
  • DOI: 10.1017/jfm.2016.615

Machine Learning for Fluid Mechanics
journal, September 2019


Dynamic Mode Decomposition
book, January 2016

  • Kutz, J. Nathan; Brunton, Steven L.; Brunton, Bingni W.
  • Society for Industrial and Applied Mathematics
  • DOI: 10.1137/1.9781611974508

POD/DEIM reduced-order strategies for efficient four dimensional variational data assimilation
journal, August 2015


A reduced order model based on Kalman filtering for sequential data assimilation of turbulent flows
journal, October 2017


CNN-LSTM based reduced order modeling of two-dimensional unsteady flows around a circular cylinder at different Reynolds numbers
journal, December 2020

  • Hasegawa, Kazuto; Fukami, Kai; Murata, Takaaki
  • Fluid Dynamics Research, Vol. 52, Issue 6
  • DOI: 10.1088/1873-7005/abb91d

Composite Gaussian process models for emulating expensive functions
journal, December 2012

  • Ba, Shan; Joseph, V. Roshan
  • The Annals of Applied Statistics, Vol. 6, Issue 4
  • DOI: 10.1214/12-AOAS570

A Comparison of Some Error Estimates for Neural Network Models
journal, January 1996


Model reduction for compressible flows using POD and Galerkin projection
journal, February 2004

  • Rowley, Clarence W.; Colonius, Tim; Murray, Richard M.
  • Physica D: Nonlinear Phenomena, Vol. 189, Issue 1-2
  • DOI: 10.1016/j.physd.2003.03.001

The Proper Orthogonal Decomposition in the Analysis of Turbulent Flows
journal, January 1993


Model predictive control: Theory and practice—A survey
journal, May 1989


Multi-level convolutional autoencoder networks for parametric prediction of spatio-temporal dynamics
journal, December 2020

  • Xu, Jiayang; Duraisamy, Karthik
  • Computer Methods in Applied Mechanics and Engineering, Vol. 372
  • DOI: 10.1016/j.cma.2020.113379

Cases for the nugget in modeling computer experiments
journal, December 2010


Deep neural networks for nonlinear model order reduction of unsteady flows
journal, October 2020

  • Eivazi, Hamidreza; Veisi, Hadi; Naderi, Mohammad Hossein
  • Physics of Fluids, Vol. 32, Issue 10
  • DOI: 10.1063/5.0020526

Leveraging reduced-order models for state estimation using deep learning
journal, June 2020


Model identification of reduced order fluid dynamics systems using deep learning: Model identification in fluid dynamics using deep learning
journal, August 2017

  • Wang, Z.; Xiao, D.; Fang, F.
  • International Journal for Numerical Methods in Fluids, Vol. 86, Issue 4
  • DOI: 10.1002/fld.4416

Construction of reduced-order models for fluid flows using deep feedforward neural networks
journal, June 2019


Turbulence and the dynamics of coherent structures. I. Coherent structures
journal, January 1987

  • Sirovich, Lawrence
  • Quarterly of Applied Mathematics, Vol. 45, Issue 3
  • DOI: 10.1090/qam/910462

Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders
journal, November 2019


Data-driven reduced order modeling for time-dependent problems
journal, March 2019

  • Guo, Mengwu; Hesthaven, Jan S.
  • Computer Methods in Applied Mechanics and Engineering, Vol. 345
  • DOI: 10.1016/j.cma.2018.10.029

Non-intrusive reduced-order modeling for fluid problems: A brief review
journal, December 2019

  • Yu, Jian; Yan, Chao; Guo, Mengwu
  • Proceedings of the Institution of Mechanical Engineers, Part G: Journal of Aerospace Engineering, Vol. 233, Issue 16
  • DOI: 10.1177/0954410019890721

Additive Gaussian Process for Computer Models With Qualitative and Quantitative Factors
journal, April 2017


Hybrid Nonlinear Variation Modeling of Compliant Metal Plate Assemblies Considering Welding Shrinkage and Angular Distortion
journal, February 2020

  • Liu, Changhui; Liu, Tao; Du, Juan
  • Journal of Manufacturing Science and Engineering, Vol. 142, Issue 4
  • DOI: 10.1115/1.4046250