Model fusion with physics-guided machine learning: Projection-based reduced-order modeling
Abstract
The unprecedented amount of data generated from experiments, field observations, and large-scale numerical simulations at a wide range of spatiotemporal scales has enabled the rapid advancement of data-driven and especially deep learning models in the field of fluid mechanics. Although these methods are proven successful for many applications, there is a grand challenge of improving their generalizability. This is particularly essential when data-driven models are employed within outer-loop applications like optimization. In this work, we put forth a physics-guided machine learning (PGML) framework that leverages the interpretable physics-based model with a deep learning model. Leveraging a concatenated neural network design from multi-modal data sources, the PGML framework is capable of enhancing the generalizability of data-driven models and effectively protects against or inform about the inaccurate predictions resulting from extrapolation. We apply the PGML framework as a novel model fusion approach combining the physics-based Galerkin projection model and long- to short-term memory (LSTM) network for parametric model order reduction of fluid flows. We demonstrate the improved generalizability of the PGML framework against a purely data-driven approach through the injection of physics features into intermediate LSTM layers. Our quantitative analysis shows that the overall model uncertainty can be reduced through the PGMLmore »
- Authors:
-
- Oklahoma State University, Stillwater, OK (United States)
- University of Nevada, Reno, NV (United States)
- Norwegian University of Science and Technology, Trondheim (Norway); SINTEF Energy Research, Trondheim (Norway)
- Publication Date:
- Research Org.:
- Oklahoma State Univ., Stillwater, OK (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
- OSTI Identifier:
- 1853024
- Alternate Identifier(s):
- OSTI ID: 1804133
- Grant/Contract Number:
- SC0019290
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Physics of Fluids
- Additional Journal Information:
- Journal Volume: 33; Journal Issue: 6; Journal ID: ISSN 1070-6631
- Publisher:
- American Institute of Physics (AIP)
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSICS; 42 ENGINEERING; fluid mechanics; computational physics; equations of fluid dynamics; artificial neural networks; vortex dynamics; computational methods; machine learning; Navier Stokes equations
Citation Formats
Pawar, Suraj, San, Omer, Nair, Aditya, Rasheed, Adil, and Kvamsdal, Trond. Model fusion with physics-guided machine learning: Projection-based reduced-order modeling. United States: N. p., 2021.
Web. doi:10.1063/5.0053349.
Pawar, Suraj, San, Omer, Nair, Aditya, Rasheed, Adil, & Kvamsdal, Trond. Model fusion with physics-guided machine learning: Projection-based reduced-order modeling. United States. https://doi.org/10.1063/5.0053349
Pawar, Suraj, San, Omer, Nair, Aditya, Rasheed, Adil, and Kvamsdal, Trond. Tue .
"Model fusion with physics-guided machine learning: Projection-based reduced-order modeling". United States. https://doi.org/10.1063/5.0053349. https://www.osti.gov/servlets/purl/1853024.
@article{osti_1853024,
title = {Model fusion with physics-guided machine learning: Projection-based reduced-order modeling},
author = {Pawar, Suraj and San, Omer and Nair, Aditya and Rasheed, Adil and Kvamsdal, Trond},
abstractNote = {The unprecedented amount of data generated from experiments, field observations, and large-scale numerical simulations at a wide range of spatiotemporal scales has enabled the rapid advancement of data-driven and especially deep learning models in the field of fluid mechanics. Although these methods are proven successful for many applications, there is a grand challenge of improving their generalizability. This is particularly essential when data-driven models are employed within outer-loop applications like optimization. In this work, we put forth a physics-guided machine learning (PGML) framework that leverages the interpretable physics-based model with a deep learning model. Leveraging a concatenated neural network design from multi-modal data sources, the PGML framework is capable of enhancing the generalizability of data-driven models and effectively protects against or inform about the inaccurate predictions resulting from extrapolation. We apply the PGML framework as a novel model fusion approach combining the physics-based Galerkin projection model and long- to short-term memory (LSTM) network for parametric model order reduction of fluid flows. We demonstrate the improved generalizability of the PGML framework against a purely data-driven approach through the injection of physics features into intermediate LSTM layers. Our quantitative analysis shows that the overall model uncertainty can be reduced through the PGML approach, especially for test data coming from a distribution different than the training data. Moreover, we demonstrate that our approach can be used as an inverse diagnostic tool providing a confidence score associated with models and observations. The proposed framework also allows for multi-fidelity computing by making use of low-fidelity models in the online deployment of quantified data-driven models.},
doi = {10.1063/5.0053349},
journal = {Physics of Fluids},
number = 6,
volume = 33,
place = {United States},
year = {Tue Jun 29 00:00:00 EDT 2021},
month = {Tue Jun 29 00:00:00 EDT 2021}
}
Works referenced in this record:
AutoML: A survey of the state-of-the-art
journal, January 2021
- He, Xin; Zhao, Kaiyong; Chu, Xiaowen
- Knowledge-Based Systems, Vol. 212
Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks
journal, May 2018
- Vlachas, Pantelis R.; Byeon, Wonmin; Wan, Zhong Y.
- Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 474, Issue 2213
Propagating uncertainties in large-scale hemodynamics models via network uncertainty quantification and reduced-order modeling
journal, January 2020
- Guzzetti, S.; Mansilla Alvarez, L. A.; Blanco, P. J.
- Computer Methods in Applied Mechanics and Engineering, Vol. 358
Model-Free Prediction of Large Spatiotemporally Chaotic Systems from Data: A Reservoir Computing Approach
journal, January 2018
- Pathak, Jaideep; Hunt, Brian; Girvan, Michelle
- Physical Review Letters, Vol. 120, Issue 2
Non-autoregressive time-series methods for stable parametric reduced-order models
journal, August 2020
- Maulik, Romit; Lusch, Bethany; Balaprakash, Prasanna
- Physics of Fluids, Vol. 32, Issue 8
Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
journal, February 2019
- Raissi, M.; Perdikaris, P.; Karniadakis, G. E.
- Journal of Computational Physics, Vol. 378
Digital Twin: Values, Challenges and Enablers From a Modeling Perspective
journal, January 2020
- Rasheed, Adil; San, Omer; Kvamsdal, Trond
- IEEE Access, Vol. 8
Turbulence Modeling in the Age of Data
journal, January 2019
- Duraisamy, Karthik; Iaccarino, Gianluca; Xiao, Heng
- Annual Review of Fluid Mechanics, Vol. 51, Issue 1
Nonlinear information fusion algorithms for data-efficient multi-fidelity modelling
journal, February 2017
- Perdikaris, P.; Raissi, M.; Damianou, A.
- Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 473, Issue 2198
Model Reduction for Flow Analysis and Control
journal, January 2017
- Rowley, Clarence W.; Dawson, Scott T. M.
- Annual Review of Fluid Mechanics, Vol. 49, Issue 1
Data‐Driven Equation Discovery of Ocean Mesoscale Closures
journal, August 2020
- Zanna, Laure; Bolton, Thomas
- Geophysical Research Letters, Vol. 47, Issue 17
Physics guided machine learning using simplified theories
journal, January 2021
- Pawar, Suraj; San, Omer; Aksoylu, Burak
- Physics of Fluids, Vol. 33, Issue 1
Modeling the dynamics of PDE systems with physics-constrained deep auto-regressive networks
journal, February 2020
- Geneva, Nicholas; Zabaras, Nicholas
- Journal of Computational Physics, Vol. 403
On Low-Dimensional Galerkin Models for Fluid Flow
journal, June 2000
- Rempfer, D.
- Theoretical and Computational Fluid Dynamics, Vol. 14, Issue 2
Complex hybrid models combining deterministic and machine learning components for numerical climate modeling and weather prediction
journal, March 2006
- Krasnopolsky, Vladimir M.; Fox-Rabinovitz, Michael S.
- Neural Networks, Vol. 19, Issue 2
Reduced-order modeling of advection-dominated systems with recurrent neural networks and convolutional autoencoders
journal, March 2021
- Maulik, Romit; Lusch, Bethany; Balaprakash, Prasanna
- Physics of Fluids, Vol. 33, Issue 3
Long Short-Term Memory
journal, November 1997
- Hochreiter, Sepp; Schmidhuber, Jürgen
- Neural Computation, Vol. 9, Issue 8
A reduced-order approach to four-dimensional variational data assimilation using proper orthogonal decomposition
journal, January 2007
- Cao, Yanhua; Zhu, Jiang; Navon, I. M.
- International Journal for Numerical Methods in Fluids, Vol. 53, Issue 10
Machine-learning-based spatio-temporal super resolution reconstruction of turbulent flows
journal, December 2020
- Fukami, Kai; Fukagata, Koji; Taira, Kunihiko
- Journal of Fluid Mechanics, Vol. 909
A Survey of Projection-Based Model Reduction Methods for Parametric Dynamical Systems
journal, January 2015
- Benner, Peter; Gugercin, Serkan; Willcox, Karen
- SIAM Review, Vol. 57, Issue 4
Modal Analysis of Fluid Flows: Applications and Outlook
journal, March 2020
- Taira, Kunihiko; Hemati, Maziar S.; Brunton, Steven L.
- AIAA Journal, Vol. 58, Issue 3
Prediction of non-stationary response functions using a Bayesian composite Gaussian process
journal, February 2021
- Davis, Casey B.; Hans, Christopher M.; Santner, Thomas J.
- Computational Statistics & Data Analysis, Vol. 154
Neural network closures for nonlinear model order reduction
journal, January 2018
- San, Omer; Maulik, Romit
- Advances in Computational Mathematics, Vol. 44, Issue 6
Enforcing statistical constraints in generative adversarial networks for modeling chaotic dynamical systems
journal, April 2020
- Wu, Jin-Long; Kashinath, Karthik; Albert, Adrian
- Journal of Computational Physics, Vol. 406
Physics-informed machine learning: case studies for weather and climate modelling
journal, February 2021
- Kashinath, K.; Mustafa, M.; Albert, A.
- Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 379, Issue 2194
Explicit Model Predictive Control for Large-Scale Systems via Model Reduction
journal, July 2008
- Hovland, Svein; Gravdahl, Jan Tommy; Willcox, Karen E.
- Journal of Guidance, Control, and Dynamics, Vol. 31, Issue 4
Interpolation Method for Adapting Reduced-Order Models and Application to Aeroelasticity
journal, July 2008
- Amsallem, David; Farhat, Charbel
- AIAA Journal, Vol. 46, Issue 7
Survey of Multifidelity Methods in Uncertainty Propagation, Inference, and Optimization
journal, January 2018
- Peherstorfer, Benjamin; Willcox, Karen; Gunzburger, Max
- SIAM Review, Vol. 60, Issue 3
Multilayer feedforward networks are universal approximators
journal, January 1989
- Hornik, Kurt; Stinchcombe, Maxwell; White, Halbert
- Neural Networks, Vol. 2, Issue 5
Deep learning and process understanding for data-driven Earth system science
journal, February 2019
- Reichstein, Markus; Camps-Valls, Gustau; Stevens, Bjorn
- Nature, Vol. 566, Issue 7743
Theory-Guided Data Science: A New Paradigm for Scientific Discovery from Data
journal, October 2017
- Karpatne, Anuj; Atluri, Gowtham; Faghmous, James H.
- IEEE Transactions on Knowledge and Data Engineering, Vol. 29, Issue 10
Reynolds averaged turbulence modelling using deep neural networks with embedded invariance
journal, October 2016
- Ling, Julia; Kurzawski, Andrew; Templeton, Jeremy
- Journal of Fluid Mechanics, Vol. 807
Machine Learning for Fluid Mechanics
journal, September 2019
- Brunton, Steven L.; Noack, Bernd R.; Koumoutsakos, Petros
- Annual Review of Fluid Mechanics, Vol. 52, Issue 1
Dynamic Mode Decomposition
book, January 2016
- Kutz, J. Nathan; Brunton, Steven L.; Brunton, Bingni W.
- Society for Industrial and Applied Mathematics
POD/DEIM reduced-order strategies for efficient four dimensional variational data assimilation
journal, August 2015
- Ştefănescu, R.; Sandu, A.; Navon, I. M.
- Journal of Computational Physics, Vol. 295
A reduced order model based on Kalman filtering for sequential data assimilation of turbulent flows
journal, October 2017
- Meldi, M.; Poux, A.
- Journal of Computational Physics, Vol. 347
CNN-LSTM based reduced order modeling of two-dimensional unsteady flows around a circular cylinder at different Reynolds numbers
journal, December 2020
- Hasegawa, Kazuto; Fukami, Kai; Murata, Takaaki
- Fluid Dynamics Research, Vol. 52, Issue 6
Composite Gaussian process models for emulating expensive functions
journal, December 2012
- Ba, Shan; Joseph, V. Roshan
- The Annals of Applied Statistics, Vol. 6, Issue 4
A Comparison of Some Error Estimates for Neural Network Models
journal, January 1996
- Tibshirani, Robert
- Neural Computation, Vol. 8, Issue 1
Model reduction for compressible flows using POD and Galerkin projection
journal, February 2004
- Rowley, Clarence W.; Colonius, Tim; Murray, Richard M.
- Physica D: Nonlinear Phenomena, Vol. 189, Issue 1-2
The Proper Orthogonal Decomposition in the Analysis of Turbulent Flows
journal, January 1993
- Berkooz, G.; Holmes, P.; Lumley, J. L.
- Annual Review of Fluid Mechanics, Vol. 25, Issue 1
Model predictive control: Theory and practice—A survey
journal, May 1989
- García, Carlos E.; Prett, David M.; Morari, Manfred
- Automatica, Vol. 25, Issue 3
Multi-level convolutional autoencoder networks for parametric prediction of spatio-temporal dynamics
journal, December 2020
- Xu, Jiayang; Duraisamy, Karthik
- Computer Methods in Applied Mechanics and Engineering, Vol. 372
Cases for the nugget in modeling computer experiments
journal, December 2010
- Gramacy, Robert B.; Lee, Herbert K. H.
- Statistics and Computing, Vol. 22, Issue 3
Deep neural networks for nonlinear model order reduction of unsteady flows
journal, October 2020
- Eivazi, Hamidreza; Veisi, Hadi; Naderi, Mohammad Hossein
- Physics of Fluids, Vol. 32, Issue 10
Leveraging reduced-order models for state estimation using deep learning
journal, June 2020
- Nair, Nirmal J.; Goza, Andres
- Journal of Fluid Mechanics, Vol. 897
Model identification of reduced order fluid dynamics systems using deep learning: Model identification in fluid dynamics using deep learning
journal, August 2017
- Wang, Z.; Xiao, D.; Fang, F.
- International Journal for Numerical Methods in Fluids, Vol. 86, Issue 4
Construction of reduced-order models for fluid flows using deep feedforward neural networks
journal, June 2019
- Lui, Hugo F. S.; Wolf, William R.
- Journal of Fluid Mechanics, Vol. 872
Turbulence and the dynamics of coherent structures. I. Coherent structures
journal, January 1987
- Sirovich, Lawrence
- Quarterly of Applied Mathematics, Vol. 45, Issue 3
Backpropagation algorithms and Reservoir Computing in Recurrent Neural Networks for the forecasting of complex spatiotemporal dynamics
journal, June 2020
- Vlachas, P. R.; Pathak, J.; Hunt, B. R.
- Neural Networks, Vol. 126
Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders
journal, November 2019
- Lee, Kookjin; Carlberg, Kevin T.
- Journal of Computational Physics
Data-driven reduced order modeling for time-dependent problems
journal, March 2019
- Guo, Mengwu; Hesthaven, Jan S.
- Computer Methods in Applied Mechanics and Engineering, Vol. 345
Non-intrusive reduced-order modeling for fluid problems: A brief review
journal, December 2019
- Yu, Jian; Yan, Chao; Guo, Mengwu
- Proceedings of the Institution of Mechanical Engineers, Part G: Journal of Aerospace Engineering, Vol. 233, Issue 16
Additive Gaussian Process for Computer Models With Qualitative and Quantitative Factors
journal, April 2017
- Deng, X.; Lin, C. Devon; Liu, K. -W.
- Technometrics, Vol. 59, Issue 3
Hybrid Nonlinear Variation Modeling of Compliant Metal Plate Assemblies Considering Welding Shrinkage and Angular Distortion
journal, February 2020
- Liu, Changhui; Liu, Tao; Du, Juan
- Journal of Manufacturing Science and Engineering, Vol. 142, Issue 4