Time-series learning of latent-space dynamics for reduced-order model closure

Maulik, Romit; Mohan, Arvind Thanam; Lusch, Bethany; Madireddy, Sandeep; Balaprakash, Prasanna; Livescu, Daniel

doi:10.1016/j.physd.2020.132368

Title: Time-series learning of latent-space dynamics for reduced-order model closure

Journal Article · 26 January 2020 · Physica. D, Nonlinear Phenomena

DOI: https://doi.org/10.1016/j.physd.2020.132368 · OSTI ID:1597352

Maulik, Romit ^[1];

^[2]; Lusch, Bethany ^[1]; Madireddy, Sandeep ^[3]; Balaprakash, Prasanna ^[4];

^[2]

Argonne National Lab. (ANL), Argonne, IL (United States). Argonne Leadership Computing Facility (ALCF)
Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
Argonne National Lab. (ANL), Argonne, IL (United States)
Argonne National Lab. (ANL), Argonne, IL (United States). Argonne Leadership Computing Facility (ALCF) and Mathematics and Computer Science Division

In this work, we study the performance of long short-term memory networks (LSTMs) and neural ordinary differential equations (NODEs) in learning latent-space representations of dynamical equations for an advection-dominated problem given by the viscous Burgers equation. Our formulation is devised in a nonintrusive manner with an equation-free evolution of dynamics in a reduced space with the latter being obtained through a proper orthogonal decomposition. In addition, we leverage the sequential nature of learning for both LSTMs and NODEs to demonstrate their capability for closure in systems that are not completely resolved in the reduced space. We assess our hypothesis for two advection-dominated problems given by the viscous Burgers equation. We observe that both LSTMs and NODEs are able to reproduce the effects of the absent scales for our test cases more effectively than does intrusive dynamics evolution through a Galerkin projection. This result empirically suggests that time-series learning techniques implicitly leverage a memory kernel for coarse-grained system closure as is suggested through the Mori–Zwanzig formalism.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Argonne National Laboratory (ANL), Argonne, IL (United States). Argonne Leadership Computing Facility (ALCF); Los Alamos National Lab. (LANL), Los Alamos, NM (United States); Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)

Sponsoring Organization:: USDOE Laboratory Directed Research and Development (LDRD) Program; USDOE National Nuclear Security Administration (NNSA); USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR); USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)

Grant/Contract Number:: 89233218CNA000001; AC02-06CH11357

OSTI ID:: 1597352

Report Number(s):: LA-UR--19-28714

Journal Information:: Physica. D, Nonlinear Phenomena, Journal Name: Physica. D, Nonlinear Phenomena Journal Issue: C Vol. 405; ISSN 0167-2789

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

References (28)

Nonlinear generalized Langevin equations Zwanzig, Robert Journal of Statistical Physics, Vol. 9, Issue 3 https://doi.org/10.1007/bf01008729	journal	November 1973
Principal interval decomposition framework for POD reduced-order modeling of convective Boussinesq flows: PRINCIPAL INTERVAL DECOMPOSITION MODEL REDUCTION FRAMEWORK San, O.; Borggaard, J. International Journal for Numerical Methods in Fluids, Vol. 78, Issue 1 https://doi.org/10.1002/fld.4006	journal	February 2015
An evolve-then-filter regularized reduced order model for convection-dominated flows: EVOLVE-THEN-FILTER REGULARIZED REDUCED ORDER MODEL Wells, D.; Wang, Z.; Xie, X. International Journal for Numerical Methods in Fluids, Vol. 84, Issue 10 https://doi.org/10.1002/fld.4363	journal	February 2017
Physically constrained data‐driven correction for reduced‐order modeling of fluid flows Mohebujjaman, M.; Rebholz, L. G.; Iliescu, T. International Journal for Numerical Methods in Fluids, Vol. 89, Issue 3 https://doi.org/10.1002/fld.4684	journal	October 2018
Efficient non-linear model reduction via a least-squares Petrov-Galerkin projection and compressive tensor approximations Carlberg, Kevin; Bou-Mosleh, Charbel; Farhat, Charbel International Journal for Numerical Methods in Engineering, Vol. 86, Issue 2 https://doi.org/10.1002/nme.3050	journal	October 2010
Supremizer stabilization of POD-Galerkin approximation of parametrized steady incompressible Navier-Stokes equations: SUPREMIZER STABILIZATION OF POD-GALERKIN APPROXIMATION OF NS EQUATIONS Ballarin, Francesco; Manzoni, Andrea; Quarteroni, Alfio International Journal for Numerical Methods in Engineering, Vol. 102, Issue 5 https://doi.org/10.1002/nme.4772	journal	November 2014
Nonlinear generalized Langevin equations Zwanzig, Robert Journal of Statistical Physics, Vol. 9, Issue 3 https://doi.org/10.1007/BF01008729	journal	November 1973
Neural network closures for nonlinear model order reduction San, Omer; Maulik, Romit Advances in Computational Mathematics, Vol. 44, Issue 6 https://doi.org/10.1007/s10444-018-9590-z	journal	January 2018
Explicit and implicit LES closures for Burgers turbulence Maulik, Romit; San, Omer Journal of Computational and Applied Mathematics, Vol. 327 https://doi.org/10.1016/j.cam.2017.06.003	journal	January 2018
Proper orthogonal decomposition closure models for turbulent flows: A numerical comparison Wang, Zhu; Akhtar, Imran; Borggaard, Jeff Computer Methods in Applied Mechanics and Engineering, Vol. 237-240 https://doi.org/10.1016/j.cma.2012.04.015	journal	September 2012
Data-driven reduced order modeling for time-dependent problems Guo, Mengwu; Hesthaven, Jan S. Computer Methods in Applied Mechanics and Engineering, Vol. 345 https://doi.org/10.1016/j.cma.2018.10.029	journal	March 2019
An artificial neural network framework for reduced order modeling of transient flows San, Omer; Maulik, Romit; Ahmed, Mansoor Communications in Nonlinear Science and Numerical Simulation, Vol. 77 https://doi.org/10.1016/j.cnsns.2019.04.025	journal	October 2019
Stationary two-dimensional turbulence statistics using a Markovian forcing scheme San, Omer; Staples, Anne E. Computers & Fluids, Vol. 71 https://doi.org/10.1016/j.compfluid.2012.10.002	journal	January 2013
Deep learning algorithm for data-driven simulation of noisy dynamical system Yeo, Kyongmin; Melnyk, Igor Journal of Computational Physics, Vol. 376 https://doi.org/10.1016/j.jcp.2018.10.024	journal	January 2019
Non-intrusive reduced order modeling of unsteady flows using artificial neural networks with application to a combustion problem Wang, Qian; Hesthaven, Jan S.; Ray, Deep Journal of Computational Physics, Vol. 384 https://doi.org/10.1016/j.jcp.2019.01.031	journal	May 2019
Burgers turbulence Bec, J.; Khanin, K. Physics Reports, Vol. 447, Issue 1-2 https://doi.org/10.1016/j.physrep.2007.04.002	journal	August 2007
Deep learning for universal linear embeddings of nonlinear dynamics Lusch, Bethany; Kutz, J. Nathan; Brunton, Steven L. Nature Communications, Vol. 9, Issue 1 https://doi.org/10.1038/s41467-018-07210-0	journal	November 2018
Statistically accurate low-order models for uncertainty quantification in turbulent dynamical systems Sapsis, Themistoklis P.; Majda, Andrew J. Proceedings of the National Academy of Sciences, Vol. 110, Issue 34 https://doi.org/10.1073/pnas.1313065110	journal	August 2013
Discovering governing equations from data by sparse identification of nonlinear dynamical systems Brunton, Steven L.; Proctor, Joshua L.; Kutz, J. Nathan Proceedings of the National Academy of Sciences, Vol. 113, Issue 15 https://doi.org/10.1073/pnas.1517384113	journal	March 2016
Stable architectures for deep neural networks Haber, Eldad; Ruthotto, Lars Inverse Problems, Vol. 34, Issue 1 https://doi.org/10.1088/1361-6420/aa9a90	journal	December 2017
Extreme learning machine for reduced order modeling of turbulent geophysical flows San, Omer; Maulik, Romit Physical Review E, Vol. 97, Issue 4 https://doi.org/10.1103/PhysRevE.97.042322	journal	April 2018
Data-driven discovery of partial differential equations Rudy, Samuel H.; Brunton, Steven L.; Proctor, Joshua L. Science Advances, Vol. 3, Issue 4 https://doi.org/10.1126/sciadv.1602614	journal	April 2017
Dynamic Mode Decomposition with Control Proctor, Joshua L.; Brunton, Steven L.; Kutz, J. Nathan SIAM Journal on Applied Dynamical Systems, Vol. 15, Issue 1 https://doi.org/10.1137/15M1013857	journal	January 2016
Optimal Model Management for Multifidelity Monte Carlo Estimation Peherstorfer, Benjamin; Willcox, Karen; Gunzburger, Max SIAM Journal on Scientific Computing, Vol. 38, Issue 5 https://doi.org/10.1137/15M1046472	journal	January 2016
Space--Time Least-Squares Petrov--Galerkin Projection for Nonlinear Model Reduction Choi, Youngsoo; Carlberg, Kevin SIAM Journal on Scientific Computing, Vol. 41, Issue 1 https://doi.org/10.1137/17M1120531	journal	January 2019
Data-Driven Filtered Reduced Order Modeling of Fluid Flows Xie, X.; Mohebujjaman, M.; Rebholz, L. G. SIAM Journal on Scientific Computing, Vol. 40, Issue 3 https://doi.org/10.1137/17M1145136	journal	January 2018
Transport, Collective Motion, and Brownian Motion Mori, Hazime Progress of Theoretical Physics, Vol. 33, Issue 3 https://doi.org/10.1143/PTP.33.423	journal	March 1965
The Proper Orthogonal Decomposition in the Analysis of Turbulent Flows Berkooz, G.; Holmes, P.; Lumley, J. L. Annual Review of Fluid Mechanics, Vol. 25, Issue 1 https://doi.org/10.1146/annurev.fl.25.010193.002543	journal	January 1993

Cited By (4)

Turbulence forecasting via Neural ODE Portwood, Gavin D.; Mitra, Peetak P.; Ribeiro, Mateus Dias arXiv https://doi.org/10.48550/arxiv.1911.05180	preprint	January 2019
Meta-modeling strategy for data-driven forecasting Skinner, Dominic J.; Maulik, Romit arXiv https://doi.org/10.48550/arxiv.2012.00678	preprint	January 2020
Neural Closure Models for Dynamical Systems Gupta, Abhinav; Lermusiaux, Pierre F. J. arXiv https://doi.org/10.48550/arxiv.2012.13869	text	January 2020
Stiff Neural Ordinary Differential Equations Kim, Suyong; Ji, Weiqi; Deng, Sili arXiv https://doi.org/10.48550/arxiv.2103.15341	text	January 2021

Similar Records

Nonintrusive projection-based reduced order modeling using stable learned differential operators

Journal Article · 2025 · Computer Methods in Applied Mechanics and Engineering · OSTI ID:2563114

Prakash, Aviral; Zhang, Yongjie Jessica

Reduced-order modeling of advection-dominated systems with recurrent neural networks and convolutional autoencoders

Journal Article · 2021 · Physics of Fluids · OSTI ID:1840551

Maulik, Romit; Lusch, Bethany; Balaprakash, Prasanna

Data-Driven Learning for the Mori--Zwanzig Formalism: A Generalization of the Koopman Learning Framework

Journal Article · 2021 · SIAM Journal on Applied Dynamical Systems · OSTI ID:1872339

Lin, Yen Ting; Tian, Yifeng; Livescu, Daniel; +1 more

Related Subjects

97 MATHEMATICS AND COMPUTING
Information Science
LSTM
LSTMs
Machine Learning
Mathematics
Neural ODE
Neural ODEs
ROMs
closures

Title: Time-series learning of latent-space dynamics for reduced-order model closure

Citation Formats

References (28)

Cited By (4)

Similar Records

Related Subjects