skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders

Journal Article · · Journal of Computational Physics
 [1];  [1]
  1. Sandia National Lab. (SNL-CA), Livermore, CA (United States)

Nearly all model-reduction techniques project the governing equations onto a linear subspace of the original state space. Such subspaces are typically computed using methods such as balanced truncation, rational interpolation, the reduced-basis method, and (balanced) proper orthogonal decomposition (POD). Unfortunately, restricting the state to evolve in a linear subspace imposes a fundamental limitation to the accuracy of the resulting reduced-order model (ROM). In particular, linear-subspace ROMs can be expected to produce low-dimensional models with high accuracy only if the problem admits a fast decaying Kolmogorov n-width (e.g., diffusion-dominated problems). Unfortunately, many problems of interest exhibit a slowly decaying Kolmogorov n-width (e.g., advection-dominated problems). To address this, we propose a novel framework for projecting dynamical systems onto nonlinear manifolds using minimum-residual formulations at the time-continuous and time-discrete levels; the former leads to manifold Galerkin projection, while the latter leads to manifold least-squares Petrov–Galerkin (LSPG) projection. We perform analyses that provide insight into the relationship between these proposed approaches and classical linear-subspace reduced-order models; we also derive a posteriori discrete-time error bounds for the proposed approaches. In addition, we propose a computationally practical approach for computing the nonlinear manifold, which is based on convolutional autoencoders from deep learning. Lastly, we demonstrate the ability of the method to significantly outperform even the optimal linear-subspace ROM on benchmark advection-dominated problems, thereby demonstrating the method's ability to overcome the intrinsic n-width limitations of linear subspaces.

Research Organization:
Sandia National Lab. (SNL-CA), Livermore, CA (United States)
Sponsoring Organization:
Sandia's Advanced Simulation and Computing (ASC) Verification and Validation (V&V) Project; USDOE National Nuclear Security Administration (NNSA)
Grant/Contract Number:
AC04-94AL85000; NA0003525
OSTI ID:
1574441
Alternate ID(s):
OSTI ID: 1691627
Report Number(s):
SAND-2019-12375J; SAND-2019-0003J; 680333; TRN: US2001261
Journal Information:
Journal of Computational Physics, Vol. 404; ISSN 0021-9991
Publisher:
ElsevierCopyright Statement
Country of Publication:
United States
Language:
English
Citation Metrics:
Cited by: 185 works
Citation information provided by
Web of Science

References (43)

Model reduction using L 1 -norm minimization as an application to nonlinear hyperbolic problems: Model reduction for hyperbolic problems journal April 2018
Nonlinear model order reduction based on local reduced-order bases: NONLINEAR MODEL REDUCTION BASED ON LOCAL REDUCED-ORDER BASES
  • Amsallem, David; Zahr, Matthew J.; Farhat, Charbel
  • International Journal for Numerical Methods in Engineering, Vol. 92, Issue 10 https://doi.org/10.1002/nme.4371
journal June 2012
Kolmogorov widths and low-rank approximations of parametric elliptic PDEs journal July 2016
Interpolatory Projection Methods for Parameterized Model Reduction journal January 2011
Laplacian Eigenmaps for Dimensionality Reduction and Data Representation journal June 2003
Convergence Rates for Greedy Algorithms in Reduced Basis Methods journal January 2011
Optimization Methods for Large-Scale Machine Learning journal January 2018
Adaptive h -refinement for reduced-order models : ADAPTIVE journal November 2014
Galerkin v. least-squares Petrov–Galerkin projection in nonlinear model reduction journal February 2017
Efficient non-linear model reduction via a least-squares Petrov-Galerkin projection and compressive tensor approximations
  • Carlberg, Kevin; Bou-Mosleh, Charbel; Farhat, Charbel
  • International Journal for Numerical Methods in Engineering, Vol. 86, Issue 2 https://doi.org/10.1002/nme.3050
journal October 2010
Conservative model reduction for finite-volume models journal October 2018
A low-cost, goal-oriented ‘compact proper orthogonal decomposition’ basis for model reduction of static systems journal December 2010
The GNAT method for nonlinear model reduction: Effective implementation and application to computational fluid dynamics and turbulent flows journal June 2013
Decreasing the temporal complexity for nonlinear, implicit reduced-order models by forecasting journal June 2015
Coupling of substructures for dynamic analyses. journal July 1968
Model order reduction for parametrized nonlinear hyperbolic problems as an application to uncertainty quantification journal March 2019
Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data journal April 2003
Approximated Lax pairs for the reduced order integration of nonlinear evolution equations journal May 2014
$\mathcal{H}_2$ Model Reduction for Large-Scale Linear Dynamical Systems journal January 2008
Reducing the Dimensionality of Data with Neural Networks journal July 2006
Analysis of a complex of statistical variables into principal components. journal January 1933
Advection modes by optimal mass transfer journal February 2014
Self-organized formation of topologically correct feature maps journal January 1982
A subspace approach to balanced truncation for model reduction of nonlinear control systems
  • Lall, Sanjay; Marsden, Jerrold E.; Glava?ki, Sonja
  • International Journal of Robust and Nonlinear Control, Vol. 12, Issue 6 https://doi.org/10.1002/rnc.657
journal January 2002
Deep learning journal May 2015
Gradient-based learning applied to document recognition journal January 1998
Practical quasi-Newton methods for solving nonlinear systems journal December 2000
Neural Network Modeling for Near Wall Turbulent Flow journal October 2002
Principal component analysis in linear systems: Controllability, observability, and model reduction journal February 1981
Nonlinear reduced basis approximation of parameterized evolution equations via the method of freezing journal December 2013
Online Adaptive Model Reduction for Nonlinear Systems via Low-Rank Updates journal January 2015
Reliable Real-Time Solution of Parametrized Partial Differential Equations: Reduced-Basis Output Bound Methods journal November 2001
Nonlinear Dimensionality Reduction by Locally Linear Embedding journal December 2000
Model Reduction for Fluids, Using Balanced Proper Orthogonal Decomposition journal March 2005
Reduced basis approximation and a posteriori error estimation for affinely parametrized elliptic coercive partial differential equations journal September 2007
Learning representations by back-propagating errors journal October 1986
Nonlinear Component Analysis as a Kernel Eigenvalue Problem journal July 1998
Reduced basis techniques for nonlinear conservation laws journal April 2015
A Global Geometric Framework for Nonlinear Dimensionality Reduction journal December 2000
Interpolation of Functions with Parameter Dependent Jumps by Transformed Snapshots journal January 2017
Balanced Model Reduction via the Proper Orthogonal Decomposition journal November 2002
Progressive construction of a parametric reduced-order model for PDE-constrained optimization: PDE-CONSTRAINED OPTIMIZATION USING A PROGRESSIVELY CONSTRUCTED ROM journal December 2014
Deep learning for universal linear embeddings of nonlinear dynamics text January 2017

Cited By (38)

A framework for self‐evolving computational material models inspired by deep learning journal August 2019
An autoencoder‐based reduced‐order model for eigenvalue problems with application to neutron diffusion
  • Phillips, Toby R. F.; Heaney, Claire E.; Smith, Paul N.
  • International Journal for Numerical Methods in Engineering, Vol. 122, Issue 15 https://doi.org/10.1002/nme.6681
journal May 2021
A priori analysis on deep learning of subgrid-scale parameterizations for Kraichnan turbulence journal January 2020
Inadequacy of Linear Methods for Minimal Sensor Placement and Feature Selection in Nonlinear Systems: A New Approach Using Secants journal August 2022
Registration-Based Model Reduction in Complex Two-Dimensional Geometries journal August 2021
Reduced order modeling for parameterized time-dependent PDEs using spatially and memory aware deep learning journal July 2021
Image-based model predictive control via dynamic mode decomposition journal August 2021
Latent-space time evolution of non-intrusive reduced-order models using Gaussian process emulation journal February 2021
Parameterized neural ordinary differential equations: applications to computational physics problems journal September 2021
Manifold Approximations via Transported Subspaces: Model Reduction for Transport-Dominated Problems journal February 2023
Deploying deep learning in OpenFOAM with TensorFlow conference January 2021
An Evolve-Then-Correct Reduced Order Model for Hidden Fluid Dynamics journal April 2020
Data-driven discovery of coordinates and governing equations text January 2019
Pressio: Enabling projection-based model reduction for large-scale nonlinear dynamical systems preprint January 2020
Model Reduction for Advection Dominated Hyperbolic Problems in an ALE Framework: Offline and Online Phases preprint January 2020
Numerical Solution of the Parametric Diffusion Equation by Deep Neural Networks preprint January 2020
The Random Feature Model for Input-Output Maps between Banach Spaces text January 2020
Reduced order modeling of fluid flows: Machine learning, Kolmogorov barrier, closure modeling, and partitioning preprint January 2020
Physics-aware registration based auto-encoder for convection dominated PDEs text January 2020
Expressivity of Deep Neural Networks preprint January 2020
A new data assimilation method of recovering turbulent flow field at high-Reynolds numbers for turbulence machine learning preprint January 2020
Depth separation for reduced deep networks in nonlinear model reduction: Distilling shock waves in nonlinear hyperbolic problems preprint January 2020
Data-Driven Snapshot Calibration via Monotonic Feature Matching preprint January 2020
Reduced order models for the incompressible Navier-Stokes equations on collocated grids using a 'discretize-then-project' approach preprint January 2020
Efficient nonlinear manifold reduced order model preprint January 2020
Efficient space-time reduced order model for linear dynamical systems in Python using less than 120 lines of code preprint January 2020
Preserving general physical properties in model reduction of dynamical systems via constrained-optimization projection text January 2020
Reduced Order Models for the Quasi-Geostrophic Equations: A Brief Survey preprint January 2020
DPM: A Novel Training Method for Physics-Informed Neural Networks in Extrapolation preprint January 2020
Variational Autoencoders for Learning Nonlinear Dynamics of Physical Systems preprint January 2020
Neural Dynamic Mode Decomposition for End-to-End Modeling of Nonlinear Dynamics preprint January 2020
Learning emergent PDEs in a learned emergent space preprint January 2020
Physics-aware, probabilistic model order reduction with guaranteed stability preprint January 2021
Meta-Learning for Koopman Spectral Analysis with Short Time-series preprint January 2021
Multifidelity Ensemble Kalman Filtering Using Surrogate Models Defined by Physics-Informed Autoencoders preprint January 2021
A Deep Learning approach to Reduced Order Modelling of Parameter Dependent Partial Differential Equations text January 2021
Physics-informed cluster analysis and a priori efficiency criterion for the construction of local reduced-order bases text January 2021
Deep Kernel Learning of Dynamical Models from High-Dimensional Noisy Data text January 2022

Similar Records

A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder
Journal Article · Fri Nov 12 00:00:00 EST 2021 · Journal of Computational Physics · OSTI ID:1574441

Nonlinear manifold-based reduced order model
Technical Report · Tue Sep 29 00:00:00 EDT 2020 · OSTI ID:1574441

Preconditioned least‐squares Petrov–Galerkin reduced order models
Journal Article · Wed Jun 29 00:00:00 EDT 2022 · International Journal for Numerical Methods in Engineering · OSTI ID:1574441