DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders

Abstract

Nearly all model-reduction techniques project the governing equations onto a linear subspace of the original state space. Such subspaces are typically computed using methods such as balanced truncation, rational interpolation, the reduced-basis method, and (balanced) proper orthogonal decomposition (POD). Unfortunately, restricting the state to evolve in a linear subspace imposes a fundamental limitation to the accuracy of the resulting reduced-order model (ROM). In particular, linear-subspace ROMs can be expected to produce low-dimensional models with high accuracy only if the problem admits a fast decaying Kolmogorov n-width (e.g., diffusion-dominated problems). Unfortunately, many problems of interest exhibit a slowly decaying Kolmogorov n-width (e.g., advection-dominated problems). To address this, we propose a novel framework for projecting dynamical systems onto nonlinear manifolds using minimum-residual formulations at the time-continuous and time-discrete levels; the former leads to manifold Galerkin projection, while the latter leads to manifold least-squares Petrov–Galerkin (LSPG) projection. We perform analyses that provide insight into the relationship between these proposed approaches and classical linear-subspace reduced-order models; we also derive a posteriori discrete-time error bounds for the proposed approaches. In addition, we propose a computationally practical approach for computing the nonlinear manifold, which is based on convolutional autoencoders from deep learning. Lastly, we demonstrate themore » ability of the method to significantly outperform even the optimal linear-subspace ROM on benchmark advection-dominated problems, thereby demonstrating the method's ability to overcome the intrinsic n-width limitations of linear subspaces.« less

Authors:
 [1];  [1]
  1. Sandia National Lab. (SNL-CA), Livermore, CA (United States)
Publication Date:
Research Org.:
Sandia National Lab. (SNL-CA), Livermore, CA (United States)
Sponsoring Org.:
Sandia's Advanced Simulation and Computing (ASC) Verification and Validation (V&V) Project; USDOE National Nuclear Security Administration (NNSA)
OSTI Identifier:
1574441
Alternate Identifier(s):
OSTI ID: 1691627
Report Number(s):
SAND-2019-12375J; SAND-2019-0003J
Journal ID: ISSN 0021-9991; 680333; TRN: US2001261
Grant/Contract Number:  
AC04-94AL85000; NA0003525
Resource Type:
Accepted Manuscript
Journal Name:
Journal of Computational Physics
Additional Journal Information:
Journal Volume: 404; Journal ID: ISSN 0021-9991
Publisher:
Elsevier
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; Model reduction; Deep learning; Autoencoders; Machine learning; Nonlinear manifolds; Optimal projection

Citation Formats

Lee, Kookjin, and Carlberg, Kevin T. Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders. United States: N. p., 2019. Web. doi:10.1016/j.jcp.2019.108973.
Lee, Kookjin, & Carlberg, Kevin T. Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders. United States. https://doi.org/10.1016/j.jcp.2019.108973
Lee, Kookjin, and Carlberg, Kevin T. Fri . "Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders". United States. https://doi.org/10.1016/j.jcp.2019.108973. https://www.osti.gov/servlets/purl/1574441.
@article{osti_1574441,
title = {Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders},
author = {Lee, Kookjin and Carlberg, Kevin T.},
abstractNote = {Nearly all model-reduction techniques project the governing equations onto a linear subspace of the original state space. Such subspaces are typically computed using methods such as balanced truncation, rational interpolation, the reduced-basis method, and (balanced) proper orthogonal decomposition (POD). Unfortunately, restricting the state to evolve in a linear subspace imposes a fundamental limitation to the accuracy of the resulting reduced-order model (ROM). In particular, linear-subspace ROMs can be expected to produce low-dimensional models with high accuracy only if the problem admits a fast decaying Kolmogorov n-width (e.g., diffusion-dominated problems). Unfortunately, many problems of interest exhibit a slowly decaying Kolmogorov n-width (e.g., advection-dominated problems). To address this, we propose a novel framework for projecting dynamical systems onto nonlinear manifolds using minimum-residual formulations at the time-continuous and time-discrete levels; the former leads to manifold Galerkin projection, while the latter leads to manifold least-squares Petrov–Galerkin (LSPG) projection. We perform analyses that provide insight into the relationship between these proposed approaches and classical linear-subspace reduced-order models; we also derive a posteriori discrete-time error bounds for the proposed approaches. In addition, we propose a computationally practical approach for computing the nonlinear manifold, which is based on convolutional autoencoders from deep learning. Lastly, we demonstrate the ability of the method to significantly outperform even the optimal linear-subspace ROM on benchmark advection-dominated problems, thereby demonstrating the method's ability to overcome the intrinsic n-width limitations of linear subspaces.},
doi = {10.1016/j.jcp.2019.108973},
journal = {Journal of Computational Physics},
number = ,
volume = 404,
place = {United States},
year = {2019},
month = {11}
}

Journal Article:

Citation Metrics:
Cited by: 10 works
Citation information provided by
Web of Science

Save / Share:

Works referenced in this record:

Model reduction using L 1 -norm minimization as an application to nonlinear hyperbolic problems: Model reduction for hyperbolic problems
journal, April 2018

  • Abgrall, R.; Crisovan, R.
  • International Journal for Numerical Methods in Fluids, Vol. 87, Issue 12
  • DOI: 10.1002/fld.4507

Nonlinear model order reduction based on local reduced-order bases: NONLINEAR MODEL REDUCTION BASED ON LOCAL REDUCED-ORDER BASES
journal, June 2012

  • Amsallem, David; Zahr, Matthew J.; Farhat, Charbel
  • International Journal for Numerical Methods in Engineering, Vol. 92, Issue 10
  • DOI: 10.1002/nme.4371

Kolmogorov widths and low-rank approximations of parametric elliptic PDEs
journal, July 2016

  • Bachmayr, Markus; Cohen, Albert
  • Mathematics of Computation, Vol. 86, Issue 304
  • DOI: 10.1090/mcom/3132

Interpolatory Projection Methods for Parameterized Model Reduction
journal, January 2011

  • Baur, Ulrike; Beattie, Christopher; Benner, Peter
  • SIAM Journal on Scientific Computing, Vol. 33, Issue 5
  • DOI: 10.1137/090776925

Laplacian Eigenmaps for Dimensionality Reduction and Data Representation
journal, June 2003


Convergence Rates for Greedy Algorithms in Reduced Basis Methods
journal, January 2011

  • Binev, Peter; Cohen, Albert; Dahmen, Wolfgang
  • SIAM Journal on Mathematical Analysis, Vol. 43, Issue 3
  • DOI: 10.1137/100795772

Optimization Methods for Large-Scale Machine Learning
journal, January 2018

  • Bottou, Léon; Curtis, Frank E.; Nocedal, Jorge
  • SIAM Review, Vol. 60, Issue 2
  • DOI: 10.1137/16M1080173

Adaptive h -refinement for reduced-order models : ADAPTIVE
journal, November 2014

  • Carlberg, Kevin
  • International Journal for Numerical Methods in Engineering, Vol. 102, Issue 5
  • DOI: 10.1002/nme.4800

Galerkin v. least-squares Petrov–Galerkin projection in nonlinear model reduction
journal, February 2017


Efficient non-linear model reduction via a least-squares Petrov-Galerkin projection and compressive tensor approximations
journal, October 2010

  • Carlberg, Kevin; Bou-Mosleh, Charbel; Farhat, Charbel
  • International Journal for Numerical Methods in Engineering, Vol. 86, Issue 2
  • DOI: 10.1002/nme.3050

Conservative model reduction for finite-volume models
journal, October 2018

  • Carlberg, Kevin; Choi, Youngsoo; Sargsyan, Syuzanna
  • Journal of Computational Physics, Vol. 371
  • DOI: 10.1016/j.jcp.2018.05.019

A low-cost, goal-oriented ‘compact proper orthogonal decomposition’ basis for model reduction of static systems
journal, December 2010

  • Carlberg, Kevin; Farhat, Charbel
  • International Journal for Numerical Methods in Engineering, Vol. 86, Issue 3
  • DOI: 10.1002/nme.3074

The GNAT method for nonlinear model reduction: Effective implementation and application to computational fluid dynamics and turbulent flows
journal, June 2013

  • Carlberg, Kevin; Farhat, Charbel; Cortial, Julien
  • Journal of Computational Physics, Vol. 242
  • DOI: 10.1016/j.jcp.2013.02.028

Decreasing the temporal complexity for nonlinear, implicit reduced-order models by forecasting
journal, June 2015

  • Carlberg, Kevin; Ray, Jaideep; van Bloemen Waanders, Bart
  • Computer Methods in Applied Mechanics and Engineering, Vol. 289
  • DOI: 10.1016/j.cma.2015.02.013

Coupling of substructures for dynamic analyses.
journal, July 1968

  • Craig, Roy R.; Bampton, Mervyn C. C.
  • AIAA Journal, Vol. 6, Issue 7
  • DOI: 10.2514/3.4741

Model order reduction for parametrized nonlinear hyperbolic problems as an application to uncertainty quantification
journal, March 2019

  • Crisovan, R.; Torlo, D.; Abgrall, R.
  • Journal of Computational and Applied Mathematics, Vol. 348
  • DOI: 10.1016/j.cam.2018.09.018

Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data
journal, April 2003

  • Donoho, D. L.; Grimes, C.
  • Proceedings of the National Academy of Sciences, Vol. 100, Issue 10
  • DOI: 10.1073/pnas.1031596100

Approximated Lax pairs for the reduced order integration of nonlinear evolution equations
journal, May 2014


$\mathcal{H}_2$ Model Reduction for Large-Scale Linear Dynamical Systems
journal, January 2008

  • Gugercin, S.; Antoulas, A. C.; Beattie, C.
  • SIAM Journal on Matrix Analysis and Applications, Vol. 30, Issue 2
  • DOI: 10.1137/060666123

Reducing the Dimensionality of Data with Neural Networks
journal, July 2006


Analysis of a complex of statistical variables into principal components.
journal, January 1933

  • Hotelling, H.
  • Journal of Educational Psychology, Vol. 24, Issue 6
  • DOI: 10.1037/h0071325

Advection modes by optimal mass transfer
journal, February 2014


Self-organized formation of topologically correct feature maps
journal, January 1982


A subspace approach to balanced truncation for model reduction of nonlinear control systems
journal, January 2002

  • Lall, Sanjay; Marsden, Jerrold E.; Glava?ki, Sonja
  • International Journal of Robust and Nonlinear Control, Vol. 12, Issue 6
  • DOI: 10.1002/rnc.657

Deep learning
journal, May 2015

  • LeCun, Yann; Bengio, Yoshua; Hinton, Geoffrey
  • Nature, Vol. 521, Issue 7553
  • DOI: 10.1038/nature14539

Gradient-based learning applied to document recognition
journal, January 1998

  • Lecun, Y.; Bottou, L.; Bengio, Y.
  • Proceedings of the IEEE, Vol. 86, Issue 11
  • DOI: 10.1109/5.726791

Practical quasi-Newton methods for solving nonlinear systems
journal, December 2000


Neural Network Modeling for Near Wall Turbulent Flow
journal, October 2002

  • Milano, Michele; Koumoutsakos, Petros
  • Journal of Computational Physics, Vol. 182, Issue 1
  • DOI: 10.1006/jcph.2002.7146

Principal component analysis in linear systems: Controllability, observability, and model reduction
journal, February 1981


Nonlinear reduced basis approximation of parameterized evolution equations via the method of freezing
journal, December 2013


Online Adaptive Model Reduction for Nonlinear Systems via Low-Rank Updates
journal, January 2015

  • Peherstorfer, Benjamin; Willcox, Karen
  • SIAM Journal on Scientific Computing, Vol. 37, Issue 4
  • DOI: 10.1137/140989169

Reliable Real-Time Solution of Parametrized Partial Differential Equations: Reduced-Basis Output Bound Methods
journal, November 2001

  • Prud’homme, C.; Rovas, D. V.; Veroy, K.
  • Journal of Fluids Engineering, Vol. 124, Issue 1
  • DOI: 10.1115/1.1448332

The Shifted Proper Orthogonal Decomposition: A Mode Decomposition for Multiple Transport Phenomena
journal, January 2018

  • Reiss, J.; Schulze, P.; Sesterhenn, J.
  • SIAM Journal on Scientific Computing, Vol. 40, Issue 3
  • DOI: 10.1137/17M1140571

Nonlinear Dimensionality Reduction by Locally Linear Embedding
journal, December 2000


Model Reduction for Fluids, Using Balanced Proper Orthogonal Decomposition
journal, March 2005


Reduced basis approximation and a posteriori error estimation for affinely parametrized elliptic coercive partial differential equations
journal, September 2007

  • Rozza, G.; Huynh, D. B. P.; Patera, A. T.
  • Archives of Computational Methods in Engineering, Vol. 15, Issue 3
  • DOI: 10.1007/BF03024948

Learning representations by back-propagating errors
journal, October 1986

  • Rumelhart, David E.; Hinton, Geoffrey E.; Williams, Ronald J.
  • Nature, Vol. 323, Issue 6088
  • DOI: 10.1038/323533a0

Nonlinear Component Analysis as a Kernel Eigenvalue Problem
journal, July 1998

  • Schölkopf, Bernhard; Smola, Alexander; Müller, Klaus-Robert
  • Neural Computation, Vol. 10, Issue 5
  • DOI: 10.1162/089976698300017467

Reduced basis techniques for nonlinear conservation laws
journal, April 2015

  • Taddei, T.; Perotto, S.; Quarteroni, A.
  • ESAIM: Mathematical Modelling and Numerical Analysis, Vol. 49, Issue 3
  • DOI: 10.1051/m2an/2014054

A Global Geometric Framework for Nonlinear Dimensionality Reduction
journal, December 2000


Interpolation of Functions with Parameter Dependent Jumps by Transformed Snapshots
journal, January 2017

  • Welper, G.
  • SIAM Journal on Scientific Computing, Vol. 39, Issue 4
  • DOI: 10.1137/16M1059904

Balanced Model Reduction via the Proper Orthogonal Decomposition
journal, November 2002

  • Willcox, K.; Peraire, J.
  • AIAA Journal, Vol. 40, Issue 11
  • DOI: 10.2514/2.1570

Progressive construction of a parametric reduced-order model for PDE-constrained optimization: PDE-CONSTRAINED OPTIMIZATION USING A PROGRESSIVELY CONSTRUCTED ROM
journal, December 2014

  • Zahr, Matthew J.; Farhat, Charbel
  • International Journal for Numerical Methods in Engineering, Vol. 102, Issue 5
  • DOI: 10.1002/nme.4770

Works referencing / citing this record:

A framework for self‐evolving computational material models inspired by deep learning
journal, August 2019

  • Cho, In Ho
  • International Journal for Numerical Methods in Engineering, Vol. 120, Issue 10
  • DOI: 10.1002/nme.6177