A Preconditioned Low-Rank Projection Method with a Rank-Reduction Scheme for Stochastic Partial Differential Equations

Lee, Kookjin; Elman, Howard C.

doi:10.1137/16M1075582

Title: A Preconditioned Low-Rank Projection Method with a Rank-Reduction Scheme for Stochastic Partial Differential Equations

Abstract

Herein, we consider the numerical solution of large systems of linear equations obtained from the stochastic Galerkin formulation of stochastic partial differential equations. We propose an iterative algorithm that exploits the Kronecker product structure of the linear systems. The proposed algorithm efficiently approximates the solutions in low-rank tensor format. Using standard Krylov subspace methods for the data in tensor format is computationally prohibitive due to the rapid growth of tensor ranks during the iterations. To keep tensor ranks low over the entire iteration process, we devise a rank-reduction scheme that can be combined with the iterative algorithm. The proposed rank-reduction scheme identifies an important subspace in the stochastic domain and compresses tensors of high rank on-the-fly during the iterations. The proposed reduction scheme is a coarse-grid method in that the important subspace can be identified inexpensively in a coarse spatial grid setting. The efficiency of the present method is displayed by numerical experiments on benchmark problems.

Authors:

Lee, Kookjin ^[1]; Elman, Howard C. ^[1]

Univ. of Maryland, College Park, MD (United States)

Publication Date:: Thu Oct 26 00:00:00 EDT 2017

Research Org.:: Univ. of Maryland, College Park, MD (United States)

Sponsoring Org.:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR); National Science Foundation (NSF)

OSTI Identifier:: 1598333

Grant/Contract Number:: SC0009301; DMS-1418754

Resource Type:: Accepted Manuscript

Journal Name:: SIAM Journal on Scientific Computing

Additional Journal Information:: Journal Volume: 39; Journal Issue: 5; Journal ID: ISSN 1064-8275

Publisher:: SIAM

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING; low-rank approximation; tensor format; stochastic Galerkin method; finite elements; GMRES; preconditioning; algebraic multigrid

Citation Formats


                    Lee, Kookjin, and Elman, Howard C. A Preconditioned Low-Rank Projection Method with a Rank-Reduction Scheme for Stochastic Partial Differential Equations.  United States: N. p., 2017. 
Web.  doi:10.1137/16M1075582.

Copy to clipboard


                    Lee, Kookjin, & Elman, Howard C. A Preconditioned Low-Rank Projection Method with a Rank-Reduction Scheme for Stochastic Partial Differential Equations.  United States.  https://doi.org/10.1137/16M1075582

Copy to clipboard


                    Lee, Kookjin, and Elman, Howard C. Thu .  
"A Preconditioned Low-Rank Projection Method with a Rank-Reduction Scheme for Stochastic Partial Differential Equations".  United States.  https://doi.org/10.1137/16M1075582.  https://www.osti.gov/servlets/purl/1598333.

Copy to clipboard


                    
@article{osti_1598333,

  title        = {A Preconditioned Low-Rank Projection Method with a Rank-Reduction Scheme for Stochastic Partial Differential Equations},

  author       = {Lee, Kookjin and Elman, Howard C.},

  abstractNote = {Herein, we consider the numerical solution of large systems of linear equations obtained from the stochastic Galerkin formulation of stochastic partial differential equations. We propose an iterative algorithm that exploits the Kronecker product structure of the linear systems. The proposed algorithm efficiently approximates the solutions in low-rank tensor format. Using standard Krylov subspace methods for the data in tensor format is computationally prohibitive due to the rapid growth of tensor ranks during the iterations. To keep tensor ranks low over the entire iteration process, we devise a rank-reduction scheme that can be combined with the iterative algorithm. The proposed rank-reduction scheme identifies an important subspace in the stochastic domain and compresses tensors of high rank on-the-fly during the iterations. The proposed reduction scheme is a coarse-grid method in that the important subspace can be identified inexpensively in a coarse spatial grid setting. The efficiency of the present method is displayed by numerical experiments on benchmark problems.},

  doi          = {10.1137/16M1075582},

  journal      = {SIAM Journal on Scientific Computing},

  number       = 5,

  volume       = 39,

  place        = {United States},

  year         = {Thu Oct 26 00:00:00 EDT 2017},

  month        = {Thu Oct 26 00:00:00 EDT 2017}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1137/16M1075582

Other availability

Search WorldCat to find libraries that may hold this journal

Citation Metrics:

Cited by: 12 works

Citation information provided by
Web of Science

Save / Share:

Export Metadata

Save to My Library

Works referenced in this record:

On solving elliptic stochastic partial differential equations
journal, August 2002

Babuška, Ivo; Chatzipantelidis, Panagiotis
Computer Methods in Applied Mechanics and Engineering, Vol. 191, Issue 37-38
DOI: 10.1016/S0045-7825(02)00354-7

Galerkin Finite Element Approximations of Stochastic Elliptic Partial Differential Equations
journal, January 2004

Babuska, Ivo; Tempone, Raúl; Zouraris, Georgios E.
SIAM Journal on Numerical Analysis, Vol. 42, Issue 2
DOI: 10.1137/S0036142902418680

Kolmogorov widths and low-rank approximations of parametric elliptic PDEs
journal, July 2016

Bachmayr, Markus; Cohen, Albert
Mathematics of Computation, Vol. 86, Issue 304
DOI: 10.1090/mcom/3132

A projection method to solve linear systems in tensor format: A PROJECTION METHOD TO SOLVE LINEAR SYSTEMS IN TENSOR FORMAT
journal, January 2012

Ballani, Jonas; Grasedyck, Lars
Numerical Linear Algebra with Applications, Vol. 20, Issue 1
DOI: 10.1002/nla.1818

Low-Rank Solution of Unsteady Diffusion Equations with Stochastic Coefficients
journal, January 2015

Benner, Peter; Onwunta, Akwum; Stoll, Martin
SIAM/ASA Journal on Uncertainty Quantification, Vol. 3, Issue 1
DOI: 10.1137/130937251

Algorithms for Numerical Analysis in High Dimensions
journal, January 2005

Beylkin, Gregory; Mohlenkamp, Martin J.
SIAM Journal on Scientific Computing, Vol. 26, Issue 6
DOI: 10.1137/040604959

Streamline upwind/Petrov-Galerkin formulations for convection dominated flows with particular emphasis on the incompressible Navier-Stokes equations
journal, September 1982

Brooks, Alexander N.; Hughes, Thomas J. R.
Computer Methods in Applied Mechanics and Engineering, Vol. 32, Issue 1-3
DOI: 10.1016/0045-7825(82)90071-8

Active Subspace Methods in Theory and Practice: Applications to Kriging Surfaces
journal, January 2014

Constantine, Paul G.; Dow, Eric; Wang, Qiqi
SIAM Journal on Scientific Computing, Vol. 36, Issue 4
DOI: 10.1137/130916138

Stochastic model reduction for chaos representations
journal, August 2007

Doostan, Alireza; Ghanem, Roger G.; Red-Horse, John
Computer Methods in Applied Mechanics and Engineering, Vol. 196, Issue 37-40
DOI: 10.1016/j.cma.2006.10.047

Stochastic Galerkin Matrices
journal, January 2010

Ernst, Oliver G.; Ullmann, Elisabeth
SIAM Journal on Matrix Analysis and Applications, Vol. 31, Issue 4
DOI: 10.1137/080742282

Existence and Computation of Low Kronecker-Rank Approximations for Large Linear Systems of Tensor Product Structure
journal, January 2004

Grasedyck, L.
Computing, Vol. 72, Issue 3-4
DOI: 10.1007/s00607-003-0037-z

Krylov Subspace Methods for Linear Systems with Tensor Product Structure
journal, January 2010

Kressner, Daniel; Tobler, Christine
SIAM Journal on Matrix Analysis and Applications, Vol. 31, Issue 4
DOI: 10.1137/090756843

Low-Rank Tensor Krylov Subspace Methods for Parametrized Linear Systems
journal, October 2011

Kressner, Daniel; Tobler, Christine
SIAM Journal on Matrix Analysis and Applications, Vol. 32, Issue 4
DOI: 10.1137/100799010

Solving stochastic systems with low-rank tensor compression
journal, May 2012

Matthies, Hermann G.; Zander, Elmar
Linear Algebra and its Applications, Vol. 436, Issue 10
DOI: 10.1016/j.laa.2011.04.017

Generalized spectral decomposition for stochastic nonlinear problems
journal, January 2009

Nouy, Anthony; Le Maıˆtre, Olivier P.
Journal of Computational Physics, Vol. 228, Issue 1
DOI: 10.1016/j.jcp.2008.09.010

Block-diagonal preconditioning for spectral stochastic finite-element systems
journal, April 2008

Powell, C. E.; Elman, H. C.
IMA Journal of Numerical Analysis, Vol. 29, Issue 2
DOI: 10.1093/imanum/drn014

An Efficient Reduced Basis Solver for Stochastic Galerkin Matrix Equations
journal, January 2017

Powell, C. E.; Silvester, D.; Simoncini, V.
SIAM Journal on Scientific Computing, Vol. 39, Issue 1
DOI: 10.1137/15M1032399

GMRES: A Generalized Minimal Residual Algorithm for Solving Nonsymmetric Linear Systems
journal, July 1986

Saad, Youcef; Schultz, Martin H.
SIAM Journal on Scientific and Statistical Computing, Vol. 7, Issue 3
DOI: 10.1137/0907058

Sparse tensor discretizations of high-dimensional parametric and stochastic PDEs
journal, April 2011

Schwab, Christoph; Gittelson, Claude Jeffrey
Acta Numerica, Vol. 20
DOI: 10.1017/S0962492911000055

Model Reduction Based on Proper Generalized Decomposition for the Stochastic Steady Incompressible Navier--Stokes Equations
journal, January 2014

Tamellini, L.; Le Maître, O.; Nouy, A.
SIAM Journal on Scientific Computing, Vol. 36, Issue 3
DOI: 10.1137/120878999

The Wiener--Askey Polynomial Chaos for Stochastic Differential Equations
journal, January 2002

Xiu, Dongbin; Karniadakis, George Em
SIAM Journal on Scientific Computing, Vol. 24, Issue 2
DOI: 10.1137/S1064827501387826

Works referencing / citing this record:

Learning classification of big medical imaging data based on partial differential equation
journal, February 2019

Zhu, Wenzhong; Xian, Lidong; Wang, Erli
Journal of Ambient Intelligence and Humanized Computing
DOI: 10.1007/s12652-019-01185-6

Similar Records in DOE PAGES and OSTI.GOV collections:

A low-rank solver for the stochastic unsteady Navier–Stokes problem

Journal Article Elman, Howard C. ; Su, Tengfei - Computer Methods in Applied Mechanics and Engineering

Here we study a low-rank iterative solver for the unsteady Navier–Stokes equations for incompressible flows with a stochastic viscosity. The equations are discretized using the stochastic Galerkin method, and we consider an all-at-once formulation where the algebraic systems at all the time steps are collected and solved simultaneously. The problem is linearized with Picard’s method. To efficiently solve the linear systems at each step, we use low-rank tensor representations within the Krylov subspace method, which leads to significant reductions in storage requirements and computational costs. Combined with effective mean-based preconditioners and the idea of inexact solve, we show that onlymore »« less
Cited by 3
https://doi.org/10.1016/j.cma.2020.112948

Full Text Available
A Low-Rank Multigrid Method for the Stochastic Steady-State Diffusion Problem

Journal Article Elman, Howard C. ; Su, Tengfei - SIAM Journal on Matrix Analysis and Applications

We study a multigrid method for solving large linear systems of equations with tensor product structure here. Such systems are obtained from stochastic finite element discretization of stochastic partial differential equations such as the steady-state diffusion problem with random coefficients. When the variance in the problem is not too large, the solution can be well approximated by a low-rank object. In the proposed multigrid algorithm, the matrix iterates are truncated to low rank to reduce memory requirements and computational effort. The technique is proved convergent with an analytic error bound. Numerical experiments show its effectiveness in solving the Galerkin systemsmore »« less
Cited by 9
https://doi.org/10.1137/17M1125170

Full Text Available
Low-Rank Solution Methods for Stochastic Eigenvalue Problems

Journal Article Elman, Howard C. ; Su, Tengfei - SIAM Journal on Scientific Computing

In this report, we study efficient solution methods for stochastic eigenvalue problems arising from discretization of self-adjoint partial differential equations with random data, where the underlying operators depend linearly on the random parameters. With the stochastic Galerkin approach, the solutions are represented as generalized polynomial chaos expansions. When these solutions can be approximated well by low-rank objects, we introduce a low-rank variant of the inverse subspace iteration algorithm for computing one or several minimal eigenvalues and corresponding eigenvectors of parameter-dependent matrices. In the algorithm, the iterates are approximated by low-rank matrices, which leads to significant cost savings. Furthermore, the algorithmmore »« less
Cited by 3
https://doi.org/10.1137/18M122100X

Full Text Available
A Low-Rank Solver for the Navier--Stokes Equations with Uncertain Viscosity

Journal Article Lee, Kookjin ; Elman, Howard C. ; Sousedík, Bedřich - SIAM/ASA Journal on Uncertainty Quantification

We study an iterative low-rank approximation method for the solution of the steady-state stochastic Navier–Stokes equations with uncertain viscosity. The method is based on linearization schemes using Picard and Newton iterations and stochastic finite element discretizations of the linearized problems. For computing the low-rank approximate solution, we adapt the nonlinear iterations to an inexact and low-rank variant, where the solution of the linear system at each nonlinear step is approximated by a quantity of low rank. This is achieved by using a tensor variant of the GMRES method as a solver for the linear systems. We explore the inexact low-rankmore »« less
Cited by 6
https://doi.org/10.1137/17M1151912

Full Text Available
Inexact Methods for Symmetric Stochastic Eigenvalue Problems

Journal Article Lee, Kookjin ; Sousedík, Bedřich - SIAM/ASA Journal on Uncertainty Quantification

We study two inexact methods for solutions of random eigenvalue problems in the context of spectral stochastic finite elements. In particular, given a parameter-dependent, symmetric matrix operator, the methods solve for eigenvalues and eigenvectors represented using polynomial chaos expansions. Both methods are based on the stochastic Galerkin formulation of the eigenvalue problem and they exploit its Kronecker-product structure. The first method is an inexact variant of the stochastic inverse subspace iteration [B. Sousedfk, H. C. Elman, SIAM/ASA Journal on Uncertainty Quantification 4(1), pp. 163-189, 2016]. The second method is based on an inexact variant of Newton iteration. In both cases,more »« less
Cited by 2
https://doi.org/10.1137/18M1176026

Full Text Available

Similar Records

Title: A Preconditioned Low-Rank Projection Method with a Rank-Reduction Scheme for Stochastic Partial Differential Equations

Abstract

Citation Formats

On solving elliptic stochastic partial differential equations journal, August 2002

Galerkin Finite Element Approximations of Stochastic Elliptic Partial Differential Equations journal, January 2004

Kolmogorov widths and low-rank approximations of parametric elliptic PDEs journal, July 2016

A projection method to solve linear systems in tensor format: A PROJECTION METHOD TO SOLVE LINEAR SYSTEMS IN TENSOR FORMAT journal, January 2012

Low-Rank Solution of Unsteady Diffusion Equations with Stochastic Coefficients journal, January 2015

Algorithms for Numerical Analysis in High Dimensions journal, January 2005

Streamline upwind/Petrov-Galerkin formulations for convection dominated flows with particular emphasis on the incompressible Navier-Stokes equations journal, September 1982

Active Subspace Methods in Theory and Practice: Applications to Kriging Surfaces journal, January 2014

Stochastic model reduction for chaos representations journal, August 2007

Stochastic Galerkin Matrices journal, January 2010

Existence and Computation of Low Kronecker-Rank Approximations for Large Linear Systems of Tensor Product Structure journal, January 2004

Krylov Subspace Methods for Linear Systems with Tensor Product Structure journal, January 2010

Low-Rank Tensor Krylov Subspace Methods for Parametrized Linear Systems journal, October 2011

Solving stochastic systems with low-rank tensor compression journal, May 2012

Generalized spectral decomposition for stochastic nonlinear problems journal, January 2009

Block-diagonal preconditioning for spectral stochastic finite-element systems journal, April 2008

An Efficient Reduced Basis Solver for Stochastic Galerkin Matrix Equations journal, January 2017

GMRES: A Generalized Minimal Residual Algorithm for Solving Nonsymmetric Linear Systems journal, July 1986

Sparse tensor discretizations of high-dimensional parametric and stochastic PDEs journal, April 2011

Model Reduction Based on Proper Generalized Decomposition for the Stochastic Steady Incompressible Navier--Stokes Equations journal, January 2014

The Wiener--Askey Polynomial Chaos for Stochastic Differential Equations journal, January 2002

Learning classification of big medical imaging data based on partial differential equation journal, February 2019

On solving elliptic stochastic partial differential equations
journal, August 2002

Galerkin Finite Element Approximations of Stochastic Elliptic Partial Differential Equations
journal, January 2004

Kolmogorov widths and low-rank approximations of parametric elliptic PDEs
journal, July 2016

A projection method to solve linear systems in tensor format: A PROJECTION METHOD TO SOLVE LINEAR SYSTEMS IN TENSOR FORMAT
journal, January 2012

Low-Rank Solution of Unsteady Diffusion Equations with Stochastic Coefficients
journal, January 2015

Algorithms for Numerical Analysis in High Dimensions
journal, January 2005

Streamline upwind/Petrov-Galerkin formulations for convection dominated flows with particular emphasis on the incompressible Navier-Stokes equations
journal, September 1982

Active Subspace Methods in Theory and Practice: Applications to Kriging Surfaces
journal, January 2014

Stochastic model reduction for chaos representations
journal, August 2007

Stochastic Galerkin Matrices
journal, January 2010

Existence and Computation of Low Kronecker-Rank Approximations for Large Linear Systems of Tensor Product Structure
journal, January 2004

Krylov Subspace Methods for Linear Systems with Tensor Product Structure
journal, January 2010

Low-Rank Tensor Krylov Subspace Methods for Parametrized Linear Systems
journal, October 2011

Solving stochastic systems with low-rank tensor compression
journal, May 2012

Generalized spectral decomposition for stochastic nonlinear problems
journal, January 2009

Block-diagonal preconditioning for spectral stochastic finite-element systems
journal, April 2008

An Efficient Reduced Basis Solver for Stochastic Galerkin Matrix Equations
journal, January 2017

GMRES: A Generalized Minimal Residual Algorithm for Solving Nonsymmetric Linear Systems
journal, July 1986

Sparse tensor discretizations of high-dimensional parametric and stochastic PDEs
journal, April 2011

Model Reduction Based on Proper Generalized Decomposition for the Stochastic Steady Incompressible Navier--Stokes Equations
journal, January 2014

The Wiener--Askey Polynomial Chaos for Stochastic Differential Equations
journal, January 2002

Learning classification of big medical imaging data based on partial differential equation
journal, February 2019