Analysis of Monte Carlo accelerated iterative methods for sparse linear systems: Analysis of Monte Carlo accelerated iterative methods for sparse linear systems

Benzi, Michele; Evans, Thomas M.; Hamilton, Steven P.; Lupo Pasini, Massimiliano; Slattery, Stuart R.

doi:10.1002/nla.2088

Title: Analysis of Monte Carlo accelerated iterative methods for sparse linear systems: Analysis of Monte Carlo accelerated iterative methods for sparse linear systems

Abstract

Summary We consider hybrid deterministic‐stochastic iterative algorithms for the solution of large, sparse linear systems. Starting from a convergent splitting of the coefficient matrix, we analyze various types of Monte Carlo acceleration schemes applied to the original preconditioned Richardson (stationary) iteration. These methods are expected to have considerable potential for resiliency to faults when implemented on massively parallel machines. We establish sufficient conditions for the convergence of the hybrid schemes, and we investigate different types of preconditioners including sparse approximate inverses. Numerical experiments on linear systems arising from the discretization of partial differential equations are presented.

Authors:

Benzi, Michele ^[1]; Evans, Thomas M. ^[2]; Hamilton, Steven P. ^[2]; Lupo Pasini, Massimiliano ^[1]; Slattery, Stuart R. ^[2]

Department of Mathematics and Computer Science, Emory University, Atlanta 30322 GA USA
Oak Ridge National Laboratory, 1 Bethel Valley Rd. Oak Ridge 37831 TN USA

Publication Date:: Sun Mar 05 00:00:00 EST 2017

Research Org.:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Org.:: USDOE

OSTI Identifier:: 1346688

Alternate Identifier(s):: OSTI ID: 1400612

Grant/Contract Number:: AC05-00OR22725; ERKJ247

Resource Type:: Accepted Manuscript

Journal Name:: Numerical Linear Algebra with Applications

Additional Journal Information:: Journal Volume: 24; Journal Issue: 3; Journal ID: ISSN 1070-5325

Publisher:: Wiley

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING; iterative methods; Monte Carlo methods; preconditioning; resilience; Richardson iteration; sparse approximation inverse; sparse linear systems

Citation Formats


                    Benzi, Michele, Evans, Thomas M., Hamilton, Steven P., Lupo Pasini, Massimiliano, and Slattery, Stuart R. Analysis of Monte Carlo accelerated iterative methods for sparse linear systems: Analysis of Monte Carlo accelerated iterative methods for sparse linear systems.  United States: N. p., 2017. 
Web.  doi:10.1002/nla.2088.

Copy to clipboard


                    Benzi, Michele, Evans, Thomas M., Hamilton, Steven P., Lupo Pasini, Massimiliano, & Slattery, Stuart R. Analysis of Monte Carlo accelerated iterative methods for sparse linear systems: Analysis of Monte Carlo accelerated iterative methods for sparse linear systems.  United States.  https://doi.org/10.1002/nla.2088

Copy to clipboard


                    Benzi, Michele, Evans, Thomas M., Hamilton, Steven P., Lupo Pasini, Massimiliano, and Slattery, Stuart R. Sun .  
"Analysis of Monte Carlo accelerated iterative methods for sparse linear systems: Analysis of Monte Carlo accelerated iterative methods for sparse linear systems".  United States.  https://doi.org/10.1002/nla.2088.  https://www.osti.gov/servlets/purl/1346688.

Copy to clipboard


                    
@article{osti_1346688,

  title        = {Analysis of Monte Carlo accelerated iterative methods for sparse linear systems: Analysis of Monte Carlo accelerated iterative methods for sparse linear systems},

  author       = {Benzi, Michele and Evans, Thomas M. and Hamilton, Steven P. and Lupo Pasini, Massimiliano and Slattery, Stuart R.},

  abstractNote = {Summary We consider hybrid deterministic‐stochastic iterative algorithms for the solution of large, sparse linear systems. Starting from a convergent splitting of the coefficient matrix, we analyze various types of Monte Carlo acceleration schemes applied to the original preconditioned Richardson (stationary) iteration. These methods are expected to have considerable potential for resiliency to faults when implemented on massively parallel machines. We establish sufficient conditions for the convergence of the hybrid schemes, and we investigate different types of preconditioners including sparse approximate inverses. Numerical experiments on linear systems arising from the discretization of partial differential equations are presented.},

  doi          = {10.1002/nla.2088},

  journal      = {Numerical Linear Algebra with Applications},

  number       = 3,

  volume       = 24,

  place        = {United States},

  year         = {Sun Mar 05 00:00:00 EST 2017},

  month        = {Sun Mar 05 00:00:00 EST 2017}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (Publisher)

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1002/nla.2088

Other availability

Search WorldCat to find libraries that may hold this journal

Citation Metrics:

Cited by: 13 works

Citation information provided by
Web of Science

Save / Share:

Export Metadata

Save to My Library

Works referenced in this record:

Sequential Monte Carlo
journal, January 1962

Halton, J. H.
Mathematical Proceedings of the Cambridge Philosophical Society, Vol. 58, Issue 1
DOI: 10.1017/S0305004100036227

Monte Carlo linear solvers with non-diagonal splitting
journal, February 2010

Srinivasan, A.
Mathematics and Computers in Simulation, Vol. 80, Issue 6
DOI: 10.1016/j.matcom.2009.03.010

A Monte Carlo synthetic-acceleration method for solving the thermal radiation diffusion equation
journal, February 2014

Evans, Thomas M.; Mosher, Scott W.; Slattery, Stuart R.
Journal of Computational Physics, Vol. 258
DOI: 10.1016/j.jcp.2013.10.043

A new highly convergent Monte Carlo method for matrix computations
journal, August 1998

Dimov, I. T.; Alexandrov, V. N.
Mathematics and Computers in Simulation, Vol. 47, Issue 2-5
DOI: 10.1016/S0378-4754(98)00101-3

A backward/forward recovery approach for the preconditioned conjugate gradient method
journal, November 2016

Fasi, Massimiliano; Langou, Julien; Robert, Yves
Journal of Computational Science, Vol. 17
DOI: 10.1016/j.jocs.2016.04.008

Sequential monte carlo techniques for the solution of linear systems
journal, June 1994

Halton, John H.
Journal of Scientific Computing, Vol. 9, Issue 2
DOI: 10.1007/BF01578388

A Sparse Approximate Inverse Preconditioner for the Conjugate Gradient Method
journal, September 1996

Benzi, Michele; Meyer, Carl D.; Tůma, Miroslav
SIAM Journal on Scientific Computing, Vol. 17, Issue 5
DOI: 10.1137/S1064827594271421

Parallel Hybrid Monte Carlo Algorithms for Matrix Computations
book, January 2005

Alexandrov, V.; Atanassov, E.; Dimov, I.
Computational Science – ICCS 2005
DOI: 10.1007/11428862_102

Block diagonally dominant matrices and generalizations of the Gerschgorin circle theorem
journal, January 1962

Feingold, David G.; Varga, Richard
Pacific Journal of Mathematics, Vol. 12, Issue 4
DOI: 10.2140/pjm.1962.12.1241

�ber die partiellen Differenzengleichungen der mathematischen Physik
journal, December 1928

Courant, R.; Friedrichs, K.; Lewy, H.
Mathematische Annalen, Vol. 100, Issue 1
DOI: 10.1007/BF01448839

Fault Resilient Domain Decomposition Preconditioner for PDEs
journal, January 2015

Sargsyan, Khachik; Rizzi, Francesco; Mycek, Paul
SIAM Journal on Scientific Computing, Vol. 37, Issue 5
DOI: 10.1137/15M1014474

Convergence Analysis of Markov Chain Monte Carlo Linear Solvers Using Ulam--von Neumann Algorithm
journal, January 2013

Ji, Hao; Mascagni, Michael; Li, Yaohang
SIAM Journal on Numerical Analysis, Vol. 51, Issue 4
DOI: 10.1137/130904867

A Sparse Approximate Inverse Preconditioner for Nonsymmetric Linear Systems
journal, May 1998

Benzi, Michele; Tuma, Miroslav
SIAM Journal on Scientific Computing, Vol. 19, Issue 3
DOI: 10.1137/S1064827595294691

Preconditioning Techniques for Large Linear Systems: A Survey
journal, November 2002

Benzi, Michele
Journal of Computational Physics, Vol. 182, Issue 2
DOI: 10.1006/jcph.2002.7176

IFISS: A Computational Laboratory for Investigating Incompressible Flow Problems
journal, January 2014

Elman, Howard C.; Ramage, Alison; Silvester, David J.
SIAM Review, Vol. 56, Issue 2
DOI: 10.1137/120891393

Numerical Analysis of Fixed Point Algorithms in the Presence of Hardware Faults
journal, January 2015

Stoyanov, Miroslav; Webster, Clayton
SIAM Journal on Scientific Computing, Vol. 37, Issue 5
DOI: 10.1137/140991406

matrices and applications
journal, September 1975

Friedland, S.; Karlin, S.
Duke Mathematical Journal, Vol. 42, Issue 3
DOI: 10.1215/S0012-7094-75-04244-1

Parallel resolvent Monte Carlo algorithms for linear algebra problems
journal, February 2001

Dimov, I.; Alexandrov, V.; Karaivanova, A.
Mathematics and Computers in Simulation, Vol. 55, Issue 1-3
DOI: 10.1016/S0378-4754(00)00243-3

Iterative Solution Methods
book, January 1994

Axelsson, Owe
DOI: 10.1017/CBO9780511624100

On the Iterative Criterion for Generalized Diagonally Dominant Matrices
journal, January 2002

Li, Lei
SIAM Journal on Matrix Analysis and Applications, Vol. 24, Issue 1
DOI: 10.1137/S0895479898348829

Iterative Methods for Sparse Linear Systems
book, January 2003

Saad, Yousef
DOI: 10.1137/1.9780898718003

Works referencing / citing this record:

A highly parallel algorithm for computing the action of a matrix exponential on a vector based on a multilevel Monte Carlo method
journal, June 2020

Acebrón, Juan A.; Herrero, José R.; Monteiro, José
Computers & Mathematics with Applications, Vol. 79, Issue 12
DOI: 10.1016/j.camwa.2020.02.013

Similar Records in DOE PAGES and OSTI.GOV collections:

Anderson acceleration of the Jacobi iterative method: An efficient alternative to Krylov methods for large, sparse linear systems

Journal Article Pratapa, Phanisri P. ; Suryanarayana, Phanish ; Pask, John E. - Journal of Computational Physics

We employ Anderson extrapolation to accelerate the classical Jacobi iterative method for large, sparse linear systems. Specifically, we utilize extrapolation at periodic intervals within the Jacobi iteration to develop the Alternating Anderson–Jacobi (AAJ) method. We verify the accuracy and efficacy of AAJ in a range of test cases, including nonsymmetric systems of equations. We demonstrate that AAJ possesses a favorable scaling with system size that is accompanied by a small prefactor, even in the absence of a preconditioner. In particular, we show that AAJ is able to accelerate the classical Jacobi iteration by over four orders of magnitude, with speed-upsmore »« less
Cited by 44
https://doi.org/10.1016/j.jcp.2015.11.018

Full Text Available
NSPCG (Nonsymmetric Preconditioned Conjugate Gradient) user's guide: Version 1. 0: A package for solving large sparse linear systems by various iterative methods

Technical Report Oppe, T C ; Joubert, W D ; Kincaid, D R

NSPCG (or Nonsymmetric Preconditioned Conjugate Gradient) is a computer package to solve the linear system Au = b by various iterative methods. The coefficient matrix A is assumed to be large and sparse with real coefficients. A wide selection of preconditioners and accelerators is available for both symmetric and nonsymmetric coefficient matrices. Several sparse matrix data structures are available to represent matrices whose structures range from highly structures to completely unstructured. 36 refs., 8 tabs.
https://doi.org/10.2172/7035748

Full Text Available
A comparison of preconditioned Krylov subspace methods for large‐scale nonsymmetric linear systems

Journal Article Ghai, Aditi ; Lu, Cao ; Jiao, Xiangmin - Numerical Linear Algebra with Applications

Summary Preconditioned Krylov subspace (KSP) methods are widely used for solving large‐scale sparse linear systems arising from numerical solutions of partial differential equations (PDEs). These linear systems are often nonsymmetric due to the nature of the PDEs, boundary or jump conditions, or discretization methods. While implementations of preconditioned KSP methods are usually readily available, it is unclear to users which methods are the best for different classes of problems. In this work, we present a comparison of some KSP methods, including GMRES, TFQMR, BiCGSTAB, and QMRCGSTAB, coupled with three classes of preconditioners, namely, Gauss–Seidel, incomplete LU factorization (including ILUT, ILUTP,more »« less
Cited by 23
https://doi.org/10.1002/nla.2215
PCG reference manual: A package for the iterative solution of large sparse linear systems on parallel computers. Version 1.0

Technical Report Joubert, W D ; Carey, G F ; Kohli, H ; ...

PCG (Preconditioned Conjugate Gradient package) is a system for solving linear equations of the form Au = b, for A a given matrix and b and u vectors. PCG, employing various gradient-type iterative methods coupled with preconditioners, is designed for general linear systems, with emphasis on sparse systems such as these arising from discretization of partial differential equations arising from physical applications. It can be used to solve linear equations efficiently on parallel computer architectures. Much of the code is reusable across architectures and the package is portable across different systems; the machines that are currently supported is listed. Thismore »« less
https://doi.org/10.2172/409864

Full Text Available
Convergence analysis of Anderson-type acceleration of Richardson's iteration

Journal Article Lupo Pasini, Massimiliano - Numerical Linear Algebra with Applications

We consider here Anderson extrapolation to accelerate the (stationary) Richardson iterative method for sparse linear systems. Using an Anderson mixing at periodic intervals, we assess how this benefits convergence to a prescribed accuracy. The method, named alternating Anderson–Richardson, has appealing properties for high-performance computing, such as the potential to reduce communication and storage in comparison to more conventional linear solvers. We establish sufficient conditions for convergence, and we evaluate the performance of this technique in combination with various preconditioners through numerical examples. Furthermore, we propose an augmented version of this technique.
Cited by 5
https://doi.org/10.1002/nla.2241

Full Text Available

Similar Records

Title: Analysis of Monte Carlo accelerated iterative methods for sparse linear systems: Analysis of Monte Carlo accelerated iterative methods for sparse linear systems

Abstract

Citation Formats

Sequential Monte Carlo journal, January 1962

Monte Carlo linear solvers with non-diagonal splitting journal, February 2010

A Monte Carlo synthetic-acceleration method for solving the thermal radiation diffusion equation journal, February 2014

A new highly convergent Monte Carlo method for matrix computations journal, August 1998

A backward/forward recovery approach for the preconditioned conjugate gradient method journal, November 2016

Sequential monte carlo techniques for the solution of linear systems journal, June 1994

A Sparse Approximate Inverse Preconditioner for the Conjugate Gradient Method journal, September 1996

Parallel Hybrid Monte Carlo Algorithms for Matrix Computations book, January 2005

Block diagonally dominant matrices and generalizations of the Gerschgorin circle theorem journal, January 1962

�ber die partiellen Differenzengleichungen der mathematischen Physik journal, December 1928

Fault Resilient Domain Decomposition Preconditioner for PDEs journal, January 2015

Convergence Analysis of Markov Chain Monte Carlo Linear Solvers Using Ulam--von Neumann Algorithm journal, January 2013

A Sparse Approximate Inverse Preconditioner for Nonsymmetric Linear Systems journal, May 1998

Preconditioning Techniques for Large Linear Systems: A Survey journal, November 2002

IFISS: A Computational Laboratory for Investigating Incompressible Flow Problems journal, January 2014

Numerical Analysis of Fixed Point Algorithms in the Presence of Hardware Faults journal, January 2015

matrices and applications journal, September 1975

Parallel resolvent Monte Carlo algorithms for linear algebra problems journal, February 2001

Iterative Solution Methods book, January 1994

On the Iterative Criterion for Generalized Diagonally Dominant Matrices journal, January 2002

Iterative Methods for Sparse Linear Systems book, January 2003

A highly parallel algorithm for computing the action of a matrix exponential on a vector based on a multilevel Monte Carlo method journal, June 2020

Sequential Monte Carlo
journal, January 1962

Monte Carlo linear solvers with non-diagonal splitting
journal, February 2010

A Monte Carlo synthetic-acceleration method for solving the thermal radiation diffusion equation
journal, February 2014

A new highly convergent Monte Carlo method for matrix computations
journal, August 1998

A backward/forward recovery approach for the preconditioned conjugate gradient method
journal, November 2016

Sequential monte carlo techniques for the solution of linear systems
journal, June 1994

A Sparse Approximate Inverse Preconditioner for the Conjugate Gradient Method
journal, September 1996

Parallel Hybrid Monte Carlo Algorithms for Matrix Computations
book, January 2005

Block diagonally dominant matrices and generalizations of the Gerschgorin circle theorem
journal, January 1962

�ber die partiellen Differenzengleichungen der mathematischen Physik
journal, December 1928

Fault Resilient Domain Decomposition Preconditioner for PDEs
journal, January 2015

Convergence Analysis of Markov Chain Monte Carlo Linear Solvers Using Ulam--von Neumann Algorithm
journal, January 2013

A Sparse Approximate Inverse Preconditioner for Nonsymmetric Linear Systems
journal, May 1998

Preconditioning Techniques for Large Linear Systems: A Survey
journal, November 2002

IFISS: A Computational Laboratory for Investigating Incompressible Flow Problems
journal, January 2014

Numerical Analysis of Fixed Point Algorithms in the Presence of Hardware Faults
journal, January 2015

matrices and applications
journal, September 1975

Parallel resolvent Monte Carlo algorithms for linear algebra problems
journal, February 2001

Iterative Solution Methods
book, January 1994

On the Iterative Criterion for Generalized Diagonally Dominant Matrices
journal, January 2002

Iterative Methods for Sparse Linear Systems
book, January 2003

A highly parallel algorithm for computing the action of a matrix exponential on a vector based on a multilevel Monte Carlo method
journal, June 2020