DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: A comparison of preconditioned Krylov subspace methods for large‐scale nonsymmetric linear systems

Abstract

Summary Preconditioned Krylov subspace (KSP) methods are widely used for solving large‐scale sparse linear systems arising from numerical solutions of partial differential equations (PDEs). These linear systems are often nonsymmetric due to the nature of the PDEs, boundary or jump conditions, or discretization methods. While implementations of preconditioned KSP methods are usually readily available, it is unclear to users which methods are the best for different classes of problems. In this work, we present a comparison of some KSP methods, including GMRES, TFQMR, BiCGSTAB, and QMRCGSTAB, coupled with three classes of preconditioners, namely, Gauss–Seidel, incomplete LU factorization (including ILUT, ILUTP, and multilevel ILU), and algebraic multigrid (including BoomerAMG and ML). Theoretically, we compare the mathematical formulations and operation counts of these methods. Empirically, we compare the convergence and serial performance for a range of benchmark problems from numerical PDEs in two and three dimensions with up to millions of unknowns and also assess the asymptotic complexity of the methods as the number of unknowns increases. Our results show that GMRES tends to deliver better performance when coupled with an effective multigrid preconditioner, but it is less competitive with an ineffective preconditioner due to restarts. BoomerAMG with a proper choice ofmore » coarsening and interpolation techniques typically converges faster than ML, but both may fail for ill‐conditioned or saddle‐point problems, whereas multilevel ILU tends to succeed. We also show that right preconditioning is more desirable. This study helps establish some practical guidelines for choosing preconditioned KSP methods and motivates the development of more effective preconditioners.« less

Authors:
 [1];  [1]; ORCiD logo [1]
  1. Department of Applied Mathematics &, Statistics Stony Brook University Stony Brook New York, Institute for Advanced Computational Science Stony Brook University Stony Brook New York
Publication Date:
Sponsoring Org.:
USDOE
OSTI Identifier:
1475222
Grant/Contract Number:  
AC02-06CH11357
Resource Type:
Publisher's Accepted Manuscript
Journal Name:
Numerical Linear Algebra with Applications
Additional Journal Information:
Journal Name: Numerical Linear Algebra with Applications Journal Volume: 26 Journal Issue: 1; Journal ID: ISSN 1070-5325
Publisher:
Wiley Blackwell (John Wiley & Sons)
Country of Publication:
United Kingdom
Language:
English

Citation Formats

Ghai, Aditi, Lu, Cao, and Jiao, Xiangmin. A comparison of preconditioned Krylov subspace methods for large‐scale nonsymmetric linear systems. United Kingdom: N. p., 2018. Web. doi:10.1002/nla.2215.
Ghai, Aditi, Lu, Cao, & Jiao, Xiangmin. A comparison of preconditioned Krylov subspace methods for large‐scale nonsymmetric linear systems. United Kingdom. https://doi.org/10.1002/nla.2215
Ghai, Aditi, Lu, Cao, and Jiao, Xiangmin. Tue . "A comparison of preconditioned Krylov subspace methods for large‐scale nonsymmetric linear systems". United Kingdom. https://doi.org/10.1002/nla.2215.
@article{osti_1475222,
title = {A comparison of preconditioned Krylov subspace methods for large‐scale nonsymmetric linear systems},
author = {Ghai, Aditi and Lu, Cao and Jiao, Xiangmin},
abstractNote = {Summary Preconditioned Krylov subspace (KSP) methods are widely used for solving large‐scale sparse linear systems arising from numerical solutions of partial differential equations (PDEs). These linear systems are often nonsymmetric due to the nature of the PDEs, boundary or jump conditions, or discretization methods. While implementations of preconditioned KSP methods are usually readily available, it is unclear to users which methods are the best for different classes of problems. In this work, we present a comparison of some KSP methods, including GMRES, TFQMR, BiCGSTAB, and QMRCGSTAB, coupled with three classes of preconditioners, namely, Gauss–Seidel, incomplete LU factorization (including ILUT, ILUTP, and multilevel ILU), and algebraic multigrid (including BoomerAMG and ML). Theoretically, we compare the mathematical formulations and operation counts of these methods. Empirically, we compare the convergence and serial performance for a range of benchmark problems from numerical PDEs in two and three dimensions with up to millions of unknowns and also assess the asymptotic complexity of the methods as the number of unknowns increases. Our results show that GMRES tends to deliver better performance when coupled with an effective multigrid preconditioner, but it is less competitive with an ineffective preconditioner due to restarts. BoomerAMG with a proper choice of coarsening and interpolation techniques typically converges faster than ML, but both may fail for ill‐conditioned or saddle‐point problems, whereas multilevel ILU tends to succeed. We also show that right preconditioning is more desirable. This study helps establish some practical guidelines for choosing preconditioned KSP methods and motivates the development of more effective preconditioners.},
doi = {10.1002/nla.2215},
journal = {Numerical Linear Algebra with Applications},
number = 1,
volume = 26,
place = {United Kingdom},
year = {Tue Oct 02 00:00:00 EDT 2018},
month = {Tue Oct 02 00:00:00 EDT 2018}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
https://doi.org/10.1002/nla.2215

Citation Metrics:
Cited by: 23 works
Citation information provided by
Web of Science

Save / Share:

Works referenced in this record:

LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares
journal, March 1982

  • Paige, Christopher C.; Saunders, Michael A.
  • ACM Transactions on Mathematical Software, Vol. 8, Issue 1
  • DOI: 10.1145/355984.355989

Algebraic multigrid by smoothed aggregation for second and fourth order elliptic problems
journal, September 1996


The principle of minimized iterations in the solution of the matrix eigenvalue problem
journal, January 1951

  • Arnoldi, W. E.
  • Quarterly of Applied Mathematics, Vol. 9, Issue 1
  • DOI: 10.1090/qam/42792

Fine-Grained Parallel Incomplete LU Factorization
journal, January 2015

  • Chow, Edmond; Patel, Aftab
  • SIAM Journal on Scientific Computing, Vol. 37, Issue 2
  • DOI: 10.1137/140968896

An extreme-scale implicit solver for complex PDEs: highly heterogeneous flow in earth's mantle
conference, January 2015

  • Rudi, Johann; Ghattas, Omar; Malossi, A. Cristiano I.
  • Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '15
  • DOI: 10.1145/2807591.2807675

Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods
book, January 1994


GMRES: A Generalized Minimal Residual Algorithm for Solving Nonsymmetric Linear Systems
journal, July 1986

  • Saad, Youcef; Schultz, Martin H.
  • SIAM Journal on Scientific and Statistical Computing, Vol. 7, Issue 3
  • DOI: 10.1137/0907058

An iteration method for the solution of the eigenvalue problem of linear differential and integral operators
journal, October 1950

  • Lanczos, C.
  • Journal of Research of the National Bureau of Standards, Vol. 45, Issue 4
  • DOI: 10.6028/jres.045.026

Algebraic multigrid methods for constrained linear systems with applications to contact problems in solid mechanics
journal, March 2004

  • Adams, Mark F.
  • Numerical Linear Algebra with Applications, Vol. 11, Issue 23
  • DOI: 10.1002/nla.374

CGS, A Fast Lanczos-Type Solver for Nonsymmetric Linear systems
journal, January 1989

  • Sonneveld, Peter
  • SIAM Journal on Scientific and Statistical Computing, Vol. 10, Issue 1
  • DOI: 10.1137/0910004

A hybrid geometric + algebraic multigrid method with semi-iterative smoothers: A HYBRID GEOMETRIC+ALGEBRAIC MULTIGRID METHOD
journal, February 2014

  • Lu, Cao; Jiao, Xiangmin; Missirlis, Nikolaos
  • Numerical Linear Algebra with Applications, Vol. 21, Issue 2
  • DOI: 10.1002/nla.1925

A Quasi-Minimal Residual Variant of the Bi-CGSTAB Algorithm for Nonsymmetric Systems
journal, March 1994

  • Chan, T. F.; Gallopoulos, E.; Simoncini, V.
  • SIAM Journal on Scientific Computing, Vol. 15, Issue 2
  • DOI: 10.1137/0915023

Algorithm 915, SuiteSparseQR
journal, November 2011


Numerical Linear Algebra
book, January 1997


Methods of conjugate gradients for solving linear systems
journal, December 1952

  • Hestenes, M. R.; Stiefel, E.
  • Journal of Research of the National Bureau of Standards, Vol. 49, Issue 6
  • DOI: 10.6028/jres.049.044

Improving algebraic multigrid interpolation operators for linear elasticity problems
journal, January 2009

  • Baker, A. H.; Kolev, Tz. V.; Yang, U. M.
  • Numerical Linear Algebra with Applications
  • DOI: 10.1002/nla.688

Solution of Sparse Indefinite Systems of Linear Equations
journal, September 1975

  • Paige, C. C.; Saunders, M. A.
  • SIAM Journal on Numerical Analysis, Vol. 12, Issue 4
  • DOI: 10.1137/0712047

How Fast are Nonsymmetric Matrix Iterations?
journal, July 1992

  • Nachtigal, Noël M.; Reddy, Satish C.; Trefethen, Lloyd N.
  • SIAM Journal on Matrix Analysis and Applications, Vol. 13, Issue 3
  • DOI: 10.1137/0613049

LSMR: An Iterative Algorithm for Sparse Least-Squares Problems
journal, January 2011

  • Fong, David Chin-Lung; Saunders, Michael
  • SIAM Journal on Scientific Computing, Vol. 33, Issue 5
  • DOI: 10.1137/10079687X

Overcoming element quality dependence of finite elements with adaptive extended stencil FEM (AES-FEM): OVERCOMING ELEMENT QUALITY DEPENDENCE OF FINITE ELEMENTS WITH AES-FEM
journal, March 2016

  • Conley, Rebecca; Delaney, Tristan J.; Jiao, Xiangmin
  • International Journal for Numerical Methods in Engineering, Vol. 108, Issue 9
  • DOI: 10.1002/nme.5246

Accurate Projection Methods for the Incompressible Navier–Stokes Equations
journal, April 2001

  • Brown, David L.; Cortez, Ricardo; Minion, Michael L.
  • Journal of Computational Physics, Vol. 168, Issue 2
  • DOI: 10.1006/jcph.2001.6715

hypre: A Library of High Performance Preconditioners
book, April 2002

  • Falgout, Robert D.; Yang, Ulrike Meier; Goos, Gerhard
  • Computational Science — ICCS 2002: International Conference Amsterdam, The Netherlands, April 21–24, 2002 Proceedings, Part III
  • DOI: 10.1007/3-540-47789-6_66

A Parallel Graph Coloring Heuristic
journal, May 1993

  • Jones, Mark T.; Plassmann, Paul E.
  • SIAM Journal on Scientific Computing, Vol. 14, Issue 3
  • DOI: 10.1137/0914041

Bi-CGSTAB: A Fast and Smoothly Converging Variant of Bi-CG for the Solution of Nonsymmetric Linear Systems
journal, March 1992

  • van der Vorst, H. A.
  • SIAM Journal on Scientific and Statistical Computing, Vol. 13, Issue 2
  • DOI: 10.1137/0913035

QMR: a quasi-minimal residual method for non-Hermitian linear systems
journal, December 1991

  • Freund, Roland W.; Nachtigal, No�l M.
  • Numerische Mathematik, Vol. 60, Issue 1
  • DOI: 10.1007/BF01385726

A Supernodal Approach to Incomplete LU Factorization with Partial Pivoting
journal, February 2011

  • Li, Xiaoye S.; Shao, Meiyue
  • ACM Transactions on Mathematical Software, Vol. 37, Issue 4
  • DOI: 10.1145/1916461.1916467

Reducing Complexity in Parallel Algebraic Multigrid Preconditioners
journal, January 2006

  • De Sterck, Hans; Yang, Ulrike Meier; Heys, Jeffrey J.
  • SIAM Journal on Matrix Analysis and Applications, Vol. 27, Issue 4
  • DOI: 10.1137/040615729

Mixed Finite Element Methods and Applications
book, January 2013


Distance-two interpolation for parallel algebraic multigrid
journal, January 2008

  • De Sterck, Hans; Falgout, Robert D.; Nolting, Joshua W.
  • Numerical Linear Algebra with Applications, Vol. 15, Issue 2-3
  • DOI: 10.1002/nla.559

Symmetric Successive Overrelaxation In Solving Diffusion Difference Equations
journal, October 1961

  • Habetler, G. J.; Wachspress, E. L.
  • Mathematics of Computation, Vol. 15, Issue 76
  • DOI: 10.2307/2003024

Triangle: Engineering a 2D quality mesh generator and Delaunay triangulator
book, January 1996

  • Shewchuk, Jonathan Richard
  • Applied Computational Geometry Towards Geometric Engineering, p. 203-222
  • DOI: 10.1007/BFb0014497

A comparative study of sparse approximate inverse preconditioners
journal, June 1999


ILUT: A dual threshold incomplete LU factorization
journal, July 1994


A Simple Parallel Algorithm for the Maximal Independent Set Problem
journal, November 1986

  • Luby, Michael
  • SIAM Journal on Computing, Vol. 15, Issue 4
  • DOI: 10.1137/0215074

Multilevel Preconditioners Constructed From Inverse-Based ILUs
journal, January 2006

  • Bollhöfer, Matthias; Saad, Yousef
  • SIAM Journal on Scientific Computing, Vol. 27, Issue 5
  • DOI: 10.1137/040608374

A Transpose-Free Quasi-Minimal Residual Algorithm for Non-Hermitian Linear Systems
journal, March 1993

  • Freund, Roland W.
  • SIAM Journal on Scientific Computing, Vol. 14, Issue 2
  • DOI: 10.1137/0914029

Geometric multigrid with applications to computational fluid dynamics
book, January 2001


CG Versus MINRES: An Empirical Comparison
journal, April 2012


Preconditioning Techniques for Large Linear Systems: A Survey
journal, November 2002