A comparison of preconditioned Krylov subspace methods for large‐scale nonsymmetric linear systems
Abstract
Summary Preconditioned Krylov subspace (KSP) methods are widely used for solving large‐scale sparse linear systems arising from numerical solutions of partial differential equations (PDEs). These linear systems are often nonsymmetric due to the nature of the PDEs, boundary or jump conditions, or discretization methods. While implementations of preconditioned KSP methods are usually readily available, it is unclear to users which methods are the best for different classes of problems. In this work, we present a comparison of some KSP methods, including GMRES, TFQMR, BiCGSTAB, and QMRCGSTAB, coupled with three classes of preconditioners, namely, Gauss–Seidel, incomplete LU factorization (including ILUT, ILUTP, and multilevel ILU), and algebraic multigrid (including BoomerAMG and ML). Theoretically, we compare the mathematical formulations and operation counts of these methods. Empirically, we compare the convergence and serial performance for a range of benchmark problems from numerical PDEs in two and three dimensions with up to millions of unknowns and also assess the asymptotic complexity of the methods as the number of unknowns increases. Our results show that GMRES tends to deliver better performance when coupled with an effective multigrid preconditioner, but it is less competitive with an ineffective preconditioner due to restarts. BoomerAMG with a proper choice ofmore »
- Authors:
-
- Department of Applied Mathematics &, Statistics Stony Brook University Stony Brook New York, Institute for Advanced Computational Science Stony Brook University Stony Brook New York
- Publication Date:
- Sponsoring Org.:
- USDOE
- OSTI Identifier:
- 1475222
- Grant/Contract Number:
- AC02-06CH11357
- Resource Type:
- Publisher's Accepted Manuscript
- Journal Name:
- Numerical Linear Algebra with Applications
- Additional Journal Information:
- Journal Name: Numerical Linear Algebra with Applications Journal Volume: 26 Journal Issue: 1; Journal ID: ISSN 1070-5325
- Publisher:
- Wiley Blackwell (John Wiley & Sons)
- Country of Publication:
- United Kingdom
- Language:
- English
Citation Formats
Ghai, Aditi, Lu, Cao, and Jiao, Xiangmin. A comparison of preconditioned Krylov subspace methods for large‐scale nonsymmetric linear systems. United Kingdom: N. p., 2018.
Web. doi:10.1002/nla.2215.
Ghai, Aditi, Lu, Cao, & Jiao, Xiangmin. A comparison of preconditioned Krylov subspace methods for large‐scale nonsymmetric linear systems. United Kingdom. https://doi.org/10.1002/nla.2215
Ghai, Aditi, Lu, Cao, and Jiao, Xiangmin. Tue .
"A comparison of preconditioned Krylov subspace methods for large‐scale nonsymmetric linear systems". United Kingdom. https://doi.org/10.1002/nla.2215.
@article{osti_1475222,
title = {A comparison of preconditioned Krylov subspace methods for large‐scale nonsymmetric linear systems},
author = {Ghai, Aditi and Lu, Cao and Jiao, Xiangmin},
abstractNote = {Summary Preconditioned Krylov subspace (KSP) methods are widely used for solving large‐scale sparse linear systems arising from numerical solutions of partial differential equations (PDEs). These linear systems are often nonsymmetric due to the nature of the PDEs, boundary or jump conditions, or discretization methods. While implementations of preconditioned KSP methods are usually readily available, it is unclear to users which methods are the best for different classes of problems. In this work, we present a comparison of some KSP methods, including GMRES, TFQMR, BiCGSTAB, and QMRCGSTAB, coupled with three classes of preconditioners, namely, Gauss–Seidel, incomplete LU factorization (including ILUT, ILUTP, and multilevel ILU), and algebraic multigrid (including BoomerAMG and ML). Theoretically, we compare the mathematical formulations and operation counts of these methods. Empirically, we compare the convergence and serial performance for a range of benchmark problems from numerical PDEs in two and three dimensions with up to millions of unknowns and also assess the asymptotic complexity of the methods as the number of unknowns increases. Our results show that GMRES tends to deliver better performance when coupled with an effective multigrid preconditioner, but it is less competitive with an ineffective preconditioner due to restarts. BoomerAMG with a proper choice of coarsening and interpolation techniques typically converges faster than ML, but both may fail for ill‐conditioned or saddle‐point problems, whereas multilevel ILU tends to succeed. We also show that right preconditioning is more desirable. This study helps establish some practical guidelines for choosing preconditioned KSP methods and motivates the development of more effective preconditioners.},
doi = {10.1002/nla.2215},
journal = {Numerical Linear Algebra with Applications},
number = 1,
volume = 26,
place = {United Kingdom},
year = {Tue Oct 02 00:00:00 EDT 2018},
month = {Tue Oct 02 00:00:00 EDT 2018}
}
https://doi.org/10.1002/nla.2215
Web of Science
Works referenced in this record:
LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares
journal, March 1982
- Paige, Christopher C.; Saunders, Michael A.
- ACM Transactions on Mathematical Software, Vol. 8, Issue 1
Algebraic multigrid by smoothed aggregation for second and fourth order elliptic problems
journal, September 1996
- Vaněk, P.; Mandel, J.; Brezina, M.
- Computing, Vol. 56, Issue 3
The principle of minimized iterations in the solution of the matrix eigenvalue problem
journal, January 1951
- Arnoldi, W. E.
- Quarterly of Applied Mathematics, Vol. 9, Issue 1
Comparison of Different Krylov Subspace Methods Embedded in an Implicit Finite Volume Scheme for the Computation of Viscous and Inviscid Flow Fields on Unstructured Grids
journal, March 1998
- Meister, Andreas
- Journal of Computational Physics, Vol. 140, Issue 2
Fine-Grained Parallel Incomplete LU Factorization
journal, January 2015
- Chow, Edmond; Patel, Aftab
- SIAM Journal on Scientific Computing, Vol. 37, Issue 2
An extreme-scale implicit solver for complex PDEs: highly heterogeneous flow in earth's mantle
conference, January 2015
- Rudi, Johann; Ghattas, Omar; Malossi, A. Cristiano I.
- Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '15
Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods
book, January 1994
- Barrett, Richard; Berry, Michael; Chan, Tony F.
GMRES: A Generalized Minimal Residual Algorithm for Solving Nonsymmetric Linear Systems
journal, July 1986
- Saad, Youcef; Schultz, Martin H.
- SIAM Journal on Scientific and Statistical Computing, Vol. 7, Issue 3
An iteration method for the solution of the eigenvalue problem of linear differential and integral operators
journal, October 1950
- Lanczos, C.
- Journal of Research of the National Bureau of Standards, Vol. 45, Issue 4
Algebraic multigrid methods for constrained linear systems with applications to contact problems in solid mechanics
journal, March 2004
- Adams, Mark F.
- Numerical Linear Algebra with Applications, Vol. 11, Issue 23
CGS, A Fast Lanczos-Type Solver for Nonsymmetric Linear systems
journal, January 1989
- Sonneveld, Peter
- SIAM Journal on Scientific and Statistical Computing, Vol. 10, Issue 1
A hybrid geometric + algebraic multigrid method with semi-iterative smoothers: A HYBRID GEOMETRIC+ALGEBRAIC MULTIGRID METHOD
journal, February 2014
- Lu, Cao; Jiao, Xiangmin; Missirlis, Nikolaos
- Numerical Linear Algebra with Applications, Vol. 21, Issue 2
A Quasi-Minimal Residual Variant of the Bi-CGSTAB Algorithm for Nonsymmetric Systems
journal, March 1994
- Chan, T. F.; Gallopoulos, E.; Simoncini, V.
- SIAM Journal on Scientific Computing, Vol. 15, Issue 2
Algorithm 915, SuiteSparseQR
journal, November 2011
- Davis, Timothy A.
- ACM Transactions on Mathematical Software, Vol. 38, Issue 1
Methods of conjugate gradients for solving linear systems
journal, December 1952
- Hestenes, M. R.; Stiefel, E.
- Journal of Research of the National Bureau of Standards, Vol. 49, Issue 6
Improving algebraic multigrid interpolation operators for linear elasticity problems
journal, January 2009
- Baker, A. H.; Kolev, Tz. V.; Yang, U. M.
- Numerical Linear Algebra with Applications
Solution of Sparse Indefinite Systems of Linear Equations
journal, September 1975
- Paige, C. C.; Saunders, M. A.
- SIAM Journal on Numerical Analysis, Vol. 12, Issue 4
How Fast are Nonsymmetric Matrix Iterations?
journal, July 1992
- Nachtigal, Noël M.; Reddy, Satish C.; Trefethen, Lloyd N.
- SIAM Journal on Matrix Analysis and Applications, Vol. 13, Issue 3
LSMR: An Iterative Algorithm for Sparse Least-Squares Problems
journal, January 2011
- Fong, David Chin-Lung; Saunders, Michael
- SIAM Journal on Scientific Computing, Vol. 33, Issue 5
Overcoming element quality dependence of finite elements with adaptive extended stencil FEM (AES-FEM): OVERCOMING ELEMENT QUALITY DEPENDENCE OF FINITE ELEMENTS WITH AES-FEM
journal, March 2016
- Conley, Rebecca; Delaney, Tristan J.; Jiao, Xiangmin
- International Journal for Numerical Methods in Engineering, Vol. 108, Issue 9
Accurate Projection Methods for the Incompressible Navier–Stokes Equations
journal, April 2001
- Brown, David L.; Cortez, Ricardo; Minion, Michael L.
- Journal of Computational Physics, Vol. 168, Issue 2
hypre: A Library of High Performance Preconditioners
book, April 2002
- Falgout, Robert D.; Yang, Ulrike Meier; Goos, Gerhard
- Computational Science — ICCS 2002: International Conference Amsterdam, The Netherlands, April 21–24, 2002 Proceedings, Part III
A Parallel Graph Coloring Heuristic
journal, May 1993
- Jones, Mark T.; Plassmann, Paul E.
- SIAM Journal on Scientific Computing, Vol. 14, Issue 3
Bi-CGSTAB: A Fast and Smoothly Converging Variant of Bi-CG for the Solution of Nonsymmetric Linear Systems
journal, March 1992
- van der Vorst, H. A.
- SIAM Journal on Scientific and Statistical Computing, Vol. 13, Issue 2
QMR: a quasi-minimal residual method for non-Hermitian linear systems
journal, December 1991
- Freund, Roland W.; Nachtigal, No�l M.
- Numerische Mathematik, Vol. 60, Issue 1
A Supernodal Approach to Incomplete LU Factorization with Partial Pivoting
journal, February 2011
- Li, Xiaoye S.; Shao, Meiyue
- ACM Transactions on Mathematical Software, Vol. 37, Issue 4
Reducing Complexity in Parallel Algebraic Multigrid Preconditioners
journal, January 2006
- De Sterck, Hans; Yang, Ulrike Meier; Heys, Jeffrey J.
- SIAM Journal on Matrix Analysis and Applications, Vol. 27, Issue 4
Mixed Finite Element Methods and Applications
book, January 2013
- Boffi, Daniele; Brezzi, Franco; Fortin, Michel
- Springer Series in Computational Mathematics
Distance-two interpolation for parallel algebraic multigrid
journal, January 2008
- De Sterck, Hans; Falgout, Robert D.; Nolting, Joshua W.
- Numerical Linear Algebra with Applications, Vol. 15, Issue 2-3
Symmetric Successive Overrelaxation In Solving Diffusion Difference Equations
journal, October 1961
- Habetler, G. J.; Wachspress, E. L.
- Mathematics of Computation, Vol. 15, Issue 76
Triangle: Engineering a 2D quality mesh generator and Delaunay triangulator
book, January 1996
- Shewchuk, Jonathan Richard
- Applied Computational Geometry Towards Geometric Engineering, p. 203-222
A comparative study of sparse approximate inverse preconditioners
journal, June 1999
- Benzi, Michele; Tûma, Miroslav
- Applied Numerical Mathematics, Vol. 30, Issue 2-3
ILUT: A dual threshold incomplete LU factorization
journal, July 1994
- Saad, Yousef
- Numerical Linear Algebra with Applications, Vol. 1, Issue 4
A Simple Parallel Algorithm for the Maximal Independent Set Problem
journal, November 1986
- Luby, Michael
- SIAM Journal on Computing, Vol. 15, Issue 4
Multilevel Preconditioners Constructed From Inverse-Based ILUs
journal, January 2006
- Bollhöfer, Matthias; Saad, Yousef
- SIAM Journal on Scientific Computing, Vol. 27, Issue 5
A Transpose-Free Quasi-Minimal Residual Algorithm for Non-Hermitian Linear Systems
journal, March 1993
- Freund, Roland W.
- SIAM Journal on Scientific Computing, Vol. 14, Issue 2
Geometric multigrid with applications to computational fluid dynamics
book, January 2001
- Wesseling, P.; Oosterlee, C. W.
- Partial Differential Equations
CG Versus MINRES: An Empirical Comparison
journal, April 2012
- Chin-Lung Fong, David; Saunders, Michael
- Sultan Qaboos University Journal for Science [SQUJS], Vol. 16
Preconditioning Techniques for Large Linear Systems: A Survey
journal, November 2002
- Benzi, Michele
- Journal of Computational Physics, Vol. 182, Issue 2