Iterated Gauss-Seidel GMRES

Thomas, Stephen; Carson, Erin; Rozloznik, Miroslav; Carr, Arielle; Swirydowicz, Katarzyna

doi:10.1137/22M1491241

Iterated Gauss-Seidel GMRES

Journal Article · Mon Jul 24 00:00:00 EDT 2023 · SIAM Journal on Scientific Computing

DOI:https://doi.org/10.1137/22M1491241· OSTI ID:2367546

Thomas, Stephen; Carson, Erin; Rozloznik, Miroslav; Carr, Arielle; Swirydowicz, Katarzyna

The GMRES algorithm of Saad and Schultz [SIAM J. Sci. Stat. Comput., 7 (1986), pp. 856-869] is an iterative method for approximately solving linear systems Ax = b, with initial guess x0 and residual r0 = b Ax0. The algorithm employs the Arnoldi process to generate the Krylov basis vectors (the columns of Vk ). It is well known that this process can be viewed as a QR factorization of the matrix Bk = [r0, AVk] at each iteration. Despite an O (..epsilon..)..kappa.. (Bk ) loss of orthogonality, for unit roundoff ..epsilon..and condition number ..kappa.. , the modified Gram-Schmidt formulation was shown to be backward stable in the seminal paper by Paige et al. [SIAM J. Matrix Anal.Appl., 28 (2006), pp. 264-284]. We present an iterated Gauss-Seidel formulation of the GMRES algorithm (IGS-GMRES) based on the ideas of Ruhe [Linear Algebra Appl., 52 (1983), pp. 591-601] and Swirydowicz et al. [Numer. Linear Algebra Appl., 28 (2020), pp. 1-20]. IGS-GMRES maintains orthogonality to the level O (..epsilon..)..kappa.. (Bk ) or O (..epsilon..), depending on the choice of one or two iterations; for two Gauss-Seidel iterations, the computed Krylov basis vectors remain orthogonal to working accuracy and the smallest singular value of Vk remains close to one. The resulting GMRES method is thus backward stable. We show that IGS-GMRES can be implemented with only a single synchronization point per iteration, making it relevant to large-scale parallel computing environments. We also demonstrate that, unlike MGS-GMRES, in IGS-GMRES the relative Arnoldi residual corresponding to the computed approximate solution no longer stagnates above machine precision even for highly nonnormal systems.

Research Organization:: National Renewable Energy Laboratory (NREL), Golden, CO (United States)

Sponsoring Organization:: USDOE Office of Science (SC); USDOE National Nuclear Security Administration (NNSA); Exascale Computing Project (ECP)

DOE Contract Number:: AC36-08GO28308

OSTI ID:: 2367546

Report Number(s):: NREL/JA-2C00-90005; MainId:91783; UUID:fabdd5ba-702e-49d1-8b6c-a52adae0bcb7; MainAdminId:72688

Journal Information:: SIAM Journal on Scientific Computing, Journal Name: SIAM Journal on Scientific Computing Journal Issue: 2 Vol. 46

Country of Publication:: United States

Language:: English

References (25)

The Stability of Block Variants of Classical Gram--Schmidt Carson, Erin; Lund, Kathryn; Rozložník, Miroslav SIAM Journal on Matrix Analysis and Applications, Vol. 42, Issue 3 https://doi.org/10.1137/21M1394424	journal	January 2021
Modified Gram-Schmidt (MGS), Least Squares, and Backward Stability of MGS-GMRES Paige, Christopher C.; Rozlozník, Miroslav; Strakos, Zdenvek SIAM Journal on Matrix Analysis and Applications, Vol. 28, Issue 1 https://doi.org/10.1137/050630416	journal	January 2006
Block Gram-Schmidt algorithms and their stability properties Carson, Erin; Lund, Kathryn; Rozložník, Miroslav Linear Algebra and its Applications, Vol. 638 https://doi.org/10.1016/j.laa.2021.12.017	journal	April 2022
Rounding error analysis of the classical Gram-Schmidt orthogonalization process Giraud, Luc; Langou, Julien; Rozložník, Miroslav Numerische Mathematik, Vol. 101, Issue 1 https://doi.org/10.1007/s00211-005-0615-4	journal	May 2005
The Design and Implementation of hypre, a Library of Parallel High Performance Preconditioners Falgout, Robert D.; Jones, Jim E.; Yang, Ulrike Meier Lecture Notes in Computational Science and Engineering https://doi.org/10.1007/3-540-31619-1_8	book	January 2006
Solving linear least squares problems by Gram-Schmidt orthogonalization Björck, Åke BIT, Vol. 7, Issue 1 https://doi.org/10.1007/BF01934122	journal	March 1967
Numerical behaviour of the modified gram-schmidt GMRES implementation Greenbaum, A.; Rozložník, M.; Strakoš, Z. BIT Numerical Mathematics, Vol. 37, Issue 3 https://doi.org/10.1007/BF02510248	journal	September 1997
Accuracy and Stability of Numerical Algorithms Higham, Nicholas J. https://doi.org/10.1137/1.9780898718027	book	January 2002
The Worst-Case GMRES for Normal Matrices Liesen, Jörg; Tichý, Petr BIT Numerical Mathematics, Vol. 44, Issue 1 https://doi.org/10.1023/B:BITN.0000025083.59864.bd	journal	January 2004
Theory of Inexact Krylov Subspace Methods and Applications to Scientific Computing Simoncini, Valeria; Szyld, Daniel B. SIAM Journal on Scientific Computing, Vol. 25, Issue 2 https://doi.org/10.1137/S1064827502406415	journal	January 2003
Spectra and Pseudospectra Trefethen, Lloyd N.; Embree, Mark Princeton University Press https://doi.org/10.1515/9780691213101	book	January 2005
Numerics of Gram-Schmidt orthogonalization Björck, Å. Linear Algebra and its Applications, Vol. 197-198 https://doi.org/10.1016/0024-3795(94)90493-6	journal	January 1994
Numerical stability of GMRES Drkošová, J.; Greenbaum, A.; Rozložník, M. BIT Numerical Mathematics, Vol. 35, Issue 3 https://doi.org/10.1007/BF01732607	journal	September 1995
Numerical aspects of gram-schmidt orthogonalization of vectors Ruhe, Axel Linear Algebra and its Applications, Vol. 52-53 https://doi.org/10.1016/0024-3795(83)80037-8	journal	July 1983
GMRES: A Generalized Minimal Residual Algorithm for Solving Nonsymmetric Linear Systems Saad, Youcef; Schultz, Martin H. SIAM Journal on Scientific and Statistical Computing, Vol. 7, Issue 3 https://doi.org/10.1137/0907058	journal	July 1986
The Accuracy of Solutions to Triangular Systems Higham, Nicholas J. SIAM Journal on Numerical Analysis, Vol. 26, Issue 5 https://doi.org/10.1137/0726070	journal	October 1989
Using Jacobi iterations and blocking for solving sparse triangular systems in incomplete factorization preconditioning Chow, Edmond; Anzt, Hartwig; Scott, Jennifer Journal of Parallel and Distributed Computing, Vol. 119 https://doi.org/10.1016/j.jpdc.2018.04.017	journal	September 2018
A Krylov--Schur Algorithm for Large Eigenproblems Stewart, G. W. SIAM Journal on Matrix Analysis and Applications, Vol. 23, Issue 3 https://doi.org/10.1137/S0895479800371529	journal	January 2002
Bounds for iterates, inverses, spectral variation and fields of values of non-normal matrices Henrici, Peter Numerische Mathematik, Vol. 4, Issue 1 https://doi.org/10.1007/BF01386294	journal	December 1962
Low-synch Gram–Schmidt with delayed reorthogonalization for Krylov solvers Bielich, Daniel; Langou, Julien; Thomas, Stephen Parallel Computing, Vol. 112 https://doi.org/10.1016/j.parco.2022.102940	journal	September 2022
A Rank- k Update Procedure for Reorthogonalizing the Orthogonal Factor from Modified Gram--Schmidt Giraud, Luc; Gratton, Serge; Langou, Julien SIAM Journal on Matrix Analysis and Applications, Vol. 25, Issue 4 https://doi.org/10.1137/S0895479803424347	journal	January 2004
Low synchronization Gram–Schmidt and generalized minimal residual algorithms Świrydowicz, Katarzyna; Langou, Julien; Ananthan, Shreyas Numerical Linear Algebra with Applications, Vol. 28, Issue 2 https://doi.org/10.1002/nla.2343	journal	October 2020
Loss and Recapture of Orthogonality in the Modified Gram–Schmidt Algorithm Björck, Å.; Paige, C. C. SIAM Journal on Matrix Analysis and Applications, Vol. 13, Issue 1 https://doi.org/10.1137/0613015	journal	January 1992
Residual and Backward Error Bounds in Minimum Residual Krylov Subspace Methods Paige, Christopher C.; Strakos, Zdenvek SIAM Journal on Scientific Computing, Vol. 23, Issue 6 https://doi.org/10.1137/S1064827500381239	journal	January 2002
Implementation of the GMRES Method Using Householder Transformations Walker, Homer F. SIAM Journal on Scientific and Statistical Computing, Vol. 9, Issue 1 https://doi.org/10.1137/0909010	journal	January 1988

Similar Records

Neumann Series in MGS-GMRES and Inner-Outer Iterations: Preprint

Conference · Tue Feb 08 23:00:00 EST 2022 · OSTI ID:1845270

Low-synch Gram–Schmidt with delayed reorthogonalization for Krylov solvers

Journal Article · Thu Sep 01 00:00:00 EDT 2022 · Parallel Computing · OSTI ID:1872059

Related Subjects

Arnoldi-QR
GMRES
Gauss-Seidel
Gram-Schmidt
Krylov
MATHEMATICS AND COMPUTING
orthogonal complement

Iterated Gauss-Seidel GMRES

Citation Formats

References (25)

Similar Records

Related Subjects