Mixed precision s–step Lanczos and conjugate gradient algorithms

Journal Article · 23 November 2021 · Numerical Linear Algebra with Applications

DOI:https://doi.org/10.1002/nla.2425· OSTI ID:1834328

Carson, Erin ^[1]; Gergelits, Tomáš ^[1]; Yamazaki, Ichitaro ^[2]

---

1. Charles Univ., Prague (Czech Republic)
2. Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)

Compared to the classical Lanczos algorithm, the s-step Lanczos variant has the potential to improve performance by asymptotically decreasing the synchronization cost per iteration. However, this comes at a price; despite being mathematically equivalent, the s-step variant may behave quite differently in finite precision, potentially exhibiting greater loss of accuracy and slower convergence relative to the classical algorithm. It has previously been shown that the errors in the s-step version follow the same structure as the errors in the classical algorithm, but are amplified by a factor depending on the square of the condition number of the O(s)-dimensional Krylov bases computed in each outer loop. As the condition number of these s-step bases grows (in some cases very quickly) with s, this limits the s values that can be chosen and thus can limit the attainable performance. In this work, we show that if a select few computations in s-step Lanczos are performed in double the working precision, the error terms then depend only linearly on the conditioning of the s-step bases. This has the potential for drastically improving the numerical behavior of the algorithm with little impact on per-iteration performance. Our numerical experiments demonstrate the improved numerical behavior possible with the mixed precision approach, and also show that this improved behavior extends to mixed precision s-step CG. Here, we present preliminary performance results on NVIDIA V100 GPUs that show that the overhead of extra precision is minimal if one uses precisions implemented in hardware.

• Research Organization: Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States)
• Sponsoring Organization: USDOE; USDOE National Nuclear Security Administration (NNSA)
• Grant/Contract Number: NA0003525
• OSTI ID: 1834328
• Alternate ID(s): OSTI ID: 1832252
• Report Number(s): SAND-2021-14314J; 701479
• Journal Information: Numerical Linear Algebra with Applications, Journal Name: Numerical Linear Algebra with Applications Journal Issue: 3 Vol. 29; ISSN 1070-5325
• Publisher: Wiley
• Country of Publication: United States
• Language: English

Related Subjects

97 MATHEMATICS AND COMPUTING
Krylov subspace methods
Lanczos
avoiding communication
error analysis
finite precision
mixed precision