Rounding Error Analysis of Mixed Precision Block Householder QR Algorithms

Yang, L. Minah; Fox, Alyson; Sanders, Geoffrey

doi:10.1137/19m1296367

Rounding Error Analysis of Mixed Precision Block Householder QR Algorithms

Journal Article · Mon May 17 00:00:00 EDT 2021 · SIAM Journal on Scientific Computing

DOI:https://doi.org/10.1137/19m1296367· OSTI ID:1806442

Yang, L. Minah ^[1]; ^[1]; Sanders, Geoffrey ^[1]

Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)

Although mixed precision arithmetic has recently garnered interest for training dense neural networks, many other applications could benefit from the speedups and lower storage cost if applied appropriately. The growing interest in employing mixed precision computations motivates the need for rounding error analysis that properly handles behavior from mixed precision arithmetic. We develop mixed precision variants of existing Householder QR algorithms and show error analyses supported by numerical experiments.

Research Organization:: Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA)

Grant/Contract Number:: AC52-07NA27344

OSTI ID:: 1806442

Report Number(s):: LLNL-JRNL--795525; 989309

Journal Information:: SIAM Journal on Scientific Computing, Journal Name: SIAM Journal on Scientific Computing Journal Issue: 3 Vol. 43; ISSN 1064-8275

Publisher:: Society for Industrial and Applied Mathematics (SIAM)Copyright Statement

Country of Publication:: United States

Language:: English

References (13)

Fast Batched Matrix Multiplication for Small Sizes Using Half-Precision Arithmetic on GPUs Abdelfattah, Ahmad; Tomov, Stanimire; Dongarra, Jack 2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS) https://doi.org/10.1109/ipdps.2019.00022	conference	May 2019
Training deep neural networks with low precision multiplications Courbariaux, Matthieu; Bengio, Yoshua; David, Jean-Pierre arXiv https://doi.org/10.48550/arxiv.1412.7024	preprint	January 2014
Fast linear algebra is stable Demmel, James; Dumitriu, Ioana; Holtz, Olga Numerische Mathematik, Vol. 108, Issue 1 https://doi.org/10.1007/s00211-007-0114-x	journal	October 2007
A tutorial on spectral clustering von Luxburg, Ulrike Statistics and Computing, Vol. 17, Issue 4 https://doi.org/10.1007/s11222-007-9033-z	journal	August 2007
Backward error analysis of the AllReduce algorithm for householder QR decomposition Mori, Daisuke; Yamamoto, Yusaku; Zhang, Shao-Liang Japan Journal of Industrial and Applied Mathematics, Vol. 29, Issue 1 https://doi.org/10.1007/s13160-011-0053-x	journal	December 2011
Communication-optimal Parallel and Sequential QR and LU Factorizations Demmel, James; Grigori, Laura; Hoemmen, Mark SIAM Journal on Scientific Computing, Vol. 34, Issue 1 https://doi.org/10.1137/080731992	journal	January 2012
The WY Representation for Products of Householder Matrices Bischof, Christian; Van Loan, Charles SIAM Journal on Scientific and Statistical Computing, Vol. 8, Issue 1 https://doi.org/10.1137/0908009	journal	January 1987
A Storage-Efficient $WY$ Representation for Products of Householder Transformations Schreiber, Robert; Van Loan, Charles SIAM Journal on Scientific and Statistical Computing, Vol. 10, Issue 1 https://doi.org/10.1137/0910005	journal	January 1989
A New Approach to Probabilistic Rounding Error Analysis Higham, Nicholas J.; Mary, Theo SIAM Journal on Scientific Computing, Vol. 41, Issue 5 https://doi.org/10.1137/18M1226312	journal	January 2019
Simulating Low Precision Floating-Point Arithmetic Higham, Nicholas J.; Pranesh, Srikara SIAM Journal on Scientific Computing, Vol. 41, Issue 5 https://doi.org/10.1137/19M1251308	journal	January 2019
Probabilistic Error Analysis for Inner Products Ipsen, Ilse C. F.; Zhou, Hua SIAM Journal on Matrix Analysis and Applications, Vol. 41, Issue 4 https://doi.org/10.1137/19M1270434	journal	January 2020
Mixed Precision Block Fused Multiply-Add: Error Analysis and Application to GPU Tensor Cores Blanchard, Pierre; Higham, Nicholas J.; Lopez, Florent SIAM Journal on Scientific Computing, Vol. 42, Issue 3 https://doi.org/10.1137/19M1289546	journal	January 2020
Unitary Triangularization of a Nonsymmetric Matrix Householder, Alston S. Journal of the ACM (JACM), Vol. 5, Issue 4 https://doi.org/10.1145/320941.320947	journal	October 1958

Similar Records

Simulated Half-Precision Implementation of Blocked QR Factorization and Graph Clustering Applications

Rounding Error Analysis of Mixed Precision Block Householder QR Algorithms

Citation Formats

References (13)

Similar Records

Related Subjects