Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

A survey of numerical linear algebra methods utilizing mixed-precision arithmetic

Journal Article · · International Journal of High Performance Computing Applications
 [1];  [2];  [3];  [4];  [5];  [6];  [7];  [1];  [8];  [9];  [3];  [1];  [8];  [3];  [5];  [10];  [11];  [11];  [1];  [1] more »;  [7] « less
  1. Univ. of Tennessee, Knoxville, TN (United States)
  2. Univ. of Tennessee, Knoxville, TN (United States); Karlsruhe Inst. of Technology (KIT) (Germany)
  3. Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
  4. Charles Univ., Prague (Czech Republic)
  5. Karlsruhe Inst. of Technology (KIT) (Germany)
  6. Univ. of Tennessee, Knoxville, TN (United States); Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States); Univ. of Manchester (United Kingdom)
  7. Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
  8. Univ. of Manchester (United Kingdom)
  9. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
  10. Argonne National Lab. (ANL), Argonne, IL (United States)
  11. National Renewable Energy Lab. (NREL), Boulder, CO (United States)
+ Show Author Affiliations
The efficient utilization of mixed-precision numerical linear algebra algorithms can offer attractive acceleration to scientific computing applications. Especially with the hardware integration of low-precision special-function units designed for machine learning applications, the traditional numerical algorithms community urgently needs to reconsider the floating point formats used in the distinct operations to efficiently leverage the available compute power. In this study, we provide a comprehensive survey of mixed-precision numerical linear algebra routines, including the underlying concepts, theoretical background, and experimental results for both dense and sparse linear algebra problems.
Research Organization:
Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States); National Renewable Energy Laboratory (NREL), Golden, CO (United States)
Sponsoring Organization:
USDOE National Nuclear Security Administration (NNSA); USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21), Exascale Computing Project
Grant/Contract Number:
AC36-08GO28308; AC52-07NA27344
OSTI ID:
1825849
Alternate ID(s):
OSTI ID: 1839924
Report Number(s):
LLNL-JRNL--826451; NREL/JA-2C00-77220; 1041053
Journal Information:
International Journal of High Performance Computing Applications, Journal Name: International Journal of High Performance Computing Applications Journal Issue: 4 Vol. 35; ISSN 1094-3420
Publisher:
SAGECopyright Statement
Country of Publication:
United States
Language:
English

References (63)

Adaptive precision in block-Jacobi preconditioning for iterative sparse linear system solvers: Adaptive precision in block-Jacobi preconditioning for iterative solvers journal March 2018
Gram-Schmidt orthogonalization: 100 years and more: GRAM-SCHMIDT ORTHOGONALIZATION: 100 YEARS AND MORE journal June 2012
Solving linear least squares problems by Gram-Schmidt orthogonalization journal March 1967
Iterative refinement of linear least squares solutions I journal December 1967
Reliable updated residuals in hybrid Bi-CG methods journal June 1996
A note on the error analysis of classical Gram–Schmidt journal November 2006
Scaling linear optimization problems prior to application of the simplex method journal July 2011
Iterative refinement for symmetric eigenvalue decomposition journal May 2018
Iterative refinement for symmetric eigenvalue decomposition II: clustered eigenvalues journal February 2019
Benefits from using mixed precision computations in the ELPA-AEO and ESSEX-II eigensolver projects journal April 2019
Accuracy and effectiveness of the Lanczos algorithm for the symmetric eigenproblem journal December 1980
Behavior of slightly perturbed Lanczos and conjugate-gradient recurrences journal February 1989
Numerics of Gram-Schmidt orthogonalization journal January 1994
Solving lattice QCD systems of equations using mixed precision solvers on GPUs journal September 2010
Incomplete Sparse Approximate Inverses for Parallel Preconditioning journal January 2018
GPU Acceleration of a Non-hydrostatic Ocean Model with a Multigrid Poisson/Helmholtz solver journal January 2016
The Lanczos and conjugate gradient algorithms in finite precision arithmetic journal May 2006
Numerical linear algebra on emerging architectures: The PLASMA and MAGMA projects journal July 2009
Iterative refinement implies numerical stability for Gaussian elimination journal September 1980
Iterative refinement for linear systems and LAPACK journal October 1997
Fast Batched Matrix Multiplication for Small Sizes Using Half-Precision Arithmetic on GPUs conference May 2019
Exploiting the Performance of 32 bit Floating Point Arithmetic in Obtaining 64 bit Accuracy (Revisiting Iterative Refinement for Linear Systems) conference November 2006
Efficiency and scalability of two parallel QR factorization algorithms conference January 1994
Modified Gram-Schmidt (MGS), Least Squares, and Backward Stability of MGS-GMRES journal January 2006
A Note on GMRES Preconditioned by a Perturbed $L D L^T$ Decomposition with Static Pivoting journal January 2007
Loss and Recapture of Orthogonality in the Modified Gram–Schmidt Algorithm journal January 1992
Improving the Accuracy of Computed Eigenvalues and Eigenvectors journal February 1983
The Accuracy of Solutions to Triangular Systems journal October 1989
GMRES: A Generalized Minimal Residual Algorithm for Solving Nonsymmetric Linear Systems journal July 1986
Implementation of the GMRES Method Using Householder Transformations journal January 1988
Modification of the Householder Method Based on the Compact WY Representation journal May 1992
Accuracy and Stability of Numerical Algorithms book January 2002
A Symmetry Preserving Algorithm for Matrix Scaling journal January 2014
Properties of a Unitary Matrix Obtained from a Sequence of Normalized Vectors journal January 2014
Improved Accuracy and Parallelism for MRRR-Based Eigensolvers---A Mixed Precision Approach journal January 2014
Mixed-Precision Cholesky QR Factorization and Its Case Studies on Multicore CPU with Multiple GPUs journal January 2015
A New Analysis of Iterative Refinement and Its Application to Accurate Solution of Ill-Conditioned Sparse Linear Systems journal January 2017
Accelerating the Solution of Linear Systems by Iterative Refinement in Three Precisions journal January 2018
Block Modified Gram--Schmidt Algorithms and Their Analysis journal January 2019
Shifted Cholesky QR for Computing the QR Factorization of Ill-Conditioned Matrices journal January 2020
A New Approach to Probabilistic Rounding Error Analysis journal January 2019
Squeezing a Matrix into Half Precision, with an Application to Solving Linear Systems journal January 2019
Simulating Low Precision Floating-Point Arithmetic journal January 2019
Mixed Precision Block Fused Multiply-Add: Error Analysis and Application to GPU Tensor Cores journal January 2020
Analysis of the Cholesky Method with Iterative Refinement for Solving the Symmetric Definite Generalized Eigenproblem journal January 2001
Inexact Krylov Subspace Methods for Linear Systems journal January 2004
A Rank- k Update Procedure for Reorthogonalizing the Orthogonal Factor from Modified Gram--Schmidt journal January 2004
Estimating the Attainable Accuracy of Recursively Computed Residual Methods journal July 1997
Newton's Method in Floating Point Arithmetic and Iterative Refinement of Generalized Eigenvalue Problems journal January 2001
Residual and Backward Error Bounds in Minimum Residual Krylov Subspace Methods journal January 2002
Theory of Inexact Krylov Subspace Methods and Applications to Scientific Computing journal January 2003
Residual Replacement Strategies for Krylov Subspace Iterative Methods for the Convergence of True Residuals journal January 2000
Accumulating Householder transformations, revisited journal June 2006
Error bounds from extra-precise iterative refinement journal June 2006
A fast and robust mixed-precision solver for the solution of sparse symmetric linear systems journal April 2010
Mixed-Precision AMG method for Many Core Accelerators
  • Sumiyoshi, Yuki; Fujii, Akihiro; Nukada, Akira
  • EuroMPI/ASIA '14: 21st European MPI Users' Group Meeting, Proceedings of the 21st European MPI Users' Group Meeting https://doi.org/10.1145/2642769.2642794
conference September 2014
Stability and Performance of Various Singular Value QR Implementations on Multicore CPU with a GPU journal September 2016
Investigating half precision arithmetic to accelerate dense linear system solvers
  • Haidar, Azzam; Wu, Panruo; Tomov, Stanimire
  • SC '17: The International Conference for High Performance Computing, Networking, Storage and Analysis, Proceedings of the 8th Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems https://doi.org/10.1145/3148226.3148237
conference November 2017
Iterative Refinement in Floating Point journal April 1967
Adaptive Precision Block-Jacobi for High Performance Preconditioning in the Ginkgo Linear Algebra Software journal April 2021
Algorithm 589: SICEDR : A FORTRAN Subroutine for Improving the Accuracy of Computed Matrix Eigenvalues journal December 1982
Toward a modular precision ecosystem for high-performance computing journal May 2019
Ginkgo: A high performance numerical linear algebra library journal August 2020

Similar Records

Mixed-precision iterative refinement using tensor cores on GPUs to accelerate solution of linear systems
Journal Article · Tue Nov 24 19:00:00 EST 2020 · Proceedings of the Royal Society. A. Mathematical, Physical and Engineering Sciences · OSTI ID:1787013
High-Performance Mixed-Precision Linear Solver for FPGAs
Journal Article · Mon Dec 31 23:00:00 EST 2007 · IEEE transactions on computers. Institute of Electrical and Electronics Engineers · OSTI ID:936294

Related Subjects

97 MATHEMATICS AND COMPUTING
GPUs
MATHEMATICS AND COMPUTING
Mixed-precision arithmetic
high performance computing
high-performance computing
linear algebra
mixed precision arithmetic
numerical mathematics