Batched matrix computations on hardware accelerators based on GPUs
Abstract
Scientific applications require solvers that work on many small size problems that are independent from each other. At the same time, the highend hardware evolves rapidly and becomes ever more throughputoriented and thus there is an increasing need for an effective approach to develop energyefficient, highperformance codes for these small matrix problems that we call batched factorizations. The many applications that need this functionality could especially benefit from the use of GPUs, which currently are four to five times more energy efficient than multicore CPUs on important scientific workloads. This study, consequently, describes the development of the most common, onesided factorizations, Cholesky, LU, and QR, for a set of small dense matrices. The algorithms we present together with their implementations are, by design, inherently parallel. In particular, our approach is based on representing the process as a sequence of batched BLAS routines that are executed entirely on a GPU. Importantly, this is unlike the LAPACK and the hybrid MAGMA factorization algorithms that work under drastically different assumptions of hardware design and efficiency of execution of the various computational kernels involved in the implementation. Thus, our approach is more efficient than what works for a combination of multicore CPUs and GPUsmore »
 Authors:

 Univ. of Tennessee, Knoxville, TN (United States)
 Univ. of Tennessee, Knoxville, TN (United States); Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States); Univ. of Manchester (United Kingdom)
 Publication Date:
 Research Org.:
 Univ. of Tennessee, Knoxville, TN (United States); Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
 Sponsoring Org.:
 USDOE; National Science Foundation (NSF); Nvidia Corporation (United States); Russian Scientific Fund (Russian Federation)
 Contributing Org.:
 Univ. of Manchester (United Kingdom)
 OSTI Identifier:
 1361289
 Grant/Contract Number:
 AC0500OR22725; ACI1339822; N141100190
 Resource Type:
 Accepted Manuscript
 Journal Name:
 International Journal of High Performance Computing Applications
 Additional Journal Information:
 Journal Volume: 29; Journal Issue: 2; Journal ID: ISSN 10943420
 Publisher:
 SAGE
 Country of Publication:
 United States
 Language:
 English
 Subject:
 97 MATHEMATICS AND COMPUTING; batched factorization; numerical linear algebra; hardware accelerators; numerical software libraries; onesided factorization algorithms
Citation Formats
Haidar, Azzam, Dong, Tingxing, Luszczek, Piotr, Tomov, Stanimire, and Dongarra, Jack. Batched matrix computations on hardware accelerators based on GPUs. United States: N. p., 2015.
Web. doi:10.1177/1094342014567546.
Haidar, Azzam, Dong, Tingxing, Luszczek, Piotr, Tomov, Stanimire, & Dongarra, Jack. Batched matrix computations on hardware accelerators based on GPUs. United States. doi:10.1177/1094342014567546.
Haidar, Azzam, Dong, Tingxing, Luszczek, Piotr, Tomov, Stanimire, and Dongarra, Jack. Mon .
"Batched matrix computations on hardware accelerators based on GPUs". United States. doi:10.1177/1094342014567546. https://www.osti.gov/servlets/purl/1361289.
@article{osti_1361289,
title = {Batched matrix computations on hardware accelerators based on GPUs},
author = {Haidar, Azzam and Dong, Tingxing and Luszczek, Piotr and Tomov, Stanimire and Dongarra, Jack},
abstractNote = {Scientific applications require solvers that work on many small size problems that are independent from each other. At the same time, the highend hardware evolves rapidly and becomes ever more throughputoriented and thus there is an increasing need for an effective approach to develop energyefficient, highperformance codes for these small matrix problems that we call batched factorizations. The many applications that need this functionality could especially benefit from the use of GPUs, which currently are four to five times more energy efficient than multicore CPUs on important scientific workloads. This study, consequently, describes the development of the most common, onesided factorizations, Cholesky, LU, and QR, for a set of small dense matrices. The algorithms we present together with their implementations are, by design, inherently parallel. In particular, our approach is based on representing the process as a sequence of batched BLAS routines that are executed entirely on a GPU. Importantly, this is unlike the LAPACK and the hybrid MAGMA factorization algorithms that work under drastically different assumptions of hardware design and efficiency of execution of the various computational kernels involved in the implementation. Thus, our approach is more efficient than what works for a combination of multicore CPUs and GPUs for the problems sizes of interest of the application use cases. The paradigm where upon a single chip (a GPU or a CPU) factorizes a single problem at a time is not at all efficient in our applications’ context. We illustrate all of these claims through a detailed performance analysis. With the help of profiling and tracing tools, we guide our development of batched factorizations to achieve up to twofold speedup and threefold better energy efficiency as compared against our highly optimized batched CPU implementations based on MKL library. Finally, the tested system featured two sockets of Intel Sandy Bridge CPUs and we compared with a batched LU factorizations featured in the CUBLAS library for GPUs, we achieve as high as 2.5× speedup on the NVIDIA K40 GPU.},
doi = {10.1177/1094342014567546},
journal = {International Journal of High Performance Computing Applications},
number = 2,
volume = 29,
place = {United States},
year = {2015},
month = {2}
}
Web of Science
Works referenced in this record:
PowerManagement Architecture of the Intel Microarchitecture CodeNamed Sandy Bridge
journal, March 2012
 Rotem, Efraim; Naveh, Alon; Ananthakrishnan, Avinash
 IEEE Micro, Vol. 32, Issue 2
Sparsity: Optimization Framework for Sparse Matrix Kernels
journal, February 2004
 Im, EunJin; Yelick, Katherine; Vuduc, Richard
 The International Journal of High Performance Computing Applications, Vol. 18, Issue 1