Load-balanced sparse MTTKRP on GPUs

Conference · Mon Sep 02 00:00:00 EDT 2019

DOI:https://doi.org/10.1109/IPDPS.2019.00023· OSTI ID:1862916

Nisa, Israt ^[1]; Li, Jiajia ^[2]; Sukumaran-Rajan, Aravind ^[1]; Vuduc, Richard ^[3]; Sadayappan, Ponnuswamy ^[1]

---

1. Ohio State University
2. BATTELLE (PACIFIC NW LAB)
3. Georgia Institute of Technology

Sparse matricized tensor times Khatri-Rao product (MT- TKRP) is one of the most computationally expensive kernels in sparse tensor computations. This work focuses on optimizing the MTTKRP for floating point operations, storage, and scalability. We begin by identifying the performance bottlenecks in directly extending the state-of-the-art CSF (compressed sparse fiber) formats from CPUs to GPUs. Our detailed analysis over the recently proposed formats shows that the lower bounds on storage and flop counts can vary significantly depending on the structure of the sparse tensor. To address this, we propose a load balanced, computation and storage-efficient scheme, HYB, which combines the best of COO (coordinate), CSF and CSL (compressed slice). With these enhancements, our GPU framework significantly out- performs the current formats on both CPU and GPU platforms.

• Research Organization: Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
• Sponsoring Organization: USDOE
• DOE Contract Number: AC05-76RL01830
• OSTI ID: 1862916
• Report Number(s): PNNL-SA-138752
• Country of Publication: United States
• Language: English