An Efficient Mixed-Mode Representation of Sparse Tensors

Conference · Sun Nov 17 00:00:00 EST 2019

DOI:https://doi.org/10.1145/3295500.3356216· OSTI ID:1580518

Nisa, Israt ^[1]; Li, Jiajia ^[2]; Sukumaran-Rajan, Aravind ^[1]; Rawat, Prashant ^[1]; Krishnamoorthy, Sriram ^[2]; Sadayappan, Ponnuswamy ^[1]

---

1. Ohio State University
2. BATTELLE (PACIFIC NW LAB)

The Compressed Sparse Fiber (CSF) representation for sparse tensors is a generalization of the Compressed Sparse Row (CSR) format for sparse matrices. For a tensor with d modes, typical tensor methods such as CANDECOMP/PARAFAC decomposition (CPD) require a sequence of d tensor computations, where efficient memory access with respect to different modes is required for each of them.The straightforward solution is to use d distinct representations of the tensor, with each one being efficient for one of the d computations. However, a d-fold space overhead is often unacceptable in practice, especially with memory-constrained GPUs. In this paper, we present a mixed-mode tensor representation that partitions the tensor’s nonzero elements into disjoint sections, each of which is compressed to create fibers along a different mode. Experimental results on the latest generation of GPU device demonstrate that better performance can be achieved by using the mixed-mode representation, while utilizing only a small fraction of the space required to keep d distinct CSF representations.

• Research Organization: Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
• Sponsoring Organization: USDOE
• DOE Contract Number: AC05-76RL01830
• OSTI ID: 1580518
• Report Number(s): PNNL-SA-142737
• Resource Relation: Conference: International Conference on High Performance Computing, Networking, Storage and Analysis, November 17-22, 2019, Denver, CO
• Country of Publication: United States
• Language: English