A Work-Efficient Parallel Sparse Matrix-Sparse Vector Multiplication Algorithm

Azad, Ariful; Buluc, Aydin

doi:10.1109/IPDPS.2017.76

Title: A Work-Efficient Parallel Sparse Matrix-Sparse Vector Multiplication Algorithm

Journal Article · Mon Jul 03 00:00:00 EDT 2017 · Proceedings - IEEE International Parallel and Distributed Processing Symposium (IPDPS)

DOI:https://doi.org/10.1109/IPDPS.2017.76· OSTI ID:1525227

Azad, Ariful ^[1]; Buluc, Aydin ^[1]

Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)

We design and develop a work-efficient multithreaded algorithm for sparse matrix-sparse vector multiplication (SpMSpV) where the matrix, the input vector, and the output vector are all sparse. SpMSpV is an important primitive in the emerging GraphBLAS standard and is the workhorse of many graph algorithms including breadth-first search, bipartite graph matching, and maximal independent set. As thread counts increase, existing multithreaded SpMSpV algorithms can spend more time accessing the sparse matrix data structure than doing arithmetic. Our shared-memory parallel SpMSpV algorithm is work efficient in the sense that its total work is proportional to the number of arithmetic operations required. The key insight is to avoid each thread individually scan the list of matrix columns. Our algorithm is simple to implement and operates on existing column-based sparse matrix formats. It performs well on diverse matrices and vectors with heterogeneous sparsity patterns. A high-performance implementation of the algorithm attains up to 15x speedup on a 24-core Intel Ivy Bridge processor and up to 49x speedup on a 64-core Intel KNL manycore processor. In contrast to implementations of existing algorithms, the performance of our algorithm is sustained on a variety of different input types include matrices representing scale-free and high-diameter graphs.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)

Sponsoring Organization:: USDOE Office of Science (SC)

Grant/Contract Number:: AC02-05CH11231

OSTI ID:: 1525227

Journal Information:: Proceedings - IEEE International Parallel and Distributed Processing Symposium (IPDPS), Vol. 2017; Conference: 2017 IEEE International Parallel and Distributed Processing Symposium (IPDPS), Orlando, FL (United States), 29 May - 2 Jun 2017; ISSN 1530-2075

Publisher:: IEEECopyright Statement

Country of Publication:: United States

Language:: English

Citation Metrics:

Cited by: 21 works

Citation information provided by
Web of Science

References (13)

Evaluation Criteria for Sparse Matrix Storage Formats Langr, Daniel; Tvrdik, Pavel IEEE Transactions on Parallel and Distributed Systems, Vol. 27, Issue 2 https://doi.org/10.1109/TPDS.2015.2401575	journal	February 2016
GraphMat: high performance graph analytics made productive Sundaram, Narayanan; Satish, Nadathur; Patwary, Md Mostofa Ali Proceedings of the VLDB Endowment, Vol. 8, Issue 11 https://doi.org/10.14778/2809974.2809983	journal	July 2015
Fast Sparse Matrix and Sparse Vector Multiplication Algorithm on the GPU Yang, Carl; Wang, Yangzihao; Owens, John D. 2015 IEEE International Parallel and Distributed Processing Symposium Workshop (IPDPSW) https://doi.org/10.1109/IPDPSW.2015.77	conference	May 2015
Sparse Matrices in MATLAB: Design and Implementation Gilbert, John R.; Moler, Cleve; Schreiber, Robert SIAM Journal on Matrix Analysis and Applications, Vol. 13, Issue 1 https://doi.org/10.1137/0613024	journal	January 1992
Optimizing Sparse Matrix-Vector Multiplication for Large-Scale Data Analytics Buono, Daniele; Petrini, Fabrizio; Checconi, Fabio ICS '16: 2016 International Conference on Supercomputing, Proceedings of the 2016 International Conference on Supercomputing https://doi.org/10.1145/2925426.2926278	conference	June 2016
Distributed-Memory Algorithms for Maximum Cardinality Matching in Bipartite Graphs Azad, Ariful; Buluc, Aydin 2016 IEEE International Parallel and Distributed Processing Symposium (IPDPS) https://doi.org/10.1109/IPDPS.2016.103	conference	May 2016
Graph programming interface (GPI): a linear algebra programming model for large scale graph computations Ekanadham, K.; Horn, W. P.; Kumar, Manoj CF'16: Computing Frontiers Conference, Proceedings of the ACM International Conference on Computing Frontiers https://doi.org/10.1145/2903150.2903164	conference	May 2016
A Local Clustering Algorithm for Massive Graphs and Its Application to Nearly Linear Time Graph Partitioning Spielman, Daniel A.; Teng, Shang-Hua SIAM Journal on Computing, Vol. 42, Issue 1 https://doi.org/10.1137/080744888	journal	January 2013
Mathematical foundations of the GraphBLAS Kepner, Jeremy; Aaltonen, Peter; Bader, David 2016 IEEE High-Performance Extreme Computing Conference (HPEC), 2016 IEEE High Performance Extreme Computing Conference (HPEC) https://doi.org/10.1109/HPEC.2016.7761646	conference	September 2016
Parallel graph analytics Lenharth, Andrew; Nguyen, Donald; Pingali, Keshav Communications of the ACM, Vol. 59, Issue 5 https://doi.org/10.1145/2901919	journal	April 2016
Two Fast Algorithms for Sparse Matrices: Multiplication and Permuted Transposition Gustavson, Fred G. ACM Transactions on Mathematical Software, Vol. 4, Issue 3 https://doi.org/10.1145/355791.355796	journal	September 1978
Local Graph Partitioning using PageRank Vectors Andersen, Reid; Chung, Fan; Lang, Kevin 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06) https://doi.org/10.1109/FOCS.2006.44	conference	October 2006
The university of Florida sparse matrix collection Davis, Timothy A.; Hu, Yifan ACM Transactions on Mathematical Software, Vol. 38, Issue 1 https://doi.org/10.1145/2049662.2049663	journal	November 2011