Brief announcement: Hypergraph parititioning for parallel sparse matrix-matrix multiplication

Ballard, Grey; Druinsky, Alex; Knight, Nicholas; Schwartz, Oded

doi:10.1145/2755573.2755613

Brief announcement: Hypergraph parititioning for parallel sparse matrix-matrix multiplication

Journal Article · Wed Dec 31 23:00:00 EST 2014 · ACM Transactions on Parallel Computing

DOI:https://doi.org/10.1145/2755573.2755613· OSTI ID:1303161

Ballard, Grey ^[1]; Druinsky, Alex ^[2]; Knight, Nicholas ^[3]; Schwartz, Oded ^[4]

Sandia National Lab. (SNL-CA), Livermore, CA (United States)
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Univ. of California, Berkeley, CA (United States)
Hebrew Univ., Jerusalem (Israel)

The performance of parallel algorithms for sparse matrix-matrix multiplication is typically determined by the amount of interprocessor communication performed, which in turn depends on the nonzero structure of the input matrices. In this paper, we characterize the communication cost of a sparse matrix-matrix multiplication algorithm in terms of the size of a cut of an associated hypergraph that encodes the computation for a given input nonzero structure. Obtaining an optimal algorithm corresponds to solving a hypergraph partitioning problem. Furthermore, our hypergraph model generalizes several existing models for sparse matrix-vector multiplication, and we can leverage hypergraph partitioners developed for that computation to improve application-specific algorithms for multiplying sparse matrices.

View Accepted Manuscript (DOE)

Research Organization:: Sandia National Laboratories (SNL-CA), Livermore, CA (United States)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA)

Grant/Contract Number:: AC04-94AL85000

OSTI ID:: 1303161

Report Number(s):: SAND--2016-4718J; 640462

Journal Information:: ACM Transactions on Parallel Computing, Journal Name: ACM Transactions on Parallel Computing

Country of Publication:: United States

Language:: English

References (8)

Graph Algorithms Lengauer, Thomas Combinatorial Algorithms for Integrated Circuit Layout https://doi.org/10.1007/978-3-322-92106-2_3	book	January 1990
Hypergraph Partitioning for Automatic Memory Hierarchy Management Krishnamoorthy, Sriram; Catalyurek, Umit; Nieplocha, Jarek ACM/IEEE SC 2006 Conference (SC'06) https://doi.org/10.1109/SC.2006.36	conference	November 2006
On Two-Dimensional Sparse Matrix Partitioning: Models, Methods, and a Recipe Çatalyürek, Ümt V.; Aykanat, Cevdet; Uçar, Bora SIAM Journal on Scientific Computing, Vol. 32, Issue 2 https://doi.org/10.1137/080737770	journal	January 2010
Parallel Sparse Matrix-Matrix Multiplication and Indexing: Implementation and Experiments Buluç, Aydin; Gilbert, John R. SIAM Journal on Scientific Computing, Vol. 34, Issue 4 https://doi.org/10.1137/110848244	journal	January 2012
Simultaneous Input and Output Matrix Partitioning for Outer-Product--Parallel Sparse Matrix-Matrix Multiplication Akbudak, Kadir; Aykanat, Cevdet SIAM Journal on Scientific Computing, Vol. 36, Issue 5 https://doi.org/10.1137/13092589X	journal	January 2014
Data management and query---Hypergraph partitioning for automatic memory hierarchy management Krishnamoorthy, Sriram; Catalyurek, Umit; Nieplocha, Jarek Proceedings of the 2006 ACM/IEEE conference on Supercomputing - SC '06 https://doi.org/10.1145/1188455.1188558	conference	January 2006
Communication optimal parallel multiplication of sparse random matrices Ballard, Grey; Buluc, Aydin; Demmel, James Proceedings of the 25th ACM symposium on Parallelism in algorithms and architectures - SPAA '13 https://doi.org/10.1145/2486159.2486196	conference	January 2013
Reducing Communication Costs for Sparse Matrix Multiplication within Algebraic Multigrid Ballard, Grey Malone; Hu, Jonathan Joseph; Siefert, Christopher https://doi.org/10.2172/1504845	report	September 2015

Cited By (18)

MPI-FAUN: An MPI-Based Framework for Alternating-Updating Nonnegative Matrix Factorization Kannan, Ramakrishnan; Ballard, Grey; Park, Haesun IEEE Transactions on Knowledge and Data Engineering, Vol. 30, Issue 3 https://doi.org/10.1109/tkde.2017.2767592	journal	March 2018
Exploiting Multiple Levels of Parallelism in Sparse Matrix-Matrix Multiplication Azad, Ariful; Ballard, Grey; Buluç, Aydin SIAM Journal on Scientific Computing, Vol. 38, Issue 6 https://doi.org/10.1137/15m104253x	journal	January 2016
Partitioning Models for Scaling Parallel Sparse Matrix-Matrix Multiplication Akbudak, Kadir; Selvitopi, Oguz; Aykanat, Cevdet ACM Transactions on Parallel Computing, Vol. 4, Issue 3 https://doi.org/10.1145/3155292	journal	April 2018
Reducing Communication Costs for Sparse Matrix Multiplication within Algebraic Multigrid Ballard, Grey Malone; Hu, Jonathan Joseph; Siefert, Christopher https://doi.org/10.2172/1504845	report	September 2015
Simulating Population Protocols in Sub-Constant Time per Interaction Berenbrink, Petra; Hammer, David; Kaaser, Dominik Schloss Dagstuhl - Leibniz-Zentrum für Informatik https://doi.org/10.4230/lipics.esa.2020.16	text	January 2020
Hypergraph Partitioning for Sparse Matrix-Matrix Multiplication Ballard, Grey; Druinsky, Alex; Knight, Nicholas arXiv https://doi.org/10.48550/arxiv.1603.05627	preprint	January 2016
Distributed-Memory Tensor Completion for Generalized Loss Functions in Python using New Sparse Tensor Kernels Singh, Navjot; Zhang, Zecheng; Wu, Xiaoxiao arXiv https://doi.org/10.48550/arxiv.1910.02371	preprint	January 2019
Optimizing High Performance Markov Clustering for Pre-Exascale Architectures Selvitopi, Oguz; Hussain, Md Taufique; Azad, Ariful arXiv https://doi.org/10.48550/arxiv.2002.10083	text	January 2020
A study on using uncertain time series matching algorithms for MapReduce applications: STATISTICAL PATTERN MATCHING ALGORITHMS FOR MAPREDUCE APPLICATIONS Rizvandi, Nikzad Babaii; Taheri, Javid; Moraveji, Reza Concurrency and Computation: Practice and Experience, Vol. 25, Issue 12 https://doi.org/10.1002/cpe.2895	journal	July 2012
Privacy-Preserving Data Analytics Quoc, Do Le; Beck, Martin; Bhatotia, Pramod Encyclopedia of Big Data Technologies https://doi.org/10.1007/978-3-319-63962-8_152-1	book	January 2018
Embedded pattern formation by asynchronous robots without chirality Cicerone, Serafino; Di Stefano, Gabriele; Navarra, Alfredo Distributed Computing, Vol. 32, Issue 4 https://doi.org/10.1007/s00446-018-0333-7	journal	May 2018
An Analysis Framework for Distributed Hierarchical Directories Sharma, Gokarna; Busch, Costas Algorithmica, Vol. 71, Issue 2 https://doi.org/10.1007/s00453-013-9803-2	journal	June 2013
Optimizing partitioned CSR-based SpGEMM on the Sunway TaihuLight Chen, Yuedan; Xiao, Guoqing; Yang, Wangdong Neural Computing and Applications, Vol. 32, Issue 10 https://doi.org/10.1007/s00521-019-04121-z	journal	March 2019
A novel robust on-line protocol for load-balancing in structured peer-to-peer systems Tsatsanifos, George; Samoladas, Vasilis Computing, Vol. 94, Issue 8-10 https://doi.org/10.1007/s00607-012-0199-7	journal	July 2012
Numerical algorithms for high-performance computational science Dongarra, Jack; Grigori, Laura; Higham, Nicholas J. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 378, Issue 2166 https://doi.org/10.1098/rsta.2019.0066	journal	January 2020
Distributed MST and Routing in Almost Mixing Time Ghaffari, Mohsen; Kuhn, Fabian; Su, Hsin-Hao PODC '17: ACM Symposium on Principles of Distributed Computing, Proceedings of the ACM Symposium on Principles of Distributed Computing https://doi.org/10.1145/3087801.3087827	conference	July 2017
Large-Scale Indexing, Discovery, and Ranking for the Internet of Things (IoT) Fathy, Yasmin; Barnaghi, Payam; Tafazolli, Rahim ACM Computing Surveys, Vol. 51, Issue 2 https://doi.org/10.1145/3154525	journal	June 2018
On Counting the Population Size Berenbrink, Petra; Kaaser, Dominik; Radzik, Tomasz PODC '19: ACM Symposium on Principles of Distributed Computing, Proceedings of the 2019 ACM Symposium on Principles of Distributed Computing https://doi.org/10.1145/3293611.3331631	conference	July 2019

Similar Records

A nested dissection approach to sparse matrix partitioning for parallel computations.

Journal Article · Fri Aug 01 00:00:00 EDT 2008 · Proposed for publication in SIAM Journal on Scientific Computing. · OSTI ID:947836

Partitioning Models for Scaling Parallel Sparse Matrix-Matrix Multiplication

Journal Article · Tue Jan 02 23:00:00 EST 2018 · ACM Transactions on Parallel Computing · OSTI ID:1525287

Parallel hypergraph partitioning for scientific computing.

Conference · Fri Jul 01 00:00:00 EDT 2005 · OSTI ID:968387

Related Subjects

97 MATHEMATICS AND COMPUTING

Brief announcement: Hypergraph parititioning for parallel sparse matrix-matrix multiplication

Citation Formats

References (8)

Cited By (18)

Similar Records

Related Subjects