Reduce Operations: Send Volume Balancing While Minimizing Latency
- Bilkent Univ., Ankara (Turkey)
- Bilkent Univ., Ankara (Turkey); Sandia National Lab. (SNL-NM), Albuquerque, NM (United States). Center for Computing Research
- Middle East Technical Univ., Ankara (Turkey); The Ohio State Univ., Columbus, OH (United States); Bilkent Univ., Ankara (Turkey). Dept. of Computer Engineering
Communication hypergraph model was introduced in a two-phase setting for encapsulating multiple communication cost metrics (bandwidth and latency), which are proven to be important in parallelizing irregular applications. In the first phase, computational-task-to-processor assignment is performed with the objective of minimizing total volume while maintaining computational load balance. In the second phase, communication-task-to-processor assignment is performed with the objective of minimizing total number of messages while maintaining communication-volume balance. The reduce-communication hypergraph model suffers from failing to correctly encapsulate send-volume balancing. We propose a novel vertex weighting scheme that enables part weights to correctly encode send-volume loads of processors for send-volume balancing. The model also suffers from increasing the total communication volume during partitioning. To decrease this increase, we propose a method that utilizes the recursive bipartitioning framework and refines each bipartition by vertex swaps. For performance evaluation, we consider column-parallel SpMV, which is one of the most widely known applications in which the reduce-task assignment problem arises. Extensive experiments on 313 matrices show that, compared to the existing model, the proposed models achieve considerable improvements in all communication cost metrics. Furthermore, these improvements lead to an average decrease of 30% in parallel SpMV time on 512 processors for 70 matrices with high irregularity.
- Research Organization:
- Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States)
- Sponsoring Organization:
- USDOE National Nuclear Security Administration (NNSA)
- Grant/Contract Number:
- AC04-94AL85000
- OSTI ID:
- 1595427
- Alternate ID(s):
- OSTI ID: 1601274
- Report Number(s):
- SAND--2020-0151J; 681894
- Journal Information:
- IEEE Transactions on Parallel and Distributed Systems, Journal Name: IEEE Transactions on Parallel and Distributed Systems Journal Issue: 6 Vol. 31; ISSN 1045-9219
- Publisher:
- IEEECopyright Statement
- Country of Publication:
- United States
- Language:
- English
Similar Records
A Recursive Hypergraph Bipartitioning Framework for Reducing Bandwidth and Latency Costs Simultaneously