SHMEMGraph: Efficient and Balanced Graph Processing Using One-Sided Communication
Abstract
State-of-the-art synchronous graph processing frameworks face both inefficiency and imbalance issues that cause their performance to be suboptimal. These issues include the inefficiency of communication and the imbalanced graph computation/communication costs in an iteration. We propose to replace their conventional two-sided communication model with the one-sided counterpart. Accordingly, we design SHMEMGraph, an efficient and balanced graph processing framework that is formulated across a global memory space and takes advantage of the flexibility and efficiency of one-sided communication for graph processing. Through an efficient one-sided communication channel, SHMEMGraph utilizes the high-performance operations with RDMA while minimizing the resource contention within a computer node. In addition, SHMEMGraph synthesizes a number of optimizations to address both computation imbalance and communication imbalance. By using a graph of 1 billion edges, our evaluation shows that compared to the state-of-the-art Gemini framework, SHMEMGraph achieves an average improvement of 35.5% in terms of job completion time for five representative graph algorithms.
- Authors:
-
- Florida State University, Tallahassee
- ORNL
- Publication Date:
- Research Org.:
- Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC)
- OSTI Identifier:
- 1468157
- DOE Contract Number:
- AC05-00OR22725
- Resource Type:
- Conference
- Resource Relation:
- Conference: IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing - Washington D.C, District of Columbia, United States of America - 5/1/2018 12:00:00 PM-5/4/2018 12:00:00 PM
- Country of Publication:
- United States
- Language:
- English
Citation Formats
Fu, Huansong, Gorentla Venkata, Manjunath, Salman, Shaeke, Imam, Neena, and Yu, Weikuan. SHMEMGraph: Efficient and Balanced Graph Processing Using One-Sided Communication. United States: N. p., 2018.
Web. doi:10.1109/CCGRID.2018.00078.
Fu, Huansong, Gorentla Venkata, Manjunath, Salman, Shaeke, Imam, Neena, & Yu, Weikuan. SHMEMGraph: Efficient and Balanced Graph Processing Using One-Sided Communication. United States. https://doi.org/10.1109/CCGRID.2018.00078
Fu, Huansong, Gorentla Venkata, Manjunath, Salman, Shaeke, Imam, Neena, and Yu, Weikuan. Tue .
"SHMEMGraph: Efficient and Balanced Graph Processing Using One-Sided Communication". United States. https://doi.org/10.1109/CCGRID.2018.00078. https://www.osti.gov/servlets/purl/1468157.
@article{osti_1468157,
title = {SHMEMGraph: Efficient and Balanced Graph Processing Using One-Sided Communication},
author = {Fu, Huansong and Gorentla Venkata, Manjunath and Salman, Shaeke and Imam, Neena and Yu, Weikuan},
abstractNote = {State-of-the-art synchronous graph processing frameworks face both inefficiency and imbalance issues that cause their performance to be suboptimal. These issues include the inefficiency of communication and the imbalanced graph computation/communication costs in an iteration. We propose to replace their conventional two-sided communication model with the one-sided counterpart. Accordingly, we design SHMEMGraph, an efficient and balanced graph processing framework that is formulated across a global memory space and takes advantage of the flexibility and efficiency of one-sided communication for graph processing. Through an efficient one-sided communication channel, SHMEMGraph utilizes the high-performance operations with RDMA while minimizing the resource contention within a computer node. In addition, SHMEMGraph synthesizes a number of optimizations to address both computation imbalance and communication imbalance. By using a graph of 1 billion edges, our evaluation shows that compared to the state-of-the-art Gemini framework, SHMEMGraph achieves an average improvement of 35.5% in terms of job completion time for five representative graph algorithms.},
doi = {10.1109/CCGRID.2018.00078},
url = {https://www.osti.gov/biblio/1468157},
journal = {},
number = ,
volume = ,
place = {United States},
year = {2018},
month = {5}
}