SHMEMGraph: Efficient and Balanced Graph Processing Using One-Sided Communication

Conference · Tue May 01 00:00:00 EDT 2018

DOI:https://doi.org/10.1109/CCGRID.2018.00078· OSTI ID:1468157

Fu, Huansong ^[1]; Gorentla Venkata, Manjunath  ^[2]; Salman, Shaeke ^[1]; Imam, Neena  ^[2]; Yu, Weikuan ^[1]

---

1. Florida State University, Tallahassee
2. ORNL

State-of-the-art synchronous graph processing frameworks face both inefficiency and imbalance issues that cause their performance to be suboptimal. These issues include the inefficiency of communication and the imbalanced graph computation/communication costs in an iteration. We propose to replace their conventional two-sided communication model with the one-sided counterpart. Accordingly, we design SHMEMGraph, an efficient and balanced graph processing framework that is formulated across a global memory space and takes advantage of the flexibility and efficiency of one-sided communication for graph processing. Through an efficient one-sided communication channel, SHMEMGraph utilizes the high-performance operations with RDMA while minimizing the resource contention within a computer node. In addition, SHMEMGraph synthesizes a number of optimizations to address both computation imbalance and communication imbalance. By using a graph of 1 billion edges, our evaluation shows that compared to the state-of-the-art Gemini framework, SHMEMGraph achieves an average improvement of 35.5% in terms of job completion time for five representative graph algorithms.

• Research Organization: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
• Sponsoring Organization: USDOE Office of Science (SC)
• DOE Contract Number: AC05-00OR22725
• OSTI ID: 1468157
• Resource Relation: Conference: IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing - Washington D.C, District of Columbia, United States of America - 5/1/2018 12:00:00 PM-5/4/2018 12:00:00 PM
• Country of Publication: United States
• Language: English