Graph processing platforms at scale: practices and experiences

Conference · Thu Jan 01 00:00:00 EST 2015

OSTI ID:1185842

Lim, Seung-Hwan ^[1]; Lee, Sangkeun ^[1]; Brown, Tyler C ^[1]; Sukumar, Sreenivas R ^[1]; Ganesh, Gautam ^[1]

---

1. ORNL

Graph analysis unveils hidden associations of data in many phenomena and artifacts, such as road network, social networks, genomic information, and scientific collaboration. Unfortunately, a wide diversity in the characteristics of graphs and graph operations make it challenging to find a right combination of tools and implementation of algorithms to discover desired knowledge from the target data set. This study presents an extensive empirical study of three representative graph processing platforms: Pegasus, GraphX, and Urika. Each system represents a combination of options in data model, processing paradigm, and infrastructure. We benchmarked each platform using three popular graph operations, degree distribution, connected components, and PageRank over a variety of real-world graphs. Our experiments show that each graph processing platform shows different strength, depending the type of graph operations. While Urika performs the best in non-iterative operations like degree distribution, GraphX outputforms iterative operations like connected components and PageRank. In addition, we discuss challenges to optimize the performance of each platform over large scale real world graphs.

• Research Organization: Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
• Sponsoring Organization: USDOE Laboratory Directed Research and Development (LDRD) Program
• DOE Contract Number: AC05-00OR22725
• OSTI ID: 1185842
• Resource Relation: Conference: IEEE International Symposium on Performance Analysis of Systems and Software, Philadelphia, PA, USA, 20150329, 20150329
• Country of Publication: United States
• Language: English