Graph processing platforms at scale: practices and experiences
Abstract
Graph analysis unveils hidden associations of data in many phenomena and artifacts, such as road network, social networks, genomic information, and scientific collaboration. Unfortunately, a wide diversity in the characteristics of graphs and graph operations make it challenging to find a right combination of tools and implementation of algorithms to discover desired knowledge from the target data set. This study presents an extensive empirical study of three representative graph processing platforms: Pegasus, GraphX, and Urika. Each system represents a combination of options in data model, processing paradigm, and infrastructure. We benchmarked each platform using three popular graph operations, degree distribution, connected components, and PageRank over a variety of real-world graphs. Our experiments show that each graph processing platform shows different strength, depending the type of graph operations. While Urika performs the best in non-iterative operations like degree distribution, GraphX outputforms iterative operations like connected components and PageRank. In addition, we discuss challenges to optimize the performance of each platform over large scale real world graphs.
- Authors:
-
- ORNL
- Publication Date:
- Research Org.:
- Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
- Sponsoring Org.:
- USDOE Laboratory Directed Research and Development (LDRD) Program
- OSTI Identifier:
- 1185842
- DOE Contract Number:
- AC05-00OR22725
- Resource Type:
- Conference
- Resource Relation:
- Conference: IEEE International Symposium on Performance Analysis of Systems and Software, Philadelphia, PA, USA, 20150329, 20150329
- Country of Publication:
- United States
- Language:
- English
Citation Formats
Lim, Seung-Hwan, Lee, Sangkeun, Brown, Tyler C, Sukumar, Sreenivas R, and Ganesh, Gautam. Graph processing platforms at scale: practices and experiences. United States: N. p., 2015.
Web.
Lim, Seung-Hwan, Lee, Sangkeun, Brown, Tyler C, Sukumar, Sreenivas R, & Ganesh, Gautam. Graph processing platforms at scale: practices and experiences. United States.
Lim, Seung-Hwan, Lee, Sangkeun, Brown, Tyler C, Sukumar, Sreenivas R, and Ganesh, Gautam. 2015.
"Graph processing platforms at scale: practices and experiences". United States. https://www.osti.gov/servlets/purl/1185842.
@article{osti_1185842,
title = {Graph processing platforms at scale: practices and experiences},
author = {Lim, Seung-Hwan and Lee, Sangkeun and Brown, Tyler C and Sukumar, Sreenivas R and Ganesh, Gautam},
abstractNote = {Graph analysis unveils hidden associations of data in many phenomena and artifacts, such as road network, social networks, genomic information, and scientific collaboration. Unfortunately, a wide diversity in the characteristics of graphs and graph operations make it challenging to find a right combination of tools and implementation of algorithms to discover desired knowledge from the target data set. This study presents an extensive empirical study of three representative graph processing platforms: Pegasus, GraphX, and Urika. Each system represents a combination of options in data model, processing paradigm, and infrastructure. We benchmarked each platform using three popular graph operations, degree distribution, connected components, and PageRank over a variety of real-world graphs. Our experiments show that each graph processing platform shows different strength, depending the type of graph operations. While Urika performs the best in non-iterative operations like degree distribution, GraphX outputforms iterative operations like connected components and PageRank. In addition, we discuss challenges to optimize the performance of each platform over large scale real world graphs.},
doi = {},
url = {https://www.osti.gov/biblio/1185842},
journal = {},
number = ,
volume = ,
place = {United States},
year = {Thu Jan 01 00:00:00 EST 2015},
month = {Thu Jan 01 00:00:00 EST 2015}
}