GoFFish: A Sub-Graph Centric Framework for Large-Scale Graph Analytics1

Conference · Mon Aug 25 04:00:00 EDT 2014

DOI:https://doi.org/10.1007/978-3-319-09873-9_38· OSTI ID:1332680

Simmhan, Yogesh ^[1]; Kumbhare, Alok; Wickramaarachchi, Charith; Nagarkar, Soonil; Ravi, Santosh; Raghavendra, Cauligi; Prasanna, Viktor

---

1. Power System Information & Advanced Technologies LADWP Power System Engineering Division

Large scale graph processing is a major research area for Big Data exploration. Vertex centric programming models like Pregel are gaining traction due to their simple abstraction that allows for scalable execution on distributed systems naturally. However, there are limitations to this approach which cause vertex centric algorithms to under-perform due to poor compute to communication overhead ratio and slow convergence of iterative superstep. In this paper we introduce GoFFish a scalable sub-graph centric framework co-designed with a distributed persistent graph storage for large scale graph analytics on commodity clusters. We introduce a sub-graph centric programming abstraction that combines the scalability of a vertex centric approach with the flexibility of shared memory sub-graph computation. We map Connected Components, SSSP and PageRank algorithms to this model to illustrate its flexibility. Further, we empirically analyze GoFFish using several real world graphs and demonstrate its significant performance improvement, orders of magnitude in some cases, compared to Apache Giraph, the leading open source vertex centric implementation. We map Connected Components, SSSP and PageRank algorithms to this model to illustrate its flexibility. Further, we empirically analyze GoFFish using several real world graphs and demonstrate its significant performance improvement, orders of magnitude in some cases, compared to Apache Giraph, the leading open source vertex centric implementation.

• Research Organization: City of Los Angeles Department
• Sponsoring Organization: USDOE Office of Electricity Delivery and Energy Reliability (OE)
• DOE Contract Number: OE0000192
• OSTI ID: 1332680
• Report Number(s): DOE-USC-00192-87
• Country of Publication: United States
• Language: English