skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Parallel processing of filtered queries in attributed semantic graphs

Journal Article · · Journal of Parallel and Distributed Computing
 [1];  [2];  [3];  [3];  [4];  [3];  [5];  [5]
  1. Univ. of California, Santa Barbara, CA (United States)
  2. Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States)
  3. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
  4. Univ. College Dublin (Ireland)
  5. Univ. of California, Berkeley, CA (United States)
+ Show Author Affiliations

Execution of complex analytic queries on massive semantic graphs is a challenging problem in big-data analytics that requires high-performance parallel computing. In a semantic graph, vertices and edges carry attributes of various types and the analytic queries typically depend on the values of these attributes. Thus, the computation must view the graph through a filter that passes only those individual vertices and edges of interest. Previous investigations have developed Knowledge Discovery Toolbox (KDT), a sophisticated Python library for parallel graph computations. In KDT, the user can write custom graph algorithms by specifying operations between edges and vertices (semiring operations). The user can also customize existing graph algorithms by writing filters. While the high-level language for this customization enables domain scientists to productively express their graph analytics requirements, the customized queries perform poorly due to the overhead of having to call into the Python virtual machine for each vertex and edge. In this work, we use the Selective Embedded Just-In-Time Specialization (SEJITS) approach to automatically translate semiring operations and filters defined by programmers into a lower-level efficiency language, bypassing the upcall into Python. We evaluate our approach by comparing it with the high-performance Combinatorial BLAS engine and show that our approach combines the benefits of programming in a high-level language with executing in a low-level parallel environment. We increase the system's flexibility by developing techniques that provide users with the ability to define new vertex and edge types from Python. We also present a new Roofline model for graph traversals and show that we achieve performance that is significantly closer to the bounds suggested by the Roofline. Finally, to further understand the complex interaction with the underlying architecture, we present an analysis using performance counters that quantifies the improvement in hardware behavior in the context our SEJITS methodology. Comprehensively, we demonstrate the first known solution to the problem of obtaining high performance from a productivity language when applying graph algorithms selectively on semantic graphs with hundreds of millions of edges and scaling to thousands of processors for graphs.

Research Organization:
Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)
Sponsoring Organization:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
Grant/Contract Number:
AC02-05CH11231; FA8750-10-1-0191; CNS-0960316; 8-482526701
OSTI ID:
1524030
Alternate ID(s):
OSTI ID: 1249863
Journal Information:
Journal of Parallel and Distributed Computing, Vol. 79-80, Issue C; ISSN 0743-7315
Publisher:
ElsevierCopyright Statement
Country of Publication:
United States
Language:
English
Citation Metrics:
Cited by: 9 works
Citation information provided by
Web of Science

Figures / Tables (29)

Figure 1(p. 4)figure Figure 1
Figure 2(p. 5)figure Figure 2
Algorithm 1(p. 6)figure Algorithm 1
Figure 3(p. 7)figure Figure 3
Figure 4(p. 7)figure Figure 4
Figure 5(p. 9)figure Figure 5
Figure 6(p. 10)figure Figure 6
Figure 7(p. 11)figure Figure 7
Figure 8(p. 12)figure Figure 8
Figure 9(p. 13)figure Figure 9
Table 1(p. 14)table Table 1
Figure 10(p. 17)figure Figure 10
Table 2(p. 17)table Table 2
Table 3(p. 18)table Table 3
Figure 11(p. 19)figure Figure 11
Table 4(p. 21)table Table 4
Table 5(p. 21)table Table 5
Figure 12(p. 22)figure Figure 12
Figure 13(p. 23)figure Figure 13
Figure 14(p. 23)figure Figure 14
Figure 15(p. 24)figure Figure 15
Figure 16(p. 25)figure Figure 16
Figure 17(p. 26)figure Figure 17
Figure 18(p. 27)figure Figure 18
Figure 19(p. 28)figure Figure 19
Table 6(p. 28)table Table 6
Figure 20(p. 29)figure Figure 20
Figure 21(p. 30)figure Figure 21
Table 7(p. 31)table Table 7

Similar Records

High-performance analysis of filtered semantic graphs
Conference · Sun Jan 01 00:00:00 EST 2012 · OSTI ID:1524030
+4 more
Scalable Pattern Matching in Metadata Graphs via Constraint Checking
Journal Article · Mon Jan 04 00:00:00 EST 2021 · ACM Transactions on Parallel Computing · OSTI ID:1524030
+2 more
Algorithms and architectures for high performance analysis of semantic graphs.
Technical Report · Thu Sep 01 00:00:00 EDT 2005 · OSTI ID:1524030

Related Subjects

97 MATHEMATICS AND COMPUTING
Graph analysis systems
Attributed semantic graphs
Graph filtering
Parallel computing
Knowledge discovery
Domain-specific languages
SEJITS
High-performance graph analysis