A Scalable Graph Analytics Framework for Programming with Big Data in R (pbdR)

Hasan, S M Shamimul; Schmidt, Drew; Kannan, Ramakrishnan {ramki}; Imam, Neena

Title: A Scalable Graph Analytics Framework for Programming with Big Data in R (pbdR)

Full Record
Other Related Research

Abstract

Many disciplines such as biology, economics, engineering, physics, and the social sciences represent their data as graphs to capture patterns, trends, and associations. There are are many commercially available graph libraries in different programming languages to analyze these complex graphs. But there is no distributed graph library package in R - the popular statistical programming language to analyze graphs that bigger than a single machine's memory. Many domain experts prefer R over the numerous other alternatives. Towards this, we present a distributed graph analytics framework for R called programming with big graph using R (pBGR.) Our proposed framework leverages the Programming with Big Data in R (pbdR) ecosystem that provides scalable R packages for distributed computing in data science. We present an early prototype implementation of this framework using the distributed-memory parallel graph library CombBLAS and evaluate the framework's performance on leadership class computing platforms. Our experimental results demonstrate that the proposed framework is capable of performing large-scale parallel graph mining through the easyto-use R language. This enhanced graph processing capability coupled with other statistical tools already available in R, should be valuable to many domain experts.

Authors:

Hasan, S M Shamimul ^[1]; Schmidt, Drew ^[1];

^[1];

^[1]

ORNL

Publication Date:: 2019-12-01

Research Org.:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Org.:: USDOE

OSTI Identifier:: 1649576

DOE Contract Number:: AC05-00OR22725

Resource Type:: Conference

Resource Relation:: Conference: IEEE Big Data 2019 - Sixth International Workshop on High Performance Big Graph Data Management, Analysis, and Mining (BigGraphs 2019) - Los Angeles, California, United States of America - 12/9/2019 6:00:00 PM-12/12/2019 6:00:00 PM

Country of Publication:: United States

Language:: English

Citation Formats


                    Hasan, S M Shamimul, Schmidt, Drew, Kannan, Ramakrishnan, and Imam, Neena. A Scalable Graph Analytics Framework for Programming with Big Data in R (pbdR).  United States: N. p., 2019. 
        Web.

Copy to clipboard


                    Hasan, S M Shamimul, Schmidt, Drew, Kannan, Ramakrishnan, & Imam, Neena. A Scalable Graph Analytics Framework for Programming with Big Data in R (pbdR).  United States.

Copy to clipboard


                    Hasan, S M Shamimul, Schmidt, Drew, Kannan, Ramakrishnan, and Imam, Neena. 2019.  
        "A Scalable Graph Analytics Framework for Programming with Big Data in R (pbdR)".  United States.  https://www.osti.gov/servlets/purl/1649576.

Copy to clipboard


                    
@article{osti_1649576,

  title        = {A Scalable Graph Analytics Framework for Programming with Big Data in R (pbdR)},

  author       = {Hasan, S M Shamimul and Schmidt, Drew and Kannan, Ramakrishnan and Imam, Neena},

  abstractNote = {Many disciplines such as biology, economics, engineering, physics, and the social sciences represent their data as graphs to capture patterns, trends, and associations. There are are many commercially available graph libraries in different programming languages to analyze these complex graphs. But there is no distributed graph library package in R - the popular statistical programming language to analyze graphs that bigger than a single machine's memory. Many domain experts prefer R over the numerous other alternatives. Towards this, we present a distributed graph analytics framework for R called programming with big graph using R (pBGR.) Our proposed framework leverages the Programming with Big Data in R (pbdR) ecosystem that provides scalable R packages for distributed computing in data science. We present an early prototype implementation of this framework using the distributed-memory parallel graph library CombBLAS and evaluate the framework's performance on leadership class computing platforms. Our experimental results demonstrate that the proposed framework is capable of performing large-scale parallel graph mining through the easyto-use R language. This enhanced graph processing capability coupled with other statistical tools already available in R, should be valuable to many domain experts.},

  doi          = {},

  url          = {https://www.osti.gov/biblio/1649576},
  journal      = {},
number       = ,

  volume       = ,

  place        = {United States},

  year         = {2019},

  month        = {12}

}

Copy to clipboard

Conference:

View Conference (1.23 MB)

Other availability

Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share:

Export Metadata

Save to My Library

Similar records in OSTI.GOV collections:

A Reasoning And Hypothesis-Generation Framework Based On Scalable Graph Analytics

Conference Sukumar, Sreenivas

Finding actionable insights from data has always been difficult. As the scale and forms of data increase tremendously, the task of finding value becomes even more challenging. Data scientists at Oak Ridge National Laboratory are leveraging unique leadership infrastructure (e.g. Urika-XA and Urika-GD appliances) to develop scalable algorithms for semantic, logical and statistical reasoning with unstructured Big Data. We present the deployment of such a framework called ORiGAMI (Oak Ridge Graph Analytics for Medical Innovations) on the National Library of Medicine s SEMANTIC Medline (archive of medical knowledge since 1994). Medline contains over 70 million knowledge nuggets published in 23.5more »« less
Programming with BIG data in R: Scaling analytics from one to thousands of nodes

Journal Article Schmidt, Drew; Chen, Wei; Matheson, Michael; ... - Big Data Research

Here, we present a tutorial overview showing how one can achieve scalable performance with R. We do so by utilizing several package extensions, including those from the pbdR project. These packages consist of high performance, high-level interfaces to and extensions of MPI, PBLAS, ScaLAPACK, I/O libraries, profiling libraries, and more. While these libraries shine brightest on large distributed platforms, they also work rather well on small clusters and often, surprisingly, even on a laptop with only two cores. Our tutorial begins with recommendations on how to get more performance out of your R code before considering parallel implementations. Because Rmore »« less
Cited by 11
https://doi.org/10.1016/j.bdr.2016.10.002

Full Text Available
Developing Mango Graph Studio and its Applications for Bioinformatics and Systems Biology (SBIR Phase I Grant Final Technical Report)

Technical Report Chou, Hui-Hsien; Vanous, Kimberly

Modern genomic and post-genomic research produced huge amounts of heterogeneous data that must be integrated and analyzed together. Systems biology aims at utilizing these data to help model complex biological systems and further our understanding of living beings, so we may cure more diseases or harvest more renewable bio-energies in the future. However, construction and manipulation of complex graph or network data structures to represent the heterogeneous data that are linked to each other require computer science skills. Software engineers who are capable of these in-depth programming efforts may not be able to form biological analysis algorithms or to interpretmore »« less
Scalable training of graph convolutional neural networks for fast and accurate predictions of HOMO-LUMO gap in molecules

Journal Article Choi, Jong; Zhang, Pei; Mehta, Kshitij; ... - Journal of Cheminformatics

Abstract Graph Convolutional Neural Network (GCNN) is a popular class of deep learning (DL) models in material science to predict material properties from the graph representation of molecular structures. Training an accurate and comprehensive GCNN surrogate for molecular design requires large-scale graph datasets and is usually a time-consuming process. Recent advances in GPUs and distributed computing open a path to reduce the computational cost for GCNN training effectively. However, efficient utilization of high performance computing (HPC) resources for training requires simultaneously optimizing large-scale data management and scalable stochastic batched optimization techniques. In this work, we focus on building GCNN modelsmore »« less
https://doi.org/10.1186/s13321-022-00652-1
Oak Ridge Graph Analytics for Medical Innovation (ORiGAMI)

Software Roberts, Larry; Lee, Sangkeun (Matt)

In this era of data-driven decisions and discovery where Big Data is producing Bigger Data, data scientists at the Oak Ridge National Laboratory are leveraging unique leadership infrastructure (e.g., Urika XA and Urika GD appliances) to develop scalable algorithms for semantic, logical and statistical reasoning with Big Data (i.e., data stored in databases as well as unstructured data in documents). ORiGAMI is a next-generation knowledge-discovery framework that is: (a) knowledge nurturing (i.e., evolves seamlessly with newer knowledge and data), (b) smart and curious (i.e. using information-foraging and reasoning algorithms to digest content) and (c) synergistic (i.e., interfaces computers with whatmore »« less
View Software

Similar Records