skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: BCL: A Cross-Platform Distributed Data Structures Library

Abstract

One-sided communication is a useful paradigm for irregular parallel applications, but most one-sided programming environments, including MPI's one-sided interface and PGAS programming languages, lack application-level libraries to support these applications. We present the Berkeley Container Library, a set of generic, cross-platform, high-performance data structures for irregular applications, including queues, hash tables, Bloom filters and more. BCL is written in C++ using an internal DSL called the BCL Core that provides one-sided communication primitives such as remote get and remote put operations. The BCL Core has backends for MPI, OpenSHMEM, GASNet-EX, and UPC++, allowing BCL data structures to be used natively in programs written using any of these programming environments. Along with our internal DSL, we present the BCL ObjectContainer abstraction, which allows BCL data structures to transparently serialize complex data types while maintaining efficiency for primitive types. We also introduce the set of BCL data structures and evaluate their performance across a number of high-performance computing systems, demonstrating that BCL programs are competitive with hand-optimized code, even while hiding many of the underlying details of message aggregation, serialization, and synchronization.

Authors:
 [1];  [1];  [1]
  1. Univ. of California, Berkeley, CA (United States); Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Publication Date:
Research Org.:
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Sponsoring Org.:
USDOE Office of Science (SC)
OSTI Identifier:
1581320
DOE Contract Number:  
AC02-05CH11231
Resource Type:
Journal Article
Journal Name:
Proceedings of the 48th International Conference on Parallel Processing - ICPP 2019
Additional Journal Information:
Conference: 48. International Conference on Parallel Processing, Kyoto (Japan), 5-8 Aug 2019; Related Information: abs/1810.13029
Country of Publication:
United States
Language:
English

Citation Formats

Brock, Benjamin, Buluç, Aydın, and Yelick, Katherine. BCL: A Cross-Platform Distributed Data Structures Library. United States: N. p., 2019. Web. doi:10.1145/3337821.3337912.
Brock, Benjamin, Buluç, Aydın, & Yelick, Katherine. BCL: A Cross-Platform Distributed Data Structures Library. United States. doi:10.1145/3337821.3337912.
Brock, Benjamin, Buluç, Aydın, and Yelick, Katherine. Thu . "BCL: A Cross-Platform Distributed Data Structures Library". United States. doi:10.1145/3337821.3337912.
@article{osti_1581320,
title = {BCL: A Cross-Platform Distributed Data Structures Library},
author = {Brock, Benjamin and Buluç, Aydın and Yelick, Katherine},
abstractNote = {One-sided communication is a useful paradigm for irregular parallel applications, but most one-sided programming environments, including MPI's one-sided interface and PGAS programming languages, lack application-level libraries to support these applications. We present the Berkeley Container Library, a set of generic, cross-platform, high-performance data structures for irregular applications, including queues, hash tables, Bloom filters and more. BCL is written in C++ using an internal DSL called the BCL Core that provides one-sided communication primitives such as remote get and remote put operations. The BCL Core has backends for MPI, OpenSHMEM, GASNet-EX, and UPC++, allowing BCL data structures to be used natively in programs written using any of these programming environments. Along with our internal DSL, we present the BCL ObjectContainer abstraction, which allows BCL data structures to transparently serialize complex data types while maintaining efficiency for primitive types. We also introduce the set of BCL data structures and evaluate their performance across a number of high-performance computing systems, demonstrating that BCL programs are competitive with hand-optimized code, even while hiding many of the underlying details of message aggregation, serialization, and synchronization.},
doi = {10.1145/3337821.3337912},
journal = {Proceedings of the 48th International Conference on Parallel Processing - ICPP 2019},
number = ,
volume = ,
place = {United States},
year = {2019},
month = {8}
}

Works referenced in this record:

DASH: A C++ PGAS Library for Distributed Data Structures and Parallel Algorithms
conference, December 2016

  • Fuerlinger, Karl; Fuchs, Tobias; Kowalewski, Roger
  • 2016 IEEE 18th International Conference on High Performance Computing and Communications; IEEE 14th International Conference on Smart City; IEEE 2nd International Conference on Data Science and Systems (HPCC/SmartCity/DSS)
  • DOI: 10.1109/HPCC-SmartCity-DSS.2016.0140

Titanium: a high-performance Java dialect
journal, September 1998