DART: distributed adaptive radix tree for efficient affix-based keyword search on HPC systems

Zhang, Wei; Tang, Houjun; Byna, Suren; Chen, Yong

doi:10.1145/3243176.3243207

DART: distributed adaptive radix tree for efficient affix-based keyword search on HPC systems

Conference · 01 January 2018

DOI:https://doi.org/10.1145/3243176.3243207· OSTI ID:1602812

Zhang, Wei ^[1]; Tang, Houjun ^[2]; Byna, Suren ^[2]; Chen, Yong ^[1]

Texas Tech University
Lawrence Berkeley National Laboratory

© 2018 Association for Computing Machinery. Affix-based search is a fundamental functionality for storage systems. It allows users to find desired datasets, where attributes of a dataset match an affix. While building inverted index to facilitate efficient affix based keyword search is a common practice for standalone databases and for desktop file systems, building local indexes or adopting indexing techniques used in a standalone data store is insufficient for highperformance computing (HPC) systems due to the massive amount of data and distributed nature of the storage devices within a system. In this paper, we propose Distributed Adaptive Radix Tree (DART), to address the challenge of distributed affix-based keyword search on HPC systems. This trie-based approach is scalable in achieving efficient affix-based search and alleviating imbalanced keyword distribution and excessive requests on keywords at scale. Our evaluation at different scales shows that, comparing with the "full string hashing" use case of the most popular distributed indexing technique - Distributed Hash Table (DHT), DART achieves up to 55× better throughput with prefix search and with suffix search, while achieving comparable throughput with exact and infix searches. Also, comparing to the "initial hashing" use case of DHT, DART maintains a balanced keyword distribution on distributed nodes and alleviates excessive query workload against popular keywords.

View Conference

Research Organization:: Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)

DOE Contract Number:: AC02-05CH11231

OSTI ID:: 1602812

Resource Relation:: Conference: Proceedings of the 27th International Conference on Parallel Architectures and Compilation Techniques - PACT '18, November 1–4, 2018, Limassol, Cyprus

Country of Publication:: United States

Language:: English

References (24)

Cost effective speculation with the omnipredictor No authors listed PACT '18: International conference on Parallel Architectures and Compilation Techniques, Proceedings of the 27th International Conference on Parallel Architectures and Compilation Techniques https://doi.org/10.1145/3243176.3243208		November 2018
Making Peer-to-Peer Keyword Searching Feasible Using Multi-level Partitioning No authors listed Lecture Notes in Computer Science https://doi.org/10.1007/978-3-540-30183-7_15		January 2005
Ubiquitous B-Tree No authors listed ACM Computing Surveys https://doi.org/10.1145/356770.356776		June 1979
Toward Scalable and Asynchronous Object-Centric Data Management for HPC No authors listed 2018 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID) https://doi.org/10.1109/CCGRID.2018.00026		May 2018
The power of two choices in randomized load balancing No authors listed IEEE Transactions on Parallel and Distributed Systems https://doi.org/10.1109/71.963420		January 2001
Wikipedia workload analysis for decentralized hosting No authors listed Computer Networks https://doi.org/10.1016/j.comnet.2009.02.019		July 2009
DeltaFS: exascale file systems scale better without dedicated servers No authors listed Proceedings of the 10th Parallel Data Storage Workshop on - PDSW '15 https://doi.org/10.1145/2834976.2834984		January 2015
The adaptive radix tree: ARTful indexing for main-memory databases No authors listed 2013 29th IEEE International Conference on Data Engineering (ICDE 2013), 2013 IEEE 29th International Conference on Data Engineering (ICDE) https://doi.org/10.1109/ICDE.2013.6544812		April 2013
Implicit sampling combined with reduced order modeling for the inversion of vadose zone hydrological data No authors listed Computers & Geosciences https://doi.org/10.1016/j.cageo.2017.04.001		November 2017
Brief announcement: prefix hash tree No authors listed Proceedings of the twenty-third annual ACM symposium on Principles of distributed computing - PODC '04 https://doi.org/10.1145/1011767.1011823		January 2004
Atributed consistent hashing for heterogeneous storage systems No authors listed PACT '18: International conference on Parallel Architectures and Compilation Techniques, Proceedings of the 27th International Conference on Parallel Architectures and Compilation Techniques https://doi.org/10.1145/3243176.3243202		November 2018
IndexFS: Scaling File System Metadata Performance with Stateless Caching and Bulk Insertion No authors listed SC14: International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1109/SC.2014.25		November 2014
Peer-to-peer systems for prefix search No authors listed Proceedings of the twenty-second annual symposium on Principles of distributed computing - PODC '03 https://doi.org/10.1145/872035.872053		January 2003
Search for Sources of High-Energy Neutrons with four Years of data from the Icetop Detector No authors listed The Astrophysical Journal https://doi.org/10.3847/0004-637X/830/2/129		October 2016
Accurate force field for molybdenum by machine learning large materials data No authors listed Physical Review Materials https://doi.org/10.1103/PhysRevMaterials.1.043603		September 2017
Complex Queries in DHT-based Peer-to-Peer Networks No authors listed Peer-to-Peer Systems https://doi.org/10.1007/3-540-45748-8_23		January 2002
Fast Pattern Matching in Strings No authors listed SIAM Journal on Computing https://doi.org/10.1137/0206024		June 1977
Searching for millions of objects in the BOSS spectroscopic survey data with H5Boss No authors listed 2017 New York Scientific Data Summit (NYSDS) https://doi.org/10.1109/NYSDS.2017.8085044		August 2017
Mining Materials Design Rules from Data: The Example of Polymer Dielectrics No authors listed Chemistry of Materials https://doi.org/10.1021/acs.chemmater.7b02027		October 2017
IMG/VR: a database of cultured and uncultured DNA Viruses and retroviruses No authors listed Nucleic Acids Research https://doi.org/10.1093/nar/gkw1030		October 2016
Mercury: Enabling remote procedure call for high-performance computing No authors listed 2013 IEEE International Conference on Cluster Computing (CLUSTER) https://doi.org/10.1109/CLUSTER.2013.6702617		September 2013
SoMeta: Scalable Object-Centric Metadata Management for High Performance Computing No authors listed 2017 IEEE International Conference on Cluster Computing (CLUSTER) https://doi.org/10.1109/CLUSTER.2017.53		September 2017
Toward Efficient and Flexible Metadata Indexing of Big Data Systems No authors listed IEEE Transactions on Big Data https://doi.org/10.1109/TBDATA.2016.2640301		March 2017
FusionFS: Toward supporting data-intensive scientific applications on extreme-scale high-performance computing systems No authors listed 2014 IEEE International Conference on Big Data (Big Data) https://doi.org/10.1109/BigData.2014.7004214		October 2014

Similar Records

Performance of Point and Range Queries for In-memory Databases using Radix Trees on GPUs

Conference · 2016 · OSTI ID:1338492

Alam, Maksudul; Yoginath, Srikanth B; Perumalla, Kalyan S

Toward higher-radix switches with co-packaged optics for improved network locality in data center and HPC networks [Invited]

Journal Article · 2022 · Journal of Optical Communications and Networking · OSTI ID:1855243

Maniotis, Pavlos; Schares, Laurent; Kuchta, Daniel M.; +1 more

DART: distributed adaptive radix tree for efficient affix-based keyword search on HPC systems

Citation Formats

References (24)

Similar Records

Related Subjects