Scaling SQL to the Supercomputer for Interactive Analysis of Simulation Data

Glaser, Jens; Aramburu, Felipe; Malpica, William; Hernandez Arreguin, Benjamin; Baker, Matthew; Aramburu, Rodrigo

doi:10.1007/978-3-030-96498-6_19

Scaling SQL to the Supercomputer for Interactive Analysis of Simulation Data

Conference · 01 March 2022

DOI:https://doi.org/10.1007/978-3-030-96498-6_19· OSTI ID:1856710

^[1]; Aramburu, Felipe ^[2]; Malpica, William ^[2]; ^[1]; ^[1]; Aramburu, Rodrigo ^[2]

ORNL
Voltron Data

AI and simulation workloads consume and generate large amounts of data that need to be searched, transformed and merged with other data. With the goal of treating data as a first-class citizen inside a traditionally compute-centric HPC environment, we explore how the use of accelerators and high-speed interconnects can speed up tasks which otherwise constitute bottlenecks in computational discovery workflows. BlazingSQL is SQL engine that runs natively on NVIDIA GPUs and supports internode communication for fast analytics on terabyte-scale tabular data sets. We show how a fast interconnect improves query performance if leveraged through the Unified Communication X (UCX) middleware. We envision that future computing platforms will integrate accelerated database query capabilities for immediate and interactive analysis of large simulation data.

View Conference

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)

DOE Contract Number:: AC05-00OR22725

OSTI ID:: 1856710

Resource Relation:: Journal Volume: 1512; Conference: Smoky Mountains Computational Sciences and Engineering Conference (SMC) - Kingsport, Tennessee, United States of America - 10/18/2021 12:00:00 PM-10/20/2021 12:00:00 PM

Country of Publication:: United States

Language:: English

References (20)

Accelerating SQL database operations on a GPU with CUDA Bakkum, Peter; Skadron, Kevin Proceedings of the 3rd Workshop on General-Purpose Computation on Graphics Processing Units https://doi.org/10.1145/1735688.1735706	conference	March 2010
Why it is time for a HyPE Breß, Sebastian; Saake, Gunter Proceedings of the VLDB Endowment, Vol. 6, Issue 12 https://doi.org/10.14778/2536274.2536325	journal	August 2013
Efficient co-processor utilization in database query processing Breß, Sebastian; Beier, Felix; Rauhe, Hannes Information Systems, Vol. 38, Issue 8 https://doi.org/10.1016/j.is.2013.05.004	journal	November 2013
Introducing OpenSHMEM: SHMEM for the PGAS community Chapman, Barbara; Curtis, Tony; Pophale, Swaroop Proceedings of the Fourth Conference on Partitioned Global Address Space Programming Model - PGAS '10 https://doi.org/10.1145/2020373.2020375	conference	January 2010
MapReduce: simplified data processing on large clusters Dean, Jeffrey; Ghemawat, Sanjay; Mehta, Brijesh Communications of the ACM, Vol. 51, Issue 1 https://doi.org/10.1145/1327452.1327492	journal	January 2008
Parallel database systems DeWitt, David; Gray, Jim Communications of the ACM, Vol. 35, Issue 6 https://doi.org/10.1145/129888.129894	journal	June 1992
Gpuqp Fang, Rui; He, Bingsheng; Lu, Mian Proceedings of the 2007 ACM SIGMOD international conference on Management of data https://doi.org/10.1145/1247480.1247606	conference	June 2007
Database compression on graphics processors Fang, Wenbin; He, Bingsheng; Luo, Qiong Proceedings of the VLDB Endowment, Vol. 3, Issue 1-2 https://doi.org/10.14778/1920841.1920927	journal	September 2010
High-throughput virtual laboratory for drug discovery using massive datasets Glaser, Jens; Vermaas, Josh V.; Rogers, David M. The International Journal of High Performance Computing Applications, Vol. 35, Issue 5 https://doi.org/10.1177/10943420211001565	journal	March 2021
Fast computation of database operations using graphics processors Govindaraju, Naga K.; Lloyd, Brandon; Wang, Wei Proceedings of the 2004 ACM SIGMOD international conference on Management of data https://doi.org/10.1145/1007568.1007594	conference	June 2004
Relational query coprocessing on graphics processors He, Bingsheng; Lu, Mian; Yang, Ke ACM Transactions on Database Systems, Vol. 34, Issue 4 https://doi.org/10.1145/1620585.1620588	journal	December 2009
Performance Analysis of Big Data ETL Process over CPU-GPU Heterogeneous Architectures Lee, Suyeon; Park, Sungyong 2021 IEEE 37th International Conference on Data Engineering Workshops (ICDEW) https://doi.org/10.1109/ICDEW53142.2021.00015	conference	April 2021
High-Performance Design of Hadoop RPC with RDMA over InfiniBand Lu, Xiaoyi; Islam, Nusrat S.; Wasi-Ur-Rahman, Md. 2013 42nd International Conference on Parallel Processing https://doi.org/10.1109/ICPP.2013.78	conference	October 2013
SQLPhi Olsen, Stuart; Romoser, Brian; Zong, Ziliang Proceedings of the 2014 International Conference on Big Data Science and Computing https://doi.org/10.1145/2640087.2644172	conference	August 2014
Transitioning from File-Based HPC Workflows to Streaming Data Pipelines with openPMD and ADIOS2 Poeschel, Franz; E., Juncheng; Godoy, William F. Driving Scientific and Engineering Discoveries Through the Integration of Experiment, Big Data, and Modeling and Simulation https://doi.org/10.1007/978-3-030-96498-6_6	book	January 2022
UCX: An Open Source Framework for HPC Network APIs and Beyond Shamis, Pavel; Venkata, Manjunath Gorentla; Lopez, M. Graham 2015 IEEE 23rd Annual Symposium on High-Performance Interconnects (HOTI) https://doi.org/10.1109/HOTI.2015.13	conference	August 2015
Accelerating relational database operations using both CPU and GPU co-processor Shehab, Esraa; Algergawy, Alsayed; Sarhan, Amany Computers & Electrical Engineering, Vol. 57 https://doi.org/10.1016/j.compeleceng.2016.12.014	journal	January 2017
pandas-dev/pandas: Pandas No authors listed https://doi.org/10.5281/zenodo.3509134	software	January 2024
SMILES, a chemical language and information system. 1. Introduction to methodology and encoding rules Weininger, David Journal of Chemical Information and Modeling, Vol. 28, Issue 1 https://doi.org/10.1021/ci00057a005	journal	February 1988
Ibex: an intelligent storage engine with support for advanced SQL offloading Woods, Louis; István, Zsolt; Alonso, Gustavo Proceedings of the VLDB Endowment, Vol. 7, Issue 11 https://doi.org/10.14778/2732967.2732972	journal	July 2014

Similar Records

Phase 1 Final Report

Technical Report · 2024 · OSTI ID:2439217

Dai, Donglai

A Quantitative Study of Deep Learning Training on Heterogeneous Supercomputers

Conference · 2019 · OSTI ID:1569375

Han, Jingoo; Xu, Luna; Rafique, Mustafa; +2 more

Evaluating Modern GPU Interconnect: PCIe, NVLink, NV-SLI, NVSwitch and GPUDirect

Journal Article · 2020 · IEEE Transactions on Parallel and Distributed Systems · OSTI ID:1598812

Li, Ang; Song, Shuaiwen; Chen, Jieyang; +4 more

Scaling SQL to the Supercomputer for Interactive Analysis of Simulation Data

Citation Formats

References (20)

Similar Records

Related Subjects