ALACRITY: Analytics-Driven Lossless Data Compression for Rapid In-Situ Indexing, Storing, and Querying. In: Transactions on Large-Scale Data- and Knowledge-Centered Systems X. Lecture Notes in Computer Science, vol 8220

Jenkins, John; Arkatkar, Isha; Lakshminarasimhan, Sriram; Boyuka II, David A.; Schendel, Eric R.; Shah, Neil; Ethier, Stephane; Chang, Choong-Seock; Chen, Jackie; Kolla, Hemanth; Klasky, Scott; Ross, Robert; Samatova, Nagiza F.

doi:10.1007/978-3-642-41221-9_4

Title: ALACRITY: Analytics-Driven Lossless Data Compression for Rapid In-Situ Indexing, Storing, and Querying. In: Transactions on Large-Scale Data- and Knowledge-Centered Systems X. Lecture Notes in Computer Science, vol 8220

Book · Tue Jan 01 00:00:00 EST 2013

DOI:https://doi.org/10.1007/978-3-642-41221-9_4· OSTI ID:1567532

Jenkins, John ^[1]; Arkatkar, Isha ^[1]; Lakshminarasimhan, Sriram ^[1]; Boyuka II, David A. ^[1]; Schendel, Eric R. ^[1]; Shah, Neil ^[1]; Ethier, Stephane ^[2]; Chang, Choong-Seock ^[2]; Chen, Jackie ^[3]; Kolla, Hemanth ^[3]; Klasky, Scott ^[4]; Ross, Robert ^[5]; Samatova, Nagiza F. ^[1]

North Carolina State Univ., Raleigh, NC (United States); Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Princeton Plasma Physics Lab. (PPPL), Princeton, NJ (United States)
Sandia National Lab. (SNL-CA), Livermore, CA (United States)
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Argonne National Lab. (ANL), Argonne, IL (United States)

High-performance computing architectures face nontrivial data processing challenges, as computational and I/O components further diverge in performance trajectories. For scientific data analysis in particular, methods based on generating heavyweight access acceleration structures, e.g. indexes, are becoming less feasible for ever-increasing dataset sizes. We present ALACRITY, demonstrating the effectiveness of a fused data and index encoding of scientific, floating-point data in generating lightweight data structures amenable to common types of queries used in scientific data analysis. We exploit the representation of floating-point values by extracting significant bytes, using the resulting unique values to bin the remaining data along fixed-precision boundaries. To optimize query processing, we use an inverted index, mapping each generated bin to a list of records contained within, allowing us to optimize query processing with attribute range constraints. Overall, the storage footprint for both index and data is shown to be below numerous configurations of bitmap indexing, while matching or outperforming query performance.

OSTI does not have a digital full text copy available. For more information, please see document availability, search WorldCat, or search Google Scholar.

Cite

Export

Save

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF)

Sponsoring Organization:: USDOE Office of Science (SC)

OSTI ID:: 1567532

Country of Publication:: United States

Language:: English

References (20)

Integrating compression and execution in column-oriented database systems Abadi, Daniel; Madden, Samuel; Ferreira, Miguel Proceedings of the 2006 ACM SIGMOD international conference on Management of data - SIGMOD '06 https://doi.org/10.1145/1142473.1142548	conference	January 2006
FLASH: An Adaptive Mesh Hydrodynamics Code for Modeling Astrophysical Thermonuclear Flashes Fryxell, B.; Olson, K.; Ricker, P. The Astrophysical Journal Supplement Series, Vol. 131, Issue 1 https://doi.org/10.1086/317361	journal	November 2000
High Throughput Compression of Double-Precision Floating-Point Data Burtscher, Martin; Ratanaworabhan, Paruj 2007 Data Compression Conference (DCC'07) https://doi.org/10.1109/DCC.2007.44	conference	March 2007
FPC: A High-Speed Compressor for Double-Precision Floating-Point Data Burtscher, Martin; Ratanaworabhan, Paruj IEEE Transactions on Computers, Vol. 58, Issue 1 https://doi.org/10.1109/TC.2008.131	journal	January 2009
Terascale direct numerical simulations of turbulent combustion using S3D Chen, J. H.; Choudhary, A.; de Supinski, B. Computational Science & Discovery, Vol. 2, Issue 1 https://doi.org/10.1088/1749-4699/2/1/015001	journal	January 2009
Ubiquitous B-Tree Comer, Douglas ACM Computing Surveys, Vol. 11, Issue 2 https://doi.org/10.1145/356770.356776	journal	June 1979
Out-of-core compression and decompression of large n-dimensional scalar fields Ibarria, Lawrence; Lindstrom, Peter; Rossignac, Jarek Computer Graphics Forum, Vol. 22, Issue 3 https://doi.org/10.1111/1467-8659.00681	journal	September 2003
Lossless compression of predicted floating-point geometry Isenburg, Martin; Lindstrom, Peter; Snoeyink, Jack Computer-Aided Design, Vol. 37, Issue 8 https://doi.org/10.1016/j.cad.2004.09.015	journal	July 2005
Analytics-Driven Lossless Data Compression for Rapid In-situ Indexing, Storing, and Querying Jenkins, John; Arkatkar, Isha; Lakshminarasimhan, Sriram Lecture Notes in Computer Science https://doi.org/10.1007/978-3-642-32597-7_2	book	January 2012
Full-f gyrokinetic particle simulation of centrally heated global ITG turbulence from magnetic axis to edge pedestal top in a realistic tokamak geometry Ku, S.; Chang, C. S.; Diamond, P. H. Nuclear Fusion, Vol. 49, Issue 11 https://doi.org/10.1088/0029-5515/49/11/115021	journal	September 2009
Fast and Efficient Compression of Floating-Point Data Lindstrom, Peter; Isenburg, Martin IEEE Transactions on Visualization and Computer Graphics, Vol. 12, Issue 5 https://doi.org/10.1109/TVCG.2006.143	journal	September 2006
ISOBAR Preconditioner for Effective and High-throughput Lossless Data Compression Schendel, Eric R.; Jin, Ye; Shah, Neil 2012 IEEE International Conference on Data Engineering (ICDE 2012), 2012 IEEE 28th International Conference on Data Engineering https://doi.org/10.1109/ICDE.2012.114	conference	April 2012
Multi-resolution bitmap indexes for scientific data Sinha, Rishi Rakesh; Winslett, Marianne ACM Transactions on Database Systems, Vol. 32, Issue 3 https://doi.org/10.1145/1272743.1272746	journal	August 2007
Gyro-kinetic simulation of global turbulent transport properties in tokamak experiments Wang, W. X.; Lin, Z.; Tang, W. M. Physics of Plasmas, Vol. 13, Issue 9 https://doi.org/10.1063/1.2338775	journal	September 2006
The implementation and performance of compressed databases Westmann, Till; Kossmann, Donald; Helmer, Sven ACM SIGMOD Record, Vol. 29, Issue 3 https://doi.org/10.1145/362084.362137	journal	September 2000
FastBit: an efficient indexing technology for accelerating data-intensive science Wu, Kesheng Journal of Physics: Conference Series, Vol. 16 https://doi.org/10.1088/1742-6596/16/1/077	journal	January 2005
On the Performance of Bitmap Indices for High Cardinality Attributes Wu, Kesheng; Otoo, Ekow; Shoshani, Arie Proceedings 2004 VLDB Conference https://doi.org/10.1016/B978-012088469-8.50006-1	book	January 2004
Optimizing bitmap indices with efficient compression Wu, Kesheng; Otoo, Ekow J.; Shoshani, Arie ACM Transactions on Database Systems, Vol. 31, Issue 1 https://doi.org/10.1145/1132863.1132864	journal	March 2006
Inverted index compression and query processing with optimized document ordering Yan, Hao; Ding, Shuai; Suel, Torsten Proceedings of the 18th international conference on World wide web - WWW '09 https://doi.org/10.1145/1526709.1526764	conference	January 2009
Inverted files for text search engines Zobel, Justin; Moffat, Alistair ACM Computing Surveys, Vol. 38, Issue 2 https://doi.org/10.1145/1132956.1132959	journal	July 2006

Similar Records

Analytics-Driven Lossless Data Compression for Rapid In-situ Indexing, Storing, and Querying

Conference · Tue Jan 01 00:00:00 EST 2013 · OSTI ID:1567532

Jenkins, John; Arkatkar, Isha; Lakshminarasimhan, Sriram; +9 more

Parallel membership queries on very large scientific data sets using bitmap indexes

Journal Article · Sat Aug 10 00:00:00 EDT 2019 · Concurrency and Computation. Practice and Experience · OSTI ID:1567532

Yildiz, Beytullah; Wu, Kesheng; Byna, Suren; +1 more

Minimizing I/O Costs of Multi-Dimensional Queries with BitmapIndices

Conference · Thu Mar 30 00:00:00 EST 2006 · OSTI ID:1567532

Rotem, Doron; Stockinger, Kurt; Wu, Kesheng

Related Subjects

Compression Ratio
Query Processing
Range Query
Inverted Index
Lossless Compression

Title: ALACRITY: Analytics-Driven Lossless Data Compression for Rapid In-Situ Indexing, Storing, and Querying. In: Transactions on Large-Scale Data- and Knowledge-Centered Systems X. Lecture Notes in Computer Science, vol 8220

Citation Formats

References (20)

Similar Records

Related Subjects