Accelerating Relative-error Bounded Lossy Compression for HPC datasets with Precomputation-Based Mechanisms

Conference · Mon May 20 04:00:00 EDT 2019

OSTI ID:1515554

Zou, Xiangyu; Lu, Tao; Xia, Wen; Wang, Xuan; Zhang, Weizhe; Di, Sheng; Tao, Dingwen; Cappello, Franck

Scientific simulations in high-performance computing (HPC) environments are producing vast volume of data, which may cause a severe I/O bottleneck at runtime and a huge burden on storage space for post-analysis. Unlike the traditional data reduction schemes (such as deduplication or lossless compression), not only can error-controlled lossy compression significantly reduce the data size but it can also hold the promise to satisfy user demand on error control. Point-wise relative error bounds (i.e., compression errors depends on the data values) are widely used by many scientific applications in the lossy compression, since error control can adapt to the precision in the dataset automatically. Point-wise relative error bounded compression is complicated and time consuming. In this work, we develop efficient precomputation-based mechanisms in the SZ lossy compression framework. Our mechanisms can avoid costly logarithmic transformation and identify quantization factor values via a fast table lookup, greatly accelerating the relative-error bounded compression with excellent compression ratios. In addition, our mechanisms also help reduce traversing operations for Huffman decoding, and thus significantly accelerate the decompression process in SZ. Experiments with four well-known real-world scientific simulation datasets show that our solution can improve the compression rate by about 30% and decompression rate by about 70% in most of cases, making our designed lossy compression strategy the best choice in class in most cases.

• Research Organization: Argonne National Laboratory (ANL)
• Sponsoring Organization: USDOE National Nuclear Security Administration (NNSA); USDOE Office of Science
• DOE Contract Number: AC02-06CH11357
• OSTI ID: 1515554
• Country of Publication: United States
• Language: English

Related Subjects

Lossy compression
compression rate
high-performance computing
scientific data