Performance Optimization for Relative-Error-Bounded Lossy Compression on Scientific Data

Journal Article · Mon Feb 10 00:00:00 EST 2020 · IEEE Transactions on Parallel and Distributed Systems

DOI:https://doi.org/10.1109/TPDS.2020.2972548· OSTI ID:1603491

Zou, Xiangyu ^[1]; Lu, Tao ^[2]; Xia, Wen ^[1]; Wang, Xuan ^[1]; Zhang, Weizhe ^[1]; Zhang, Haijun ^[1]; Di, Sheng ^[3]; Tao, Dingwen ^[4]; Cappello, Franck ^[3]

---

1. Harbin Inst. of Technology (China)
2. Marvell Technology Group, Santa Clara, CA (United States)
3. Argonne National Lab. (ANL), Argonne, IL (United States)
4. Univ. of Alabama, Tuscaloosa, AL (United States)

Scientific simulations in high-performance computing (HPC) environments generate vast volume of data, which may cause a severe I/O bottleneck at runtime and a huge burden on storage space for postanalysis. Unlike traditional data reduction schemes such as deduplication or lossless compression, not only can error-controlled lossy compression significantly reduce the data size but it also holds the promise to satisfy user demand on error control. Pointwise relative error bounds (i.e., compression errors depends on the data values) are widely used by many scientific applications with lossy compression since error control can adapt to the error bound in the dataset automatically. Pointwise relative-error-bounded compression is complicated and time consuming. In this article, we develop efficient precomputation-based mechanisms based on the SZ lossy compression framework. Our mechanisms can avoid costly logarithmic transformation and identify quantization factor values via a fast table lookup, greatly accelerating the relative-error-bounded compression with excellent compression ratios. In addition, we reduce traversing operations for Huffman decoding, significantly accelerating the decompression process in SZ. Experiments with eight well-known real-world scientific simulation datasets show that our solution can improve the compression and decompression rates (i.e., the speed) by about 40 and 80 p, respectively, in most of cases, making our designed lossy compression strategy the best-in-class solution in most cases.

• Research Organization: Argonne National Laboratory (ANL), Argonne, IL (United States)
• Sponsoring Organization: National Science Foundation (NSF); USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)
• Contributing Organization: National Key Research and Development Program of China
• Grant/Contract Number: AC02-06CH11357
• OSTI ID: 1603491
• Journal Information: IEEE Transactions on Parallel and Distributed Systems, Journal Name: IEEE Transactions on Parallel and Distributed Systems Journal Issue: 7 Vol. 31; ISSN 1045-9219
• Publisher: IEEE
• Country of Publication: United States
• Language: English

Related Subjects

96 KNOWLEDGE MANAGEMENT AND PRESERVATION
97 MATHEMATICS AND COMPUTING
Lossy compression
compression rate
high-performance computing
scientific data