OptZConfig: Efficient Parallel Optimization of Lossy Compression Configuration

Underwood, Robert; Calhoun, Jon C.; Di, Sheng; Apon, Amy; Cappello, Franck

doi:10.1109/tpds.2022.3154096

Title: OptZConfig: Efficient Parallel Optimization of Lossy Compression Configuration

Journal Article · 07 March 2022 · IEEE Transactions on Parallel and Distributed Systems

DOI: https://doi.org/10.1109/tpds.2022.3154096 · OSTI ID:2377211

^[1];

^[2];

^[1];

^[2]; Cappello, Franck ^[1]

Argonne National Laboratory (ANL), Argonne, IL (United States)
Clemson Univ., SC (United States)

Lossless compressors have very low compression ratios that do not meet the needs of today's large-scale scientific applications that produce vast volumes of data. Error-bounded lossy compression (EBLC) is considered a critical technique for the success of scientific research. Although EBLC allows users to set an error bound for the compression, users have been unable to specify the requirements on the compression quality, limiting practical use. Our contributions are: (1) We formulate the problem of configuring EBLC to preserve a user-defined metric as an optimization problem. This allows many classes of new metrics to be preserved, which improves over current practices. (2) We present a framework, OptZConfig, that can adapt to improvements in the search algorithm, compressor, and metrics with minimal changes, enabling future advancements in this area. (3) We demonstrate the advantages of our approach against the leading methods to configure compressors to preserve specific metrics. Here, our approach improves compression ratios against a specialized compressor by up to 3 x, has a 56x speedup over FRaZ, 1000x speedup over MGARD-QOI post tuning, and 110x speedup over systematic approaches which had not been bounded by compressors before.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Argonne National Laboratory (ANL), Argonne, IL (United States)

Sponsoring Organization:: National Science Foundation (NSF); USDOE National Nuclear Security Administration (NNSA); USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)

Grant/Contract Number:: AC02-06CH11357

OSTI ID:: 2377211

Journal Information:: IEEE Transactions on Parallel and Distributed Systems, Journal Name: IEEE Transactions on Parallel and Distributed Systems Journal Issue: 12 Vol. 33; ISSN 1045-9219

Publisher:: IEEECopyright Statement

Country of Publication:: United States

Language:: English

References (22)

The NEWUOA software for unconstrained optimization without derivatives Powell, M. J. D. Large-Scale Nonlinear Optimization https://doi.org/10.1007/0-387-30065-1_16	book	January 2006
Toward a Multi-method Approach: Lossy Data Compression for Climate Simulation Data Baker, Allison H.; Xu, Haiying; Hammerling, Dorit M. Lecture Notes in Computer Science https://doi.org/10.1007/978-3-319-67630-2_3	book	January 2017
Multilevel techniques for compression and reduction of scientific data—the univariate case Ainsworth, Mark; Tugluk, Ozan; Whitney, Ben Computing and Visualization in Science, Vol. 19, Issue 5-6 https://doi.org/10.1007/s00791-018-00303-9	journal	November 2018
Dark matter haloes: a multistream view Ramachandra, Nesar S.; Shandarin, Sergei F. Monthly Notices of the Royal Astronomical Society, Vol. 470, Issue 3 https://doi.org/10.1093/mnras/stx1474	journal	June 2017
An Efficient Transformation Scheme for Lossy Data Compression with Point-Wise Relative Error Bound Liang, Xin; Di, Sheng; Tao, Dingwen 2018 IEEE International Conference on Cluster Computing (CLUSTER) https://doi.org/10.1109/CLUSTER.2018.00036	conference	September 2018
Fixed-PSNR Lossy Compression for Scientific Data Tao, Dingwen; Di, Sheng; Liang, Xin 2018 IEEE International Conference on Cluster Computing (CLUSTER) https://doi.org/10.1109/CLUSTER.2018.00048	conference	September 2018
Fast Error-Bounded Lossy HPC Data Compression with SZ Di, Sheng; Cappello, Franck 2016 IEEE International Parallel and Distributed Processing Symposium (IPDPS) https://doi.org/10.1109/IPDPS.2016.11	conference	May 2016
Significantly Improving Lossy Compression for Scientific Data Sets Based on Multidimensional Prediction and Error-Controlled Quantization Tao, Dingwen; Di, Sheng; Chen, Zizhong 2017 IEEE International Parallel and Distributed Processing Symposium (IPDPS) https://doi.org/10.1109/IPDPS.2017.115	conference	May 2017
FRaZ: A Generic High-Fidelity Fixed-Ratio Lossy Compression Framework for Scientific Floating-point Data Underwood, Robert; Di, Sheng; Calhoun, Jon C. 2020 IEEE International Parallel and Distributed Processing Symposium (IPDPS) https://doi.org/10.1109/IPDPS47924.2020.00065	conference	May 2020
FPC: A High-Speed Compressor for Double-Precision Floating-Point Data Burtscher, Martin; Ratanaworabhan, Paruj IEEE Transactions on Computers, Vol. 58, Issue 1 https://doi.org/10.1109/TC.2008.131	journal	January 2009
Optimizing Lossy Compression Rate-Distortion from Automatic Online Selection between SZ and ZFP Tao, Dingwen; Di, Sheng; Liang, Xin IEEE Transactions on Parallel and Distributed Systems, Vol. 30, Issue 8 https://doi.org/10.1109/TPDS.2019.2894404	journal	August 2019
libEnsemble: A Library to Coordinate the Concurrent Evaluation of Dynamic Ensembles of Calculations Hudson, Stephen; Larson, Jeffrey; Navarro, John-Luke IEEE Transactions on Parallel and Distributed Systems, Vol. 33, Issue 4 https://doi.org/10.1109/TPDS.2021.3082815	journal	April 2022
Fixed-Rate Compressed Floating-Point Arrays Lindstrom, Peter IEEE Transactions on Visualization and Computer Graphics, Vol. 20, Issue 12 https://doi.org/10.1109/TVCG.2014.2346458	journal	December 2014
Evaluating image quality measures to assess the impact of lossy data compression applied to climate simulation data Baker, A. H.; Hammerling, D. M.; Turton, T. L. Computer Graphics Forum, Vol. 38, Issue 3 https://doi.org/10.1111/cgf.13707	journal	June 2019
Multilevel Techniques for Compression and Reduction of Scientific Data---The Multivariate Case Ainsworth, Mark; Tugluk, Ozan; Whitney, Ben SIAM Journal on Scientific Computing, Vol. 41, Issue 2 https://doi.org/10.1137/18M1166651	journal	January 2019
Multilevel Techniques for Compression and Reduction of Scientific Data-Quantitative Control of Accuracy in Derived Quantities Ainsworth, Mark; Tugluk, Ozan; Whitney, Ben SIAM Journal on Scientific Computing, Vol. 41, Issue 4 https://doi.org/10.1137/18M1208885	journal	January 2019
Stability Analysis of Inline ZFP Compression for Floating-Point Data in Iterative Methods Fox, Alyson; Diffenderfer, James; Hittinger, Jeffrey SIAM Journal on Scientific Computing, Vol. 42, Issue 5 https://doi.org/10.1137/19M126904X	journal	January 2020
Aequitas Ribic, Haris; Liu, Yu David Proceedings of the 2016 International Conference on Supercomputing https://doi.org/10.1145/2925426.2926260	conference	June 2016
HACC: extreme scaling and performance across diverse architectures Habib, Salman; Insley, Joe; Daniel, David Communications of the ACM, Vol. 60, Issue 1 https://doi.org/10.1145/3015569	journal	December 2016
Significantly Improving Lossy Compression for HPC Datasets with Second-Order Prediction and Parameter Optimization Zhao, Kai; Di, Sheng; Liang, Xin HPDC '20: The 29th International Symposium on High-Performance Parallel and Distributed Computing, Proceedings of the 29th International Symposium on High-Performance Parallel and Distributed Computing https://doi.org/10.1145/3369583.3392688	conference	June 2020
Use cases of lossy compression for floating-point data in scientific data sets Cappello, Franck; Di, Sheng; Li, Sihuan The International Journal of High Performance Computing Applications, Vol. 33, Issue 6 https://doi.org/10.1177/1094342019853336	journal	May 2019
Composable Multi-Threading for Python Libraries Malakhov, Anton Proceedings of the Python in Science Conference https://doi.org/10.25080/Majora-629e541a-002	conference	January 2016

Similar Records

Toward Quantity-of-Interest Preserving Lossy Compression for Scientific Data

Conference · 2022 · OSTI ID:2427976

Jiao, Pu; Di, Sheng; Guo, Hanqi; +5 more

Black-box statistical prediction of lossy compression ratios for scientific data

Journal Article · 2023 · International Journal of High Performance Computing Applications · OSTI ID:1985174

Underwood, Robert; Bessac, Julie; Krasowska, David; +3 more

Optimizing Error-Bounded Lossy Compression for Scientific Data With Diverse Constraints

Journal Article · 2022 · IEEE Transactions on Parallel and Distributed Systems · OSTI ID:1908133

Liu, Yuanjian; Di, Sheng; Zhao, Kai; +6 more

Related Subjects

97 MATHEMATICS AND COMPUTING
LibPressio
PSNR
compressors
distortion
error bounded lossy compression
measurement
meteorology
non-linear optimization
optimization
parallel computing
quantization (signal)

Title: OptZConfig: Efficient Parallel Optimization of Lossy Compression Configuration

Citation Formats

References (22)

Similar Records

Related Subjects