Exploring the feasibility of lossy compression for PDE simulations

Calhoun, Jon; Cappello, Franck; Olson, Luke N.; Snir, Marc; Gropp, William D.

doi:10.1177/1094342018762036

Exploring the feasibility of lossy compression for PDE simulations

Journal Article · Mon Mar 12 00:00:00 EDT 2018 · International Journal of High Performance Computing Applications

DOI:https://doi.org/10.1177/1094342018762036· OSTI ID:1425688

Calhoun, Jon ^[1]; Cappello, Franck ^[2]; Olson, Luke N. ^[3]; Snir, Marc ^[3]; Gropp, William D. ^[3]

Holcombe Department of Electrical and Computer Engineering, Clemson University, Clemson, SC, USA
Mathematics and Computer Science Division, Argonne National Laboratory, Lemont, IL, USA
Department of Computer Science, University of Illinois at Urbana-Champaign, Urbana, IL, USA

Checkpoint restart plays an important role in high-performance computing (HPC) applications, allowing simulation runtime to extend beyond a single job allocation and facilitating recovery from hardware failure. Yet, as machines grow in size and in complexity, traditional approaches to checkpoint restart are becoming prohibitive. Current methods store a subset of the application’s state and exploit the memory hierarchy in the machine. However, as the energy cost of data movement continues to dominate, further reductions in checkpoint size are needed. Lossy compression, which can significantly reduce checkpoint sizes, offers a potential to reduce computational cost in checkpoint restart. This article investigates the use of numerical properties of partial differential equation (PDE) simulations, such as bounds on the truncation error, to evaluate the feasibility of using lossy compression in checkpointing PDE simulations. Restart from a checkpoint with lossy compression is considered for a fail-stop error in two time-dependent HPC application codes: PlasComCM and Nek5000. Results show that error in application variables due to a restart from a lossy compressed checkpoint can be masked by the numerical error in the discretization, leading to increased efficiency in checkpoint restart without influencing overall accuracy in the simulation.

View Journal Article

Research Organization:: Argonne National Lab. (ANL), Argonne, IL (United States)

Sponsoring Organization:: Air Force Research Laboratory (AFRL), Air Force Office of Scientific Research (AFOSR); National Science Foundation (NSF); USDOE; USDOE National Nuclear Security Administration (NNSA)

Grant/Contract Number:: AC02-06CH11357; NA0002374

OSTI ID:: 1425688

Alternate ID(s):: OSTI ID: 1510066

Journal Information:: International Journal of High Performance Computing Applications, Journal Name: International Journal of High Performance Computing Applications Journal Issue: 2 Vol. 33; ISSN 1094-3420

Publisher:: SAGE PublicationsCopyright Statement

Country of Publication:: United States

Language:: English

References (17)

Exascale Computing Technology Challenges Shalf, John; Dosanjh, Sudip; Morrison, John Lecture Notes in Computer Science https://doi.org/10.1007/978-3-642-19328-6_1	book	January 2010
Collective I/O Tuning Using Analytical and Machine Learning Models Isaila, Florin; Balaprakash, Prasanna; Wild, Stefan M. 2015 IEEE International Conference on Cluster Computing (CLUSTER) https://doi.org/10.1109/CLUSTER.2015.29	conference	September 2015
High Throughput Compression of Double-Precision Floating-Point Data Burtscher, Martin; Ratanaworabhan, Paruj 2007 Data Compression Conference (DCC'07) https://doi.org/10.1109/DCC.2007.44	conference	March 2007
On the Viability of Compression for Reducing the Overheads of Checkpoint/Restart-Based Fault Tolerance Ibtesham, Dewan; Arnold, Dorian; Bridges, Patrick G. 2012 41st International Conference on Parallel Processing (ICPP) https://doi.org/10.1109/ICPP.2012.45	conference	September 2012
Exploration of Lossy Compression for Application-Level Checkpoint/Restart Sasaki, Naoto; Sato, Kento; Endo, Toshio 2015 IEEE International Parallel and Distributed Processing Symposium (IPDPS) https://doi.org/10.1109/IPDPS.2015.67	conference	May 2015
Fast Error-Bounded Lossy HPC Data Compression with SZ Di, Sheng; Cappello, Franck 2016 IEEE International Parallel and Distributed Processing Symposium (IPDPS) https://doi.org/10.1109/IPDPS.2016.11	conference	May 2016
Significantly Improving Lossy Compression for Scientific Data Sets Based on Multidimensional Prediction and Error-Controlled Quantization Tao, Dingwen; Di, Sheng; Chen, Zizhong 2017 IEEE International Parallel and Distributed Processing Symposium (IPDPS) https://doi.org/10.1109/IPDPS.2017.115	conference	May 2017
On the role of burst buffers in leadership-class storage systems Liu, Ning; Cope, Jason; Carns, Philip 2012 IEEE 28th Symposium on Mass Storage Systems and Technologies (MSST) https://doi.org/10.1109/MSST.2012.6232369	conference	April 2012
Design, Modeling, and Evaluation of a Scalable Multi-level Checkpointing System Moody, Adam; Bronevetsky, Greg; Mohror, Kathryn 2010 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1109/SC.2010.18	conference	November 2010
NUMARCK: Machine Learning Algorithm for Resiliency and Checkpointing Chen, Zhengzhang; Son, Seung Woo; Hendrix, William SC14: International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1109/SC.2014.65	conference	November 2014
Fast and Efficient Compression of Floating-Point Data Lindstrom, Peter; Isenburg, Martin IEEE Transactions on Visualization and Computer Graphics, Vol. 12, Issue 5 https://doi.org/10.1109/TVCG.2006.143	journal	September 2006
Fixed-Rate Compressed Floating-Point Arrays Lindstrom, Peter IEEE Transactions on Visualization and Computer Graphics, Vol. 20, Issue 12 https://doi.org/10.1109/TVCG.2014.2346458	journal	December 2014
FTI: high performance fault tolerance interface for hybrid systems Bautista-Gomez, Leonardo; Tsuboi, Seiji; Komatitsch, Dimitri Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '11 https://doi.org/10.1145/2063384.2063427	conference	January 2011
Assessing the effects of data compression in simulations using physically motivated metrics Laney, Daniel; Langer, Steven; Weber, Christopher Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '13 https://doi.org/10.1145/2503210.2503283	conference	January 2013
A methodology for evaluating the impact of data compression on climate simulation data Baker, Allison H.; Xu, Haiying; Dennis, John M. Proceedings of the 23rd international symposium on High-performance parallel and distributed computing - HPDC '14 https://doi.org/10.1145/2600212.2600217	conference	January 2014
A Multiplatform Study of I/O Behavior on Petascale Supercomputers Luu, Huong; Winslett, Marianne; Gropp, William Proceedings of the 24th International Symposium on High-Performance Parallel and Distributed Computing - HPDC '15 https://doi.org/10.1145/2749246.2749269	conference	January 2015
Toward Exascale Resilience Cappello, Franck; Geist, Al; Gropp, Bill The International Journal of High Performance Computing Applications, Vol. 23, Issue 4 https://doi.org/10.1177/1094342009347767	journal	September 2009

Similar Records

Performance Optimization for Relative-Error-Bounded Lossy Compression on Scientific Data

Journal Article · Sun Feb 09 19:00:00 EST 2020 · IEEE Transactions on Parallel and Distributed Systems · OSTI ID:1603491

Accelerating Relative-error Bounded Lossy Compression for HPC datasets with Precomputation-Based Mechanisms

Conference · Mon May 20 00:00:00 EDT 2019 · OSTI ID:1515554

Related Subjects

97 MATHEMATICS AND COMPUTING
checkpoint restart
compression
error propagation
error tolerance selection
exascale
fault tolerance
lossy compression

Exploring the feasibility of lossy compression for PDE simulations

Citation Formats

References (17)

Similar Records

Related Subjects