Exploring the feasibility of lossy compression for PDE simulations

Calhoun, Jon; Cappello, Franck; Olson, Luke N.; Snir, Marc; Gropp, William D.

doi:10.1177/1094342018762036

Title: Exploring the feasibility of lossy compression for PDE simulations

Abstract

Checkpoint restart plays an important role in high-performance computing (HPC) applications, allowing simulation runtime to extend beyond a single job allocation and facilitating recovery from hardware failure. Yet, as machines grow in size and in complexity, traditional approaches to checkpoint restart are becoming prohibitive. Current methods store a subset of the application's state and exploit the memory hierarchy in the machine. However, as the energy cost of data movement continues to dominate, further reductions in checkpoint size are needed. Lossy compression, which can significantly reduce checkpoint sizes, offers a potential to reduce computational cost in checkpoint restart. This article investigates the use of numerical properties of partial differential equation (PDE) simulations, such as bounds on the truncation error, to evaluate the feasibility of using lossy compression in checkpointing PDE simulations. Restart from a checkpoint with lossy compression is considered for a fail-stop error in two time-dependent HPC application codes: PlasComCM and Nek5000. Here, the results show that error in application variables due to a restart from a lossy compressed checkpoint can be masked by the numerical error in the discretization, leading to increased efficiency in checkpoint restart without influencing overall accuracy in the simulation.

Authors:

Calhoun, Jon ^[1]; Cappello, Franck ^[2]; Olson, Luke N. ^[3]; Snir, Marc ^[3]; Gropp, William D. ^[3]

Holcombe Department of Electrical and Computer Engineering, Clemson University, Clemson, SC, USA
Mathematics and Computer Science Division, Argonne National Laboratory, Lemont, IL, USA
Department of Computer Science, University of Illinois at Urbana-Champaign, Urbana, IL, USA

Publication Date:: Mon Mar 12 00:00:00 EDT 2018

Research Org.:: Argonne National Laboratory (ANL), Argonne, IL (United States)

Sponsoring Org.:: Air Force Research Laboratory (AFRL), Air Force Office of Scientific Research (AFOSR); National Science Foundation (NSF); USDOE National Nuclear Security Administration (NNSA)

OSTI Identifier:: 1425688

Alternate Identifier(s):: OSTI ID: 1510066

Grant/Contract Number:: NA0002374; AC02-06CH11357

Resource Type:: Published Article

Journal Name:: International Journal of High Performance Computing Applications

Additional Journal Information:: Journal Name: International Journal of High Performance Computing Applications Journal Volume: 33 Journal Issue: 2; Journal ID: ISSN 1094-3420

Publisher:: SAGE Publications

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING; lossy compression; checkpoint restart; compression; error propagation; error tolerance selection; exascale; fault tolerance

Citation Formats


                    Calhoun, Jon, Cappello, Franck, Olson, Luke N., Snir, Marc, and Gropp, William D. Exploring the feasibility of lossy compression for PDE simulations.  United States: N. p., 2018. 
Web.  doi:10.1177/1094342018762036.

Copy to clipboard


                    Calhoun, Jon, Cappello, Franck, Olson, Luke N., Snir, Marc, & Gropp, William D. Exploring the feasibility of lossy compression for PDE simulations.  United States.  https://doi.org/10.1177/1094342018762036

Copy to clipboard


                    Calhoun, Jon, Cappello, Franck, Olson, Luke N., Snir, Marc, and Gropp, William D. Mon .  
"Exploring the feasibility of lossy compression for PDE simulations".  United States.  https://doi.org/10.1177/1094342018762036.

Copy to clipboard


                    
@article{osti_1425688,

  title        = {Exploring the feasibility of lossy compression for PDE simulations},

  author       = {Calhoun, Jon and Cappello, Franck and Olson, Luke N. and Snir, Marc and Gropp, William D.},

  abstractNote = {Checkpoint restart plays an important role in high-performance computing (HPC) applications, allowing simulation runtime to extend beyond a single job allocation and facilitating recovery from hardware failure. Yet, as machines grow in size and in complexity, traditional approaches to checkpoint restart are becoming prohibitive. Current methods store a subset of the application's state and exploit the memory hierarchy in the machine. However, as the energy cost of data movement continues to dominate, further reductions in checkpoint size are needed. Lossy compression, which can significantly reduce checkpoint sizes, offers a potential to reduce computational cost in checkpoint restart. This article investigates the use of numerical properties of partial differential equation (PDE) simulations, such as bounds on the truncation error, to evaluate the feasibility of using lossy compression in checkpointing PDE simulations. Restart from a checkpoint with lossy compression is considered for a fail-stop error in two time-dependent HPC application codes: PlasComCM and Nek5000. Here, the results show that error in application variables due to a restart from a lossy compressed checkpoint can be masked by the numerical error in the discretization, leading to increased efficiency in checkpoint restart without influencing overall accuracy in the simulation.},

  doi          = {10.1177/1094342018762036},

  journal      = {International Journal of High Performance Computing Applications},

  number       = 2,

  volume       = 33,

  place        = {United States},

  year         = {Mon Mar 12 00:00:00 EDT 2018},

  month        = {Mon Mar 12 00:00:00 EDT 2018}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Publisher's Version of Record
https://doi.org/10.1177/1094342018762036

Other availability

Search WorldCat to find libraries that may hold this journal

Citation Metrics:

Cited by: 21 works

Citation information provided by
Web of Science

Save / Share:

Export Metadata

Save to My Library

Works referenced in this record:

High Throughput Compression of Double-Precision Floating-Point Data
conference, March 2007

Burtscher, Martin; Ratanaworabhan, Paruj
2007 Data Compression Conference (DCC'07)
DOI: 10.1109/DCC.2007.44

Significantly Improving Lossy Compression for Scientific Data Sets Based on Multidimensional Prediction and Error-Controlled Quantization
conference, May 2017

Tao, Dingwen; Di, Sheng; Chen, Zizhong
2017 IEEE International Parallel and Distributed Processing Symposium (IPDPS)
DOI: 10.1109/IPDPS.2017.115

Fast Error-Bounded Lossy HPC Data Compression with SZ
conference, May 2016

Di, Sheng; Cappello, Franck
2016 IEEE International Parallel and Distributed Processing Symposium (IPDPS)
DOI: 10.1109/IPDPS.2016.11

Collective I/O Tuning Using Analytical and Machine Learning Models
conference, September 2015

Isaila, Florin; Balaprakash, Prasanna; Wild, Stefan M.
2015 IEEE International Conference on Cluster Computing (CLUSTER)
DOI: 10.1109/CLUSTER.2015.29

Fixed-Rate Compressed Floating-Point Arrays
journal, December 2014

Lindstrom, Peter
IEEE Transactions on Visualization and Computer Graphics, Vol. 20, Issue 12
DOI: 10.1109/TVCG.2014.2346458

Toward Exascale Resilience
journal, September 2009

Cappello, Franck; Geist, Al; Gropp, Bill
The International Journal of High Performance Computing Applications, Vol. 23, Issue 4
DOI: 10.1177/1094342009347767

On the Viability of Compression for Reducing the Overheads of Checkpoint/Restart-Based Fault Tolerance
conference, September 2012

Ibtesham, Dewan; Arnold, Dorian; Bridges, Patrick G.
2012 41st International Conference on Parallel Processing (ICPP)
DOI: 10.1109/ICPP.2012.45

Similar Records in DOE PAGES and OSTI.GOV collections:

DPM: A deep learning PDE augmentation method with application to large-eddy simulation

Journal Article Sirignano, Justin ; MacArt, Jonathan F. ; Freund, Jonathan B. - Journal of Computational Physics

A framework is introduced that leverages known physics to reduce overfitting in machine learning for scientific applications. The partial differential equation (PDE) that expresses the physics is augmented with a neural network that uses available data to learn a description of the corresponding unknown or unrepresented physics. Training within this combined system corrects for missing, unknown, or erroneously represented physics, including discretization errors associated with the PDE's numerical solution. For optimization of the network within the PDE, an adjoint PDE is solved to provide high-dimensional gradients, and a stochastic adjoint method (SAM) further accelerates training. Additionally, the approach is demonstratedmore »« less
https://doi.org/10.1016/j.jcp.2020.109811

Full Text Available
Performance Optimization for Relative-Error-Bounded Lossy Compression on Scientific Data

Journal Article Zou, Xiangyu ; Lu, Tao ; Xia, Wen ; ... - IEEE Transactions on Parallel and Distributed Systems

Scientific simulations in high-performance computing (HPC) environments generate vast volume of data, which may cause a severe I/O bottleneck at runtime and a huge burden on storage space for postanalysis. Unlike traditional data reduction schemes such as deduplication or lossless compression, not only can error-controlled lossy compression significantly reduce the data size but it also holds the promise to satisfy user demand on error control. Pointwise relative error bounds (i.e., compression errors depends on the data values) are widely used by many scientific applications with lossy compression since error control can adapt to the error bound in the dataset automatically.more »« less
Cited by 5
https://doi.org/10.1109/TPDS.2020.2972548

Full Text Available
Accelerating Relative-error Bounded Lossy Compression for HPC datasets with Precomputation-Based Mechanisms

Conference Zou, Xiangyu ; Lu, Tao ; Xia, Wen ; ...

Scientific simulations in high-performance computing (HPC) environments are producing vast volume of data, which may cause a severe I/O bottleneck at runtime and a huge burden on storage space for post-analysis. Unlike the traditional data reduction schemes (such as deduplication or lossless compression), not only can error-controlled lossy compression significantly reduce the data size but it can also hold the promise to satisfy user demand on error control. Point-wise relative error bounds (i.e., compression errors depends on the data values) are widely used by many scientific applications in the lossy compression, since error control can adapt to the precision inmore »« less
Towards Low-Overhead Resilience for Data Parallel Deep Learning

Conference Nicolae, Bogdan ; Hobson, Tanner ; Yildiz, Orcun ; ...

Data parallel techniques have been widely adopted both in academia and industry as a tool to enable scalable training of deep learning models. At scale, DL training jobs can fail due to software or hardware bugs, may need to be preempted or terminated due to unexpected events, or may perform suboptimally because they were misconfigured. Under such circumstances, there is a need to recover and/or reconfigure data-parallel DL training jobs on-the-fly, while minimizing the impact on the accuracy of the DNN model and the runtime overhead. In this regard, state-of-art techniques adopted by the HPC community mostly rely on checkpoint-restart,more »« less
https://doi.org/10.1109/CCGrid54584.2022.00043
...And Eat it Too: High Read Performance in Write-Optimized HPC I/O Middleware File Formats

Conference Klasky, Scott A ; Lofstead, J. ; Bent, John ; ...

As HPC applications run on increasingly high process counts on larger and larger machines, both the frequency of checkpoints needed for fault tolerance and the resolution and size of Data Analysis Dumps are expected to increase proportionally. In order to maintain an acceptable ratio of time spent performing useful computation work to time spent performing I/O, write bandwidth to the underlying storage system must increase proportionally to this increase in the checkpoint and computation size. Unfortunately, popular scientific self-describing file formats such as netCDF and HDF5 are designed with a focus on portability and flexibility. Extra care and careful craftingmore »« less

Similar Records

Title: Exploring the feasibility of lossy compression for PDE simulations

Abstract

Citation Formats

High Throughput Compression of Double-Precision Floating-Point Data conference, March 2007

Significantly Improving Lossy Compression for Scientific Data Sets Based on Multidimensional Prediction and Error-Controlled Quantization conference, May 2017

Fast Error-Bounded Lossy HPC Data Compression with SZ conference, May 2016

Collective I/O Tuning Using Analytical and Machine Learning Models conference, September 2015

Fixed-Rate Compressed Floating-Point Arrays journal, December 2014

Toward Exascale Resilience journal, September 2009

On the Viability of Compression for Reducing the Overheads of Checkpoint/Restart-Based Fault Tolerance conference, September 2012

High Throughput Compression of Double-Precision Floating-Point Data
conference, March 2007

Significantly Improving Lossy Compression for Scientific Data Sets Based on Multidimensional Prediction and Error-Controlled Quantization
conference, May 2017

Fast Error-Bounded Lossy HPC Data Compression with SZ
conference, May 2016

Collective I/O Tuning Using Analytical and Machine Learning Models
conference, September 2015

Fixed-Rate Compressed Floating-Point Arrays
journal, December 2014

Toward Exascale Resilience
journal, September 2009

On the Viability of Compression for Reducing the Overheads of Checkpoint/Restart-Based Fault Tolerance
conference, September 2012