McrEngine: A Scalable Checkpointing System Using Data-Aware Aggregation and Compression

Islam, Tanzima Zerin; Mohror, Kathryn; Bagchi, Saurabh; Moody, Adam; de Supinski, Bronis R.; Eigenmann, Rudolf

doi:10.1155/2013/341672

Title: McrEngine: A Scalable Checkpointing System Using Data-Aware Aggregation and Compression

Full Record
Other Related Research

Abstract

High performance computing (HPC) systems use checkpoint-restart to tolerate failures. Typically, applications store their states in checkpoints on a parallel file system (PFS). As applications scale up, checkpoint-restart incurs high overheads due to contention for PFS resources. The high overheads force large-scale applications to reduce checkpoint frequency, which means more compute time is lost in the event of failure. We alleviate this problem through a scalable checkpoint-restart system, mcrEngine. McrEngine aggregates checkpoints from multiple application processes with knowledge of the data semantics available through widely-used I/O libraries, e.g., HDF5 and netCDF, and compresses them. Our novel scheme improves compressibility of checkpoints up to 115% over simple concatenation and compression. Our evaluation with large-scale application checkpoints show that mcrEngine reduces checkpointing overhead by up to 87% and restart overhead by up to 62% over a baseline with no aggregation or compression.

Authors:

Islam, Tanzima Zerin ^[1]; Mohror, Kathryn ^[2]; Bagchi, Saurabh ^[1]; Moody, Adam ^[2]; de Supinski, Bronis R. ^[2]; Eigenmann, Rudolf ^[1]

School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN, USA
Lawrence Livermore National Laboratory, Livermore, CA, USA

Publication Date:: Tue Jan 01 00:00:00 EST 2013

Sponsoring Org.:: USDOE

OSTI Identifier:: 1197891

Grant/Contract Number:: AC52-07NA27344

Resource Type:: Published Article

Journal Name:: Scientific Programming

Additional Journal Information:: Journal Name: Scientific Programming Journal Volume: 21 Journal Issue: 3-4; Journal ID: ISSN 1058-9244

Publisher:: Hindawi Publishing Corporation

Country of Publication:: Egypt

Language:: English

Citation Formats


                    Islam, Tanzima Zerin, Mohror, Kathryn, Bagchi, Saurabh, Moody, Adam, de Supinski, Bronis R., and Eigenmann, Rudolf. McrEngine: A Scalable Checkpointing System Using Data-Aware Aggregation and Compression.  Egypt: N. p., 2013. 
Web.  doi:10.1155/2013/341672.

Copy to clipboard


                    Islam, Tanzima Zerin, Mohror, Kathryn, Bagchi, Saurabh, Moody, Adam, de Supinski, Bronis R., & Eigenmann, Rudolf. McrEngine: A Scalable Checkpointing System Using Data-Aware Aggregation and Compression.  Egypt.  https://doi.org/10.1155/2013/341672

Copy to clipboard


                    Islam, Tanzima Zerin, Mohror, Kathryn, Bagchi, Saurabh, Moody, Adam, de Supinski, Bronis R., and Eigenmann, Rudolf. Tue .  
"McrEngine: A Scalable Checkpointing System Using Data-Aware Aggregation and Compression".  Egypt.  https://doi.org/10.1155/2013/341672.

Copy to clipboard


                    
@article{osti_1197891,

  title        = {McrEngine: A Scalable Checkpointing System Using Data-Aware Aggregation and Compression},

  author       = {Islam, Tanzima Zerin and Mohror, Kathryn and Bagchi, Saurabh and Moody, Adam and de Supinski, Bronis R. and Eigenmann, Rudolf},

  abstractNote = {High performance computing (HPC) systems use checkpoint-restart to tolerate failures. Typically, applications store their states in checkpoints on a parallel file system (PFS). As applications scale up, checkpoint-restart incurs high overheads due to contention for PFS resources. The high overheads force large-scale applications to reduce checkpoint frequency, which means more compute time is lost in the event of failure. We alleviate this problem through a scalable checkpoint-restart system, mcrEngine. McrEngine aggregates checkpoints from multiple application processes with knowledge of the data semantics available through widely-used I/O libraries, e.g., HDF5 and netCDF, and compresses them. Our novel scheme improves compressibility of checkpoints up to 115% over simple concatenation and compression. Our evaluation with large-scale application checkpoints show that mcrEngine reduces checkpointing overhead by up to 87% and restart overhead by up to 62% over a baseline with no aggregation or compression.},

  doi          = {10.1155/2013/341672},

  journal      = {Scientific Programming},

  number       = 3-4,

  volume       = 21,

  place        = {Egypt},

  year         = {Tue Jan 01 00:00:00 EST 2013},

  month        = {Tue Jan 01 00:00:00 EST 2013}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Publisher's Version of Record
https://doi.org/10.1155/2013/341672

Other availability

Search WorldCat to find libraries that may hold this journal

Citation Metrics:

Cited by: 24 works

Citation information provided by
Web of Science

Save / Share:

Export Metadata

Save to My Library

Similar Records in DOE PAGES and OSTI.GOV collections:

...And Eat it Too: High Read Performance in Write-Optimized HPC I/O Middleware File Formats

Conference Klasky, Scott A ; Lofstead, J. ; Bent, John ; ...

As HPC applications run on increasingly high process counts on larger and larger machines, both the frequency of checkpoints needed for fault tolerance and the resolution and size of Data Analysis Dumps are expected to increase proportionally. In order to maintain an acceptable ratio of time spent performing useful computation work to time spent performing I/O, write bandwidth to the underlying storage system must increase proportionally to this increase in the checkpoint and computation size. Unfortunately, popular scientific self-describing file formats such as netCDF and HDF5 are designed with a focus on portability and flexibility. Extra care and careful craftingmore »« less
Orchestrating Fault Prediction with Live Migration and Checkpointing

Conference Behera, Subhendu ; Wan, Lipeng ; Mueller, Frank ; ...

Checkpoint/Restart (C/R) is widely used to provide fault tolerance on High-Performance Computing (HPC) systems. However, Parallel File System (PFS) overhead and failure uncertainty cause significant application overhead. This paper develops an adaptive multi-level C/R model that incorporates a failure prediction and analysis model, which orchestrates failure prediction, checkpointing, checkpoint frequency, and proactive live migration along with the additional benefit of Burst Buffers (BB). It effectively reduces the overheads due to failures, checkpointing, and recovery. Simulation results for the Summit supercomputer yield a reduction of ~20%-86% in application overhead due to BBs, orchestrated failure prediction, and migration. We also observe amore »« less
Full Text Available
Detailed Modeling and Evaluation of a Scalable Multilevel Checkpointing System

Journal Article Mohror, Kathryn ; Moody, Adam ; Bronevetsky, Greg ; ... - IEEE Transactions on Parallel and Distributed Systems

High-performance computing (HPC) systems are growing more powerful by utilizing more components. As the system mean time before failure correspondingly drops, applications must checkpoint frequently to make progress. But, at scale, the cost of checkpointing becomes prohibitive. A solution to this problem is multilevel checkpointing, which employs multiple types of checkpoints in a single run. Moreover, lightweight checkpoints can handle the most common failure modes, while more expensive checkpoints can handle severe failures. We designed a multilevel checkpointing library, the Scalable Checkpoint/Restart (SCR) library, that writes lightweight checkpoints to node-local storage in addition to the parallel file system. We presentmore »« less
https://doi.org/10.1109/TPDS.2013.100

Full Text Available
Detailed Modeling, Design, and Evaluation of a Scalable Multi-level Checkpointing System

Technical Report Moody, A T ; Bronevetsky, G ; Mohror, K M ; ...

High-performance computing (HPC) systems are growing more powerful by utilizing more hardware components. As the system mean-time-before-failure correspondingly drops, applications must checkpoint more frequently to make progress. However, as the system memory sizes grow faster than the bandwidth to the parallel file system, the cost of checkpointing begins to dominate application run times. A potential solution to this problem is to use multi-level checkpointing, which employs multiple types of checkpoints with different costs and different levels of resiliency in a single run. The goal is to design light-weight checkpoints to handle the most common failure modes and rely on moremore »« less
https://doi.org/10.2172/984082

Full Text Available
Combining Partial Redundancy and Checkpointing for HPC

Conference Elliott, James ; Kharbas, Kishor H ; Fiala, David J ; ...

Today's largest High Performance Computing (HPC) systems exceed one Petaflops (10^15 floating point operations per second) and exascale systems are projected within seven years. But reliability is becoming one of the major challenges faced by exascale computing. With billion-core parallelism, the mean time to failure is projected to be in the range of minutes or hours instead of days. Failures are becoming the norm rather than the exception during execution of HPC applications. Current fault tolerance techniques in HPC focus on reactive ways to mitigate faults, namely via checkpoint and restart (C/R). Apart from storage overheads, C/R-based fault recovery comesmore »« less

Similar Records