Template based parallel checkpointing in a massively parallel computer system
- Rochester, MN
A method and apparatus for a template based parallel checkpoint save for a massively parallel super computer system using a parallel variation of the rsync protocol, and network broadcast. In preferred embodiments, the checkpoint data for each node is compared to a template checkpoint file that resides in the storage and that was previously produced. Embodiments herein greatly decrease the amount of data that must be transmitted and stored for faster checkpointing and increased efficiency of the computer system. Embodiments are directed to a parallel computer system with nodes arranged in a cluster with a high speed interconnect that can perform broadcast communication. The checkpoint contains a set of actual small data blocks with their corresponding checksums from all nodes in the system. The data blocks may be compressed using conventional non-lossy data compression algorithms to further reduce the overall checkpoint size.
- Research Organization:
- International Business Machines Corp., Armonk, NY (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- B519700
- Assignee:
- International Business Machines Corporation (Armonk, NY)
- Patent Number(s):
- 7,478,278
- Application Number:
- 11/106,010
- OSTI ID:
- 985865
- Country of Publication:
- United States
- Language:
- English
Similar Records
Detailed Modeling, Design, and Evaluation of a Scalable Multi-level Checkpointing System
Cloud object store for checkpoints of high performance computing applications using decoupling middleware