Accelerating shared file checkpoint with local burst buffers
A data management system and method for accelerating shared file checkpointing. Written application data is aggregated in an application data file created in a local burst buffer memory at a compute node, and an associated data mapping built index to maintain information related to the offsets into a shared file at which segments of the application data is to be stored in a parallel file system, and where in the buffer those segments are located. The node asynchronously transfers a data file containing the application data and the associated data mapping index to a file server for shared file storage. The data management system and method further accelerates shared file checkpointing in which a shared file, together with a map file that specifies how the shared file is to be distributed, is asynchronously transferred to local burst buffer memories at the nodes to accelerate reading of the shared file.
- Research Organization:
- International Business Machines Corp., Armonk, NY (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- B604142
- Assignee:
- International Business Machines Corporation (Armonk, NY)
- Patent Number(s):
- 11,301,165
- Application Number:
- 15/963,700
- OSTI ID:
- 1892825
- Resource Relation:
- Patent File Date: 04/26/2018
- Country of Publication:
- United States
- Language:
- English
Similar Records
Optimizing checkpoint data placement with guaranteed burst buffer endurance in large-scale hierarchical storage systems
TRIO: Burst Buffer Based I/O Orchestration, In: 2015 IEEE International Conference on Cluster Computing