Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

Accelerating shared file checkpoint with local burst buffers

Patent ·
OSTI ID:1892825
; ;
A data management system and method for accelerating shared file checkpointing. Written application data is aggregated in an application data file created in a local burst buffer memory at a compute node, and an associated data mapping built index to maintain information related to the offsets into a shared file at which segments of the application data is to be stored in a parallel file system, and where in the buffer those segments are located. The node asynchronously transfers a data file containing the application data and the associated data mapping index to a file server for shared file storage. The data management system and method further accelerates shared file checkpointing in which a shared file, together with a map file that specifies how the shared file is to be distributed, is asynchronously transferred to local burst buffer memories at the nodes to accelerate reading of the shared file.
Research Organization:
International Business Machines Corp., Armonk, NY (United States)
Sponsoring Organization:
USDOE
Assignee:
International Business Machines Corporation (Armonk, NY)
Patent Number(s):
11,301,165
Application Number:
15/963,700
OSTI ID:
1892825
Country of Publication:
United States
Language:
English

References (5)

PLFS: a checkpoint filesystem for parallel applications conference January 2009
BurstMem: A high-performance burst buffer system for scientific applications conference October 2014
Integrated in-system storage architecture for high performance computing conference June 2012
How Much SSD Is Useful for Resilience in Supercomputers conference January 2015
A User-Level InfiniBand-Based File System and Checkpoint Strategy for Burst Buffers conference May 2014

Similar Records

Using the Sirocco File System for high-bandwidth checkpoints.
Technical Report · Tue Jan 31 23:00:00 EST 2012 · OSTI ID:1039010
An Ephemeral Burst-Buffer File System for Scientific Applications
Software · Tue Apr 11 00:00:00 EDT 2017 · OSTI ID:1351607
Optimizing checkpoint data placement with guaranteed burst buffer endurance in large-scale hierarchical storage systems
Journal Article · Thu Oct 13 20:00:00 EDT 2016 · Journal of Parallel and Distributed Computing · OSTI ID:1648989

Related Subjects