Cloud object store for archive storage of high performance computing data using decoupling middleware
Cloud object storage is enabled for archived data, such as checkpoints and results, of high performance computing applications using a middleware process. A plurality of archived files, such as checkpoint files and results, generated by a plurality of processes in a parallel computing system are stored by obtaining the plurality of archived files from the parallel computing system; converting the plurality of archived files to objects using a log structured file system middleware process; and providing the objects for storage in a cloud object storage system. The plurality of processes may run, for example, on a plurality of compute nodes. The log structured file system middleware process may be embodied, for example, as a Parallel Log-Structured File System (PLFS). The log structured file system middleware process optionally executes on a burst buffer node.
- Research Organization:
- Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- AC52-06NA25396
- Assignee:
- Los Alamos National Security, LLC (Los Alamos, NM)
- Patent Number(s):
- 9,069,778
- Application Number:
- 13/730,097
- OSTI ID:
- 1187939
- Country of Publication:
- United States
- Language:
- English
The design and implementation of a log-structured file system
|
journal | February 1992 |
| PLFS: a checkpoint filesystem for parallel applications | conference | January 2009 |
Similar Records
Parallel file system with metadata distributed across partitioned key-value store c
Parallel checksumming of data chunks of a shared data object using a log-structured file system