Cloud object store for checkpoints of high performance computing applications using decoupling middleware
Cloud object storage is enabled for checkpoints of high performance computing applications using a middleware process. A plurality of files, such as checkpoint files, generated by a plurality of processes in a parallel computing system are stored by obtaining said plurality of files from said parallel computing system; converting said plurality of files to objects using a log structured file system middleware process; and providing said objects for storage in a cloud object storage system. The plurality of processes may run, for example, on a plurality of compute nodes. The log structured file system middleware process may be embodied, for example, as a Parallel Log-Structured File System (PLFS). The log structured file system middleware process optionally executes on a burst buffer node.
- Research Organization:
- Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- AC52-06NA25396
- Assignee:
- EMC Corporation (Hopkinton, MA) Los Alamos National Security, LLC (Los Alamos, NM)
- Patent Number(s):
- 9,317,521
- Application Number:
- 13/730,058
- OSTI ID:
- 1247993
- Resource Relation:
- Patent File Date: 2012 Dec 28
- Country of Publication:
- United States
- Language:
- English
| PLFS: a checkpoint filesystem for parallel applications | conference | January 2009 |
Parallel Log Structured File System Collective Buffering to Achieve a Compact Representation of Scientific and/or Dimensional Data
|
patent-application | June 2013 |
Active Non-Volatile Memory Post-Processing
|
patent-application | August 2013 |
Similar Records
...And Eat it Too: High Read Performance in Write-Optimized HPC I/O Middleware File Formats
Parallel file system with metadata distributed across partitioned key-value store c