Cloud object store for checkpoints of high performance computing applications using decoupling middleware
Abstract
Cloud object storage is enabled for checkpoints of high performance computing applications using a middleware process. A plurality of files, such as checkpoint files, generated by a plurality of processes in a parallel computing system are stored by obtaining said plurality of files from said parallel computing system; converting said plurality of files to objects using a log structured file system middleware process; and providing said objects for storage in a cloud object storage system. The plurality of processes may run, for example, on a plurality of compute nodes. The log structured file system middleware process may be embodied, for example, as a Parallel Log-Structured File System (PLFS). The log structured file system middleware process optionally executes on a burst buffer node.
- Inventors:
- Issue Date:
- Research Org.:
- Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
- Sponsoring Org.:
- USDOE
- OSTI Identifier:
- 1247993
- Patent Number(s):
- 9317521
- Application Number:
- 13/730,058
- Assignee:
- EMC Corporation (Hopkinton, MA) Los Alamos National Security, LLC (Los Alamos, NM)
- Patent Classifications (CPCs):
-
G - PHYSICS G06 - COMPUTING G06F - ELECTRIC DIGITAL DATA PROCESSING
- DOE Contract Number:
- AC52-06NA25396
- Resource Type:
- Patent
- Resource Relation:
- Patent File Date: 2012 Dec 28
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 97 MATHEMATICS AND COMPUTING
Citation Formats
Bent, John M., Faibish, Sorin, and Grider, Gary. Cloud object store for checkpoints of high performance computing applications using decoupling middleware. United States: N. p., 2016.
Web.
Bent, John M., Faibish, Sorin, & Grider, Gary. Cloud object store for checkpoints of high performance computing applications using decoupling middleware. United States.
Bent, John M., Faibish, Sorin, and Grider, Gary. Tue .
"Cloud object store for checkpoints of high performance computing applications using decoupling middleware". United States. https://www.osti.gov/servlets/purl/1247993.
@article{osti_1247993,
title = {Cloud object store for checkpoints of high performance computing applications using decoupling middleware},
author = {Bent, John M. and Faibish, Sorin and Grider, Gary},
abstractNote = {Cloud object storage is enabled for checkpoints of high performance computing applications using a middleware process. A plurality of files, such as checkpoint files, generated by a plurality of processes in a parallel computing system are stored by obtaining said plurality of files from said parallel computing system; converting said plurality of files to objects using a log structured file system middleware process; and providing said objects for storage in a cloud object storage system. The plurality of processes may run, for example, on a plurality of compute nodes. The log structured file system middleware process may be embodied, for example, as a Parallel Log-Structured File System (PLFS). The log structured file system middleware process optionally executes on a burst buffer node.},
doi = {},
journal = {},
number = ,
volume = ,
place = {United States},
year = {Tue Apr 19 00:00:00 EDT 2016},
month = {Tue Apr 19 00:00:00 EDT 2016}
}
Works referenced in this record:
PLFS: a checkpoint filesystem for parallel applications
conference, January 2009
- Bent, John; Gibson, Garth; Grider, Gary
Parallel Log Structured File System Collective Buffering to Achieve a Compact Representation of Scientific and/or Dimensional Data
patent-application, June 2013
- Grider, Gary A.; Poole, Stephen W.
- US Patent Document 13/722946; 20130159364
Active Non-Volatile Memory Post-Processing
patent-application, August 2013
- Kannan, Sudarsun; Milojicic, Dejan S.; Talwar, Vanish
- US Patent Application 13/404619; 20130227194