Cloud object store for archive storage of high performance computing data using decoupling middleware
Abstract
Cloud object storage is enabled for archived data, such as checkpoints and results, of high performance computing applications using a middleware process. A plurality of archived files, such as checkpoint files and results, generated by a plurality of processes in a parallel computing system are stored by obtaining the plurality of archived files from the parallel computing system; converting the plurality of archived files to objects using a log structured file system middleware process; and providing the objects for storage in a cloud object storage system. The plurality of processes may run, for example, on a plurality of compute nodes. The log structured file system middleware process may be embodied, for example, as a Parallel Log-Structured File System (PLFS). The log structured file system middleware process optionally executes on a burst buffer node.
- Inventors:
- Issue Date:
- Research Org.:
- Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
- Sponsoring Org.:
- USDOE
- OSTI Identifier:
- 1187939
- Patent Number(s):
- 9069778
- Application Number:
- 13/730,097
- Assignee:
- Los Alamos National Security, LLC (Los Alamos, NM)
- Patent Classifications (CPCs):
-
G - PHYSICS G06 - COMPUTING G06F - ELECTRIC DIGITAL DATA PROCESSING
- DOE Contract Number:
- AC52-06NA25396
- Resource Type:
- Patent
- Resource Relation:
- Patent File Date: 2012 Dec 28
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 97 MATHEMATICS AND COMPUTING
Citation Formats
Bent, John M., Faibish, Sorin, and Grider, Gary. Cloud object store for archive storage of high performance computing data using decoupling middleware. United States: N. p., 2015.
Web.
Bent, John M., Faibish, Sorin, & Grider, Gary. Cloud object store for archive storage of high performance computing data using decoupling middleware. United States.
Bent, John M., Faibish, Sorin, and Grider, Gary. Tue .
"Cloud object store for archive storage of high performance computing data using decoupling middleware". United States. https://www.osti.gov/servlets/purl/1187939.
@article{osti_1187939,
title = {Cloud object store for archive storage of high performance computing data using decoupling middleware},
author = {Bent, John M. and Faibish, Sorin and Grider, Gary},
abstractNote = {Cloud object storage is enabled for archived data, such as checkpoints and results, of high performance computing applications using a middleware process. A plurality of archived files, such as checkpoint files and results, generated by a plurality of processes in a parallel computing system are stored by obtaining the plurality of archived files from the parallel computing system; converting the plurality of archived files to objects using a log structured file system middleware process; and providing the objects for storage in a cloud object storage system. The plurality of processes may run, for example, on a plurality of compute nodes. The log structured file system middleware process may be embodied, for example, as a Parallel Log-Structured File System (PLFS). The log structured file system middleware process optionally executes on a burst buffer node.},
doi = {},
journal = {},
number = ,
volume = ,
place = {United States},
year = {Tue Jun 30 00:00:00 EDT 2015},
month = {Tue Jun 30 00:00:00 EDT 2015}
}
Works referenced in this record:
Parallel Log Structured File System Collective Buffering to Achieve a Compact Representation of Scientific and/or Dimensional Data
patent-application, June 2013
- Grider, Gary A.; Poole, Stephen W.
- US Patent Document 13/722946; 20130159364
PLFS: a checkpoint filesystem for parallel applications
conference, January 2009
- Bent, John; Gibson, Garth; Grider, Gary
The design and implementation of a log-structured file system
journal, February 1992
- Rosenblum, Mendel; Ousterhout, John K.
- ACM Transactions on Computer Systems, Vol. 10, Issue 1, p. 26-52