skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Cloud object store for checkpoints of high performance computing applications using decoupling middleware

Abstract

Cloud object storage is enabled for checkpoints of high performance computing applications using a middleware process. A plurality of files, such as checkpoint files, generated by a plurality of processes in a parallel computing system are stored by obtaining said plurality of files from said parallel computing system; converting said plurality of files to objects using a log structured file system middleware process; and providing said objects for storage in a cloud object storage system. The plurality of processes may run, for example, on a plurality of compute nodes. The log structured file system middleware process may be embodied, for example, as a Parallel Log-Structured File System (PLFS). The log structured file system middleware process optionally executes on a burst buffer node.

Inventors:
; ;
Publication Date:
Research Org.:
Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
1247993
Patent Number(s):
9,317,521
Application Number:
13/730,058
Assignee:
EMC Corporation (Hopkinton, MA) Los Alamos National Security, LLC (Los Alamos, NM) LANL
DOE Contract Number:  
AC52-06NA25396
Resource Type:
Patent
Resource Relation:
Patent File Date: 2012 Dec 28
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING

Citation Formats

Bent, John M., Faibish, Sorin, and Grider, Gary. Cloud object store for checkpoints of high performance computing applications using decoupling middleware. United States: N. p., 2016. Web.
Bent, John M., Faibish, Sorin, & Grider, Gary. Cloud object store for checkpoints of high performance computing applications using decoupling middleware. United States.
Bent, John M., Faibish, Sorin, and Grider, Gary. Tue . "Cloud object store for checkpoints of high performance computing applications using decoupling middleware". United States. doi:. https://www.osti.gov/servlets/purl/1247993.
@article{osti_1247993,
title = {Cloud object store for checkpoints of high performance computing applications using decoupling middleware},
author = {Bent, John M. and Faibish, Sorin and Grider, Gary},
abstractNote = {Cloud object storage is enabled for checkpoints of high performance computing applications using a middleware process. A plurality of files, such as checkpoint files, generated by a plurality of processes in a parallel computing system are stored by obtaining said plurality of files from said parallel computing system; converting said plurality of files to objects using a log structured file system middleware process; and providing said objects for storage in a cloud object storage system. The plurality of processes may run, for example, on a plurality of compute nodes. The log structured file system middleware process may be embodied, for example, as a Parallel Log-Structured File System (PLFS). The log structured file system middleware process optionally executes on a burst buffer node.},
doi = {},
journal = {},
number = ,
volume = ,
place = {United States},
year = {Tue Apr 19 00:00:00 EDT 2016},
month = {Tue Apr 19 00:00:00 EDT 2016}
}

Patent:

Save / Share:

Works referenced in this record:

PLFS: a checkpoint filesystem for parallel applications
conference, January 2009