Cloud object store for checkpoints of high performance computing applications using decoupling middleware

Title: Cloud object store for checkpoints of high performance computing applications using decoupling middleware

Cloud object storage is enabled for checkpoints of high performance computing applications using a middleware process. A plurality of files, such as checkpoint files, generated by a plurality of processes in a parallel computing system are stored by obtaining said plurality of files from said parallel computing system; converting said plurality of files to objects using a log structured file system middleware process; and providing said objects for storage in a cloud object storage system. The plurality of processes may run, for example, on a plurality of compute nodes. The log structured file system middleware process may be embodied, for example, as a Parallel Log-Structured File System (PLFS). The log structured file system middleware process optionally executes on a burst buffer node.

Inventors:: Bent, John M.; Faibish, Sorin; Grider, Gary

Issue Date:: 2016-04-19

OSTI Identifier:: 1247993

Assignee:: EMC Corporation (Hopkinton, MA) Los Alamos National Security, LLC (Los Alamos, NM) LANL

Patent Number(s):: 9,317,521

Application Number:: 13/730,058

Contract Number:: AC52-06NA25396

Resource Relation:: Patent File Date: 2012 Dec 28

Research Org:: Los Alamos National Lab. (LANL), Los Alamos, NM (United States)

Sponsoring Org:: USDOE

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING

Full Text
View Full Text

Other works cited in this record:

PLFS: a checkpoint filesystem for parallel applications
conference, January 2009

Bent, John; Gibson, Garth; Grider, Gary
DOI: 10.1145/1654059.1654081

Similar records in DOepatents and OSTI.GOV collections:

Cloud object store for archive storage of high performance computing data using decoupling middleware

Patent Bent, John M. ; Faibish, Sorin ; Grider, Gary

Cloud object storage is enabled for archived data, such as checkpoints and results, of high performance computing applications using a middleware process. A plurality of archived files, such as checkpoint files and results, generated by a plurality of processes in a parallel computing system are stored by obtaining the plurality of archived files from the parallel computing system; converting the plurality of archived files to objects using a log structured file system middleware process; and providing the objects for storage in a cloud object storage system. The plurality of processes may run, for example, on a plurality of compute nodes.more »« less
Full Text Available June 2015
Parallel sort with a ranged, partitioned key-value store in a high perfomance computing environment

Patent Bent, John M. ; Faibish, Sorin ; Grider, Gary ; Torres, Aaron ; Poole, Stephen W.

Improved sorting techniques are provided that perform a parallel sort using a ranged, partitioned key-value store in a high performance computing (HPC) environment. A plurality of input data files comprising unsorted key-value data in a partitioned key-value store are sorted. The partitioned key-value store comprises a range server for each of a plurality of ranges. Each input data file has an associated reader thread. Each reader thread reads the unsorted key-value data in the corresponding input data file and performs a local sort of the unsorted key-value data to generate sorted key-value data. A plurality of sorted, ranged subsets ofmore »« less
Full Text Available January 2016
Debugging a high performance computing program

Patent Gooding, Thomas M.

Methods, apparatus, and computer program products are disclosed for debugging a high performance computing program by gathering lists of addresses of calling instructions for a plurality of threads of execution of the program, assigning the threads to groups in dependence upon the addresses, and displaying the groups to identify defective threads.
Full Text Available August 2013
Matrix multiplication operations with data pre-conditioning in a high performance computing architecture

Patent Eichenberger, Alexandre E ; Gschwind, Michael K ; Gunnels, John A

Mechanisms for performing matrix multiplication operations with data pre-conditioning in a high performance computing architecture are provided. A vector load operation is performed to load a first vector operand of the matrix multiplication operation to a first target vector register. A load and splat operation is performed to load an element of a second vector operand and replicating the element to each of a plurality of elements of a second target vector register. A multiply add operation is performed on elements of the first target vector register and elements of the second target vector register to generate a partial productmore »« less
Full Text Available November 2013
Complex matrix multiplication operations with data pre-conditioning in a high performance computing architecture

Patent Eichenberger, Alexandre E ; Gschwind, Michael K ; Gunnels, John A

Mechanisms for performing a complex matrix multiplication operation are provided. A vector load operation is performed to load a first vector operand of the complex matrix multiplication operation to a first target vector register. The first vector operand comprises a real and imaginary part of a first complex vector value. A complex load and splat operation is performed to load a second complex vector value of a second vector operand and replicate the second complex vector value within a second target vector register. The second complex vector value has a real and imaginary part. A cross multiply add operation ismore »« less
Full Text Available February 2014