skip to main content

SciTech ConnectSciTech Connect

Title: Parallel compression of data chunks of a shared data object using a log-structured file system

Techniques are provided for parallel compression of data chunks being written to a shared object. A client executing on a compute node or a burst buffer node in a parallel computing system stores a data chunk generated by the parallel computing system to a shared data object on a storage node by compressing the data chunk; and providing the data compressed data chunk to the storage node that stores the shared object. The client and storage node may employ Log-Structured File techniques. The compressed data chunk can be de-compressed by the client when the data chunk is read. A storage node stores a data chunk as part of a shared object by receiving a compressed version of the data chunk from a compute node; and storing the compressed version of the data chunk to the shared data object on the storage node.
Authors:
; ;
Publication Date:
OSTI Identifier:
1330322
Report Number(s):
9,477,682
13/799,228
DOE Contract Number:
AC52-06NA25396
Resource Type:
Patent
Resource Relation:
Patent File Date: 2013 Mar 13
Research Org:
Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
Sponsoring Org:
USDOE
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING