Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

Parallel compression of data chunks of a shared data object using a log-structured file system

Patent ·
OSTI ID:1330322
; ;
Techniques are provided for parallel compression of data chunks being written to a shared object. A client executing on a compute node or a burst buffer node in a parallel computing system stores a data chunk generated by the parallel computing system to a shared data object on a storage node by compressing the data chunk; and providing the data compressed data chunk to the storage node that stores the shared object. The client and storage node may employ Log-Structured File techniques. The compressed data chunk can be de-compressed by the client when the data chunk is read. A storage node stores a data chunk as part of a shared object by receiving a compressed version of the data chunk from a compute node; and storing the compressed version of the data chunk to the shared data object on the storage node.
Research Organization:
Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
Sponsoring Organization:
USDOE
DOE Contract Number:
AC52-06NA25396
Assignee:
EMC Corporation (Hopkinton, MA); Los Alamos National Security, LLC (Los Alamos, NM)
Patent Number(s):
9,477,682
Application Number:
13/799,228
OSTI ID:
1330322
Country of Publication:
United States
Language:
English

References (3)

Using Active NVRAM for Cloud I/O conference October 2011
PLFS: a checkpoint filesystem for parallel applications conference January 2009
An Evolutionary Path to Object Storage Access
  • Goodell, David; Kim, Seong Jo; Latham, Robert
  • 2012 SC Companion: High Performance Computing, Networking, Storage and Analysis (SCC), 2012 SC Companion: High Performance Computing, Networking Storage and Analysis https://doi.org/10.1109/SC.Companion.2012.17
conference November 2012

Similar Records

Related Subjects

97 MATHEMATICS AND COMPUTING