skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Small file aggregation in a parallel computing system

Patent ·
OSTI ID:1151820
; ; ; ;

Techniques are provided for small file aggregation in a parallel computing system. An exemplary method for storing a plurality of files generated by a plurality of processes in a parallel computing system comprises aggregating the plurality of files into a single aggregated file; and generating metadata for the single aggregated file. The metadata comprises an offset and a length of each of the plurality of files in the single aggregated file. The metadata can be used to unpack one or more of the files from the single aggregated file.

Research Organization:
Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
Sponsoring Organization:
USDOE
DOE Contract Number:
AC52-06NA25396
Assignee:
EMC Corporation (Hopkinton, MA); Los Alamos National Security, LLC (Los Alamos, NM)
Patent Number(s):
8,825,652
Application Number:
13/536,315
OSTI ID:
1151820
Country of Publication:
United States
Language:
English

References (7)

Document storing and managing system patent October 1998
Write anywhere file-system layout patent October 1999
Storage management system with file aggregation patent August 2000
Rule based aggregation of files and transactions in a switched file system patent-application July 2004
Maintaining an aggregate including active files in a storage pool patent-application February 2007
Maintaining active-only copy storage pools patent-application February 2007
PLFS: a checkpoint filesystem for parallel applications conference January 2009

Cited By (3)

Storing data files in a file system patent May 2016
Architecture and method for a burst buffer using flash technology patent March 2016
Burst buffer appliance with small file aggregation patent March 2015

Similar Records

Related Subjects

97 MATHEMATICS AND COMPUTING