skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Small file aggregation in a parallel computing system

Abstract

Techniques are provided for small file aggregation in a parallel computing system. An exemplary method for storing a plurality of files generated by a plurality of processes in a parallel computing system comprises aggregating the plurality of files into a single aggregated file; and generating metadata for the single aggregated file. The metadata comprises an offset and a length of each of the plurality of files in the single aggregated file. The metadata can be used to unpack one or more of the files from the single aggregated file.

Inventors:
; ; ; ;
Publication Date:
Research Org.:
LANL (Los Alamos National Laboratory (LANL), Los Alamos, NM (United States))
Sponsoring Org.:
USDOE
OSTI Identifier:
1151820
Patent Number(s):
8,825,652
Application Number:
13/536,315
Assignee:
EMC Corporation (Hopkinton, MA); Los Alamos National Security, LLC (Los Alamos, NM) LANL
DOE Contract Number:
AC52-06NA25396
Resource Type:
Patent
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING

Citation Formats

Faibish, Sorin, Bent, John M., Tzelnic, Percy, Grider, Gary, and Zhang, Jingwang. Small file aggregation in a parallel computing system. United States: N. p., 2014. Web.
Faibish, Sorin, Bent, John M., Tzelnic, Percy, Grider, Gary, & Zhang, Jingwang. Small file aggregation in a parallel computing system. United States.
Faibish, Sorin, Bent, John M., Tzelnic, Percy, Grider, Gary, and Zhang, Jingwang. Tue . "Small file aggregation in a parallel computing system". United States. doi:. https://www.osti.gov/servlets/purl/1151820.
@article{osti_1151820,
title = {Small file aggregation in a parallel computing system},
author = {Faibish, Sorin and Bent, John M. and Tzelnic, Percy and Grider, Gary and Zhang, Jingwang},
abstractNote = {Techniques are provided for small file aggregation in a parallel computing system. An exemplary method for storing a plurality of files generated by a plurality of processes in a parallel computing system comprises aggregating the plurality of files into a single aggregated file; and generating metadata for the single aggregated file. The metadata comprises an offset and a length of each of the plurality of files in the single aggregated file. The metadata can be used to unpack one or more of the files from the single aggregated file.},
doi = {},
journal = {},
number = ,
volume = ,
place = {United States},
year = {Tue Sep 02 00:00:00 EDT 2014},
month = {Tue Sep 02 00:00:00 EDT 2014}
}

Patent:

Save / Share: