Small file aggregation in a parallel computing system
Abstract
Techniques are provided for small file aggregation in a parallel computing system. An exemplary method for storing a plurality of files generated by a plurality of processes in a parallel computing system comprises aggregating the plurality of files into a single aggregated file; and generating metadata for the single aggregated file. The metadata comprises an offset and a length of each of the plurality of files in the single aggregated file. The metadata can be used to unpack one or more of the files from the single aggregated file.
- Inventors:
- Issue Date:
- Research Org.:
- Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
- Sponsoring Org.:
- USDOE
- OSTI Identifier:
- 1151820
- Patent Number(s):
- 8825652
- Application Number:
- 13/536,315
- Assignee:
- EMC Corporation (Hopkinton, MA); Los Alamos National Security, LLC (Los Alamos, NM)
- Patent Classifications (CPCs):
-
G - PHYSICS G06 - COMPUTING G06F - ELECTRIC DIGITAL DATA PROCESSING
- DOE Contract Number:
- AC52-06NA25396
- Resource Type:
- Patent
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 97 MATHEMATICS AND COMPUTING
Citation Formats
Faibish, Sorin, Bent, John M., Tzelnic, Percy, Grider, Gary, and Zhang, Jingwang. Small file aggregation in a parallel computing system. United States: N. p., 2014.
Web.
Faibish, Sorin, Bent, John M., Tzelnic, Percy, Grider, Gary, & Zhang, Jingwang. Small file aggregation in a parallel computing system. United States.
Faibish, Sorin, Bent, John M., Tzelnic, Percy, Grider, Gary, and Zhang, Jingwang. Tue .
"Small file aggregation in a parallel computing system". United States. https://www.osti.gov/servlets/purl/1151820.
@article{osti_1151820,
title = {Small file aggregation in a parallel computing system},
author = {Faibish, Sorin and Bent, John M. and Tzelnic, Percy and Grider, Gary and Zhang, Jingwang},
abstractNote = {Techniques are provided for small file aggregation in a parallel computing system. An exemplary method for storing a plurality of files generated by a plurality of processes in a parallel computing system comprises aggregating the plurality of files into a single aggregated file; and generating metadata for the single aggregated file. The metadata comprises an offset and a length of each of the plurality of files in the single aggregated file. The metadata can be used to unpack one or more of the files from the single aggregated file.},
doi = {},
journal = {},
number = ,
volume = ,
place = {United States},
year = {2014},
month = {9}
}
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
Rule based aggregation of files and transactions in a switched file system
patent-application, July 2004
- Miloushev, Vladimir; Nickolov, Peter
- US Patent Application 10/336832; 20040133577
Maintaining an aggregate including active files in a storage pool
patent-application, February 2007
- Cannon, David Maxwell; Martin, Howard Newton
- US Patent Application 11/206488; 20070043785
Maintaining active-only copy storage pools
patent-application, February 2007
- Cannon, David Maxwell; Martin, Howard Newton
- US Patent Application 11/224852; 20070043789
PLFS: a checkpoint filesystem for parallel applications
conference, January 2009
- Bent, John; Gibson, Garth; Grider, Gary