Small file aggregation in a parallel computing system
Patent
·
OSTI ID:1151820
Techniques are provided for small file aggregation in a parallel computing system. An exemplary method for storing a plurality of files generated by a plurality of processes in a parallel computing system comprises aggregating the plurality of files into a single aggregated file; and generating metadata for the single aggregated file. The metadata comprises an offset and a length of each of the plurality of files in the single aggregated file. The metadata can be used to unpack one or more of the files from the single aggregated file.
- Research Organization:
- Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- AC52-06NA25396
- Assignee:
- EMC Corporation (Hopkinton, MA); Los Alamos National Security, LLC (Los Alamos, NM)
- Patent Number(s):
- 8,825,652
- Application Number:
- 13/536,315
- OSTI ID:
- 1151820
- Country of Publication:
- United States
- Language:
- English
Storing data files in a file system
|
patent | May 2016 |
Architecture and method for a burst buffer using flash technology
|
patent | March 2016 |
Burst buffer appliance with small file aggregation
|
patent | March 2015 |
Similar Records
Storing files in a parallel computing system based on user-specified parser function
Storage of sparse files using parallel log-structured file system
Distributed metadata servers in a file system with separate metadata servers for file metadata and directory metadata
Patent
·
Tue Oct 21 00:00:00 EDT 2014
·
OSTI ID:1151820
+3 more
Storage of sparse files using parallel log-structured file system
Patent
·
Tue Nov 07 00:00:00 EST 2017
·
OSTI ID:1151820
+1 more
Distributed metadata servers in a file system with separate metadata servers for file metadata and directory metadata
Patent
·
Tue Nov 27 00:00:00 EST 2018
·
OSTI ID:1151820