skip to main content

SciTech ConnectSciTech Connect

Title: Small file aggregation in a parallel computing system

Techniques are provided for small file aggregation in a parallel computing system. An exemplary method for storing a plurality of files generated by a plurality of processes in a parallel computing system comprises aggregating the plurality of files into a single aggregated file; and generating metadata for the single aggregated file. The metadata comprises an offset and a length of each of the plurality of files in the single aggregated file. The metadata can be used to unpack one or more of the files from the single aggregated file.
Authors:
; ; ; ;
Publication Date:
OSTI Identifier:
1151820
Report Number(s):
8,825,652
13/536,315
DOE Contract Number:
AC52-06NA25396
Resource Type:
Patent
Research Org:
LANL (Los Alamos National Laboratory (LANL), Los Alamos, NM (United States))
Sponsoring Org:
USDOE
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING