Global to push GA events into
skip to main content

Title: Methods and apparatus for capture and storage of semantic information with sub-files in a parallel computing system

Techniques are provided for storing files in a parallel computing system using sub-files with semantically meaningful boundaries. A method is provided for storing at least one file generated by a distributed application in a parallel computing system. The file comprises one or more of a complete file and a plurality of sub-files. The method comprises the steps of obtaining a user specification of semantic information related to the file; providing the semantic information as a data structure description to a data formatting library write function; and storing the semantic information related to the file with one or more of the sub-files in one or more storage nodes of the parallel computing system. The semantic information provides a description of data in the file. The sub-files can be replicated based on semantically meaningful boundaries.
Inventors:
; ; ; ;
Issue Date:
OSTI Identifier:
1169058
Assignee:
EMC Corporation (Hopkinton, MA) LANL
Patent Number(s):
8,949,255
Application Number:
13/536,384
Contract Number:
AC52-06NA25396
Research Org:
Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
Sponsoring Org:
USDOE
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING

Works referenced in this record:

PLFS: a checkpoint filesystem for parallel applications
conference, January 2009