Methods and apparatus for capture and storage of semantic information with sub-files in a parallel computing system
Abstract
Techniques are provided for storing files in a parallel computing system using sub-files with semantically meaningful boundaries. A method is provided for storing at least one file generated by a distributed application in a parallel computing system. The file comprises one or more of a complete file and a plurality of sub-files. The method comprises the steps of obtaining a user specification of semantic information related to the file; providing the semantic information as a data structure description to a data formatting library write function; and storing the semantic information related to the file with one or more of the sub-files in one or more storage nodes of the parallel computing system. The semantic information provides a description of data in the file. The sub-files can be replicated based on semantically meaningful boundaries.
- Inventors:
- Issue Date:
- Research Org.:
- Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
- Sponsoring Org.:
- USDOE
- OSTI Identifier:
- 1169058
- Patent Number(s):
- 8949255
- Application Number:
- 13/536,384
- Assignee:
- EMC Corporation (Hopkinton, MA)
- Patent Classifications (CPCs):
-
G - PHYSICS G06 - COMPUTING G06F - ELECTRIC DIGITAL DATA PROCESSING
- DOE Contract Number:
- AC52-06NA25396
- Resource Type:
- Patent
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 97 MATHEMATICS AND COMPUTING
Citation Formats
Faibish, Sorin, Bent, John M, Tzelnic, Percy, Grider, Gary, and Torres, Aaron. Methods and apparatus for capture and storage of semantic information with sub-files in a parallel computing system. United States: N. p., 2015.
Web.
Faibish, Sorin, Bent, John M, Tzelnic, Percy, Grider, Gary, & Torres, Aaron. Methods and apparatus for capture and storage of semantic information with sub-files in a parallel computing system. United States.
Faibish, Sorin, Bent, John M, Tzelnic, Percy, Grider, Gary, and Torres, Aaron. Tue .
"Methods and apparatus for capture and storage of semantic information with sub-files in a parallel computing system". United States. https://www.osti.gov/servlets/purl/1169058.
@article{osti_1169058,
title = {Methods and apparatus for capture and storage of semantic information with sub-files in a parallel computing system},
author = {Faibish, Sorin and Bent, John M and Tzelnic, Percy and Grider, Gary and Torres, Aaron},
abstractNote = {Techniques are provided for storing files in a parallel computing system using sub-files with semantically meaningful boundaries. A method is provided for storing at least one file generated by a distributed application in a parallel computing system. The file comprises one or more of a complete file and a plurality of sub-files. The method comprises the steps of obtaining a user specification of semantic information related to the file; providing the semantic information as a data structure description to a data formatting library write function; and storing the semantic information related to the file with one or more of the sub-files in one or more storage nodes of the parallel computing system. The semantic information provides a description of data in the file. The sub-files can be replicated based on semantically meaningful boundaries.},
doi = {},
journal = {},
number = ,
volume = ,
place = {United States},
year = {Tue Feb 03 00:00:00 EST 2015},
month = {Tue Feb 03 00:00:00 EST 2015}
}
Works referenced in this record:
Systems and methods for managing portions of files in multi-tier storage systems
patent, January 2013
- Mamidi, Murthy V.; Malige, Raghupathi; Ravi, Gautham
- US Patent Document 8,352,429
Rule based aggregation of files and transactions in a switched file system
patent-application, July 2004
- Miloushev, Vladimir; Nickolov, Peter
- US Patent Application 10/336832; 20040133577
PLFS: a checkpoint filesystem for parallel applications
conference, January 2009
- Bent, John; Gibson, Garth; Grider, Gary
Works referencing / citing this record:
Methods and apparatus for capture and storage of semantic information with sub-files in a parallel computing system
patent, February 2015
- Faibish, Sorin; Bent, John M.; Tzelnic, Percy
- US Patent Document 8,949,255