skip to main content
DOE Patents title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Storing files in a parallel computing system based on user-specified parser function

Abstract

Techniques are provided for storing files in a parallel computing system based on a user-specified parser function. A plurality of files generated by a distributed application in a parallel computing system are stored by obtaining a parser from the distributed application for processing the plurality of files prior to storage; and storing one or more of the plurality of files in one or more storage nodes of the parallel computing system based on the processing by the parser. The plurality of files comprise one or more of a plurality of complete files and a plurality of sub-files. The parser can optionally store only those files that satisfy one or more semantic requirements of the parser. The parser can also extract metadata from one or more of the files and the extracted metadata can be stored with one or more of the plurality of files and used for searching for files.

Inventors:
; ; ; ; ;
Issue Date:
Research Org.:
Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
1160235
Patent Number(s):
8,868,576
Application Number:
13/536,369
Assignee:
EMC Corporation (Hopkinton, MA) LANL
DOE Contract Number:  
AC52-06NA25396
Resource Type:
Patent
Resource Relation:
Patent File Date: 2012 Jun 28
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING

Citation Formats

Faibish, Sorin, Bent, John M, Tzelnic, Percy, Grider, Gary, Manzanares, Adam, and Torres, Aaron. Storing files in a parallel computing system based on user-specified parser function. United States: N. p., 2014. Web.
Faibish, Sorin, Bent, John M, Tzelnic, Percy, Grider, Gary, Manzanares, Adam, & Torres, Aaron. Storing files in a parallel computing system based on user-specified parser function. United States.
Faibish, Sorin, Bent, John M, Tzelnic, Percy, Grider, Gary, Manzanares, Adam, and Torres, Aaron. Tue . "Storing files in a parallel computing system based on user-specified parser function". United States. https://www.osti.gov/servlets/purl/1160235.
@article{osti_1160235,
title = {Storing files in a parallel computing system based on user-specified parser function},
author = {Faibish, Sorin and Bent, John M and Tzelnic, Percy and Grider, Gary and Manzanares, Adam and Torres, Aaron},
abstractNote = {Techniques are provided for storing files in a parallel computing system based on a user-specified parser function. A plurality of files generated by a distributed application in a parallel computing system are stored by obtaining a parser from the distributed application for processing the plurality of files prior to storage; and storing one or more of the plurality of files in one or more storage nodes of the parallel computing system based on the processing by the parser. The plurality of files comprise one or more of a plurality of complete files and a plurality of sub-files. The parser can optionally store only those files that satisfy one or more semantic requirements of the parser. The parser can also extract metadata from one or more of the files and the extracted metadata can be stored with one or more of the plurality of files and used for searching for files.},
doi = {},
journal = {},
number = ,
volume = ,
place = {United States},
year = {2014},
month = {10}
}

Patent:

Save / Share:

Works referenced in this record:

PLFS: a checkpoint filesystem for parallel applications
conference, January 2009