Storing files in a parallel computing system using list-based index to identify replica files
Improved techniques are provided for storing files in a parallel computing system using a list-based index to identify file replicas. A file and at least one replica of the file are stored in one or more storage nodes of the parallel computing system. An index for the file comprises at least one list comprising a pointer to a storage location of the file and a storage location of the at least one replica of the file. The file comprises one or more of a complete file and one or more sub-files. The index may also comprise a checksum value for one or more of the file and the replica(s) of the file. The checksum value can be evaluated to validate the file and/or the file replica(s). A query can be processed using the list.
- Research Organization:
- Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- AC52-06NA25396
- Assignee:
- EMC Corporation (Hopkinton, MA)
- Patent Number(s):
- 9,087,075
- Application Number:
- 13/536,331
- OSTI ID:
- 1195932
- Country of Publication:
- United States
- Language:
- English
A Self-Organizing Storage Cluster for Parallel Data-Intensive Applications
|
conference | January 2004 |
| PLFS: a checkpoint filesystem for parallel applications | conference | January 2009 |
Similar Records
Parallel checksumming of data chunks of a shared data object using a log-structured file system
Storing files in a parallel computing system based on user or application specification