Storing files in a parallel computing system based on user or application specification
Techniques are provided for storing files in a parallel computing system based on a user-specification. A plurality of files generated by a distributed application in a parallel computing system are stored by obtaining a specification from the distributed application indicating how the plurality of files should be stored; and storing one or more of the plurality of files in one or more storage nodes of a multi-tier storage system based on the specification. The plurality of files comprise a plurality of complete files and/or a plurality of sub-files. The specification can optionally be processed by a daemon executing on one or more nodes in a multi-tier storage system. The specification indicates how the plurality of files should be stored, for example, identifying one or more storage nodes where the plurality of files should be stored.
- Research Organization:
- Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- AC52-06NA25396
- Assignee:
- EMC Corporation (Hopkinton, MA) Los Alamos National Security, LLC (Los Alamos, NM)
- Patent Number(s):
- 9,298,733
- Application Number:
- 13/536,289
- OSTI ID:
- 1244229
- Country of Publication:
- United States
- Language:
- English
| PLFS: a checkpoint filesystem for parallel applications | conference | January 2009 |
Similar Records
Parallel file system with metadata distributed across partitioned key-value store c
Methods and apparatus for capture and storage of semantic information with sub-files in a parallel computing system