Parallel file system with metadata distributed across partitioned key-value store c
Patent
·
OSTI ID:1393429
Improved techniques are provided for storing metadata associated with a plurality of sub-files associated with a single shared file in a parallel file system. The shared file is generated by a plurality of applications executing on a plurality of compute nodes. A compute node implements a Parallel Log Structured File System (PLFS) library to store at least one portion of the shared file generated by an application executing on the compute node and metadata for the at least one portion of the shared file on one or more object storage servers. The compute node is also configured to implement a partitioned data store for storing a partition of the metadata for the shared file, wherein the partitioned data store communicates with partitioned data stores on other compute nodes using a message passing interface. The partitioned data store can be implemented, for example, using Multidimensional Data Hashing Indexing Middleware (MDHIM).
- Research Organization:
- Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- AC52-06NA25396
- Assignee:
- EMC IP Holding Company LLC
- Patent Number(s):
- 9,767,107
- Application Number:
- 13/931,832
- OSTI ID:
- 1393429
- Country of Publication:
- United States
- Language:
- English
Parallel File System Analysis Through Application I/O Tracing
|
journal | May 2012 |
Similar Records
Distributed metadata servers in a file system with separate metadata servers for file metadata and directory metadata
Distributed metadata servers for cluster file systems using shared low latency persistent key-value metadata store
Cloud object store for archive storage of high performance computing data using decoupling middleware
Patent
·
Mon Nov 26 23:00:00 EST 2018
·
OSTI ID:1495211
Distributed metadata servers for cluster file systems using shared low latency persistent key-value metadata store
Patent
·
Mon Dec 25 23:00:00 EST 2017
·
OSTI ID:1414930
Cloud object store for archive storage of high performance computing data using decoupling middleware
Patent
·
Tue Jun 30 00:00:00 EDT 2015
·
OSTI ID:1187939