Global to push GA events into
skip to main content

Title: Parallel file system with metadata distributed across partitioned key-value store c

Improved techniques are provided for storing metadata associated with a plurality of sub-files associated with a single shared file in a parallel file system. The shared file is generated by a plurality of applications executing on a plurality of compute nodes. A compute node implements a Parallel Log Structured File System (PLFS) library to store at least one portion of the shared file generated by an application executing on the compute node and metadata for the at least one portion of the shared file on one or more object storage servers. The compute node is also configured to implement a partitioned data store for storing a partition of the metadata for the shared file, wherein the partitioned data store communicates with partitioned data stores on other compute nodes using a message passing interface. The partitioned data store can be implemented, for example, using Multidimensional Data Hashing Indexing Middleware (MDHIM).
Inventors:
; ; ;
Issue Date:
OSTI Identifier:
1393429
Assignee:
EMC IP Holding Company LLC LANL
Patent Number(s):
9,767,107
Application Number:
13/931,832
Contract Number:
AC52-06NA25396
Resource Relation:
Patent File Date: 2013 Jun 29
Research Org:
Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
Sponsoring Org:
USDOE
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING

Other works cited in this record:

System method and computer program product for dynamically sizing hash tables
patent, September 1999

File I/O control method
patent-application, July 2002

Method of controlling access to computing resource within shared computing environment
patent, April 2006

Distributed file system and method of operating a distributed file system
patent-application, May 2006

Method for providing high performance scalable file I/O through persistent file domain and functional partitioning
patent-application, May 2008

Switch-Aware Parallel File System
patent-application, December 2011

Methods for optimizing data movement in solid state devices
patent-application, June 2013

Parallel File System Analysis Through Application I/O Tracing
journal, May 2012
  • Wright, S. A.; Hammond, S. D.; Pennycook, S. J.
  • The Computer Journal, Vol. 56, Issue 2, p. 141-155
  • DOI: 10.1093/comjnl/bxs044

Similar records in DOepatents and OSTI.GOV collections: