DOE Patents title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Scalable filesystem enumeration and metadata operations

Abstract

Systems, apparatus, and methods are disclosed for performing scalable operations in a file system, including POSIX-like file systems. Metadata entries in a namespace or directory tree are sharded across multiple file metadata servers. An enumeration operation, such as listing a directory, is parallelized across the multiple file metadata servers, while retaining standard functionality transparently to clients. Other enumeration operations include no-output operations such as changing file attributes or deleting a file, and cumulative operations such as counting total disk space usage. The parallelization is compatible with tree-level parallelization and storage-level parallelization. Disclosed technologies can be applied to other fields requiring scalable enumeration, such as database and network applications.

Inventors:
Issue Date:
Research Org.:
Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
1805635
Patent Number(s):
10949387
Application Number:
15/713,492
Assignee:
Triad National Security, LLC (Los Alamos, NM)
DOE Contract Number:  
AC52-06NA25396
Resource Type:
Patent
Resource Relation:
Patent File Date: 09/22/2017
Country of Publication:
United States
Language:
English

Citation Formats

Grider, Gary A. Scalable filesystem enumeration and metadata operations. United States: N. p., 2021. Web.
Grider, Gary A. Scalable filesystem enumeration and metadata operations. United States.
Grider, Gary A. Tue . "Scalable filesystem enumeration and metadata operations". United States. https://www.osti.gov/servlets/purl/1805635.
@article{osti_1805635,
title = {Scalable filesystem enumeration and metadata operations},
author = {Grider, Gary A.},
abstractNote = {Systems, apparatus, and methods are disclosed for performing scalable operations in a file system, including POSIX-like file systems. Metadata entries in a namespace or directory tree are sharded across multiple file metadata servers. An enumeration operation, such as listing a directory, is parallelized across the multiple file metadata servers, while retaining standard functionality transparently to clients. Other enumeration operations include no-output operations such as changing file attributes or deleting a file, and cumulative operations such as counting total disk space usage. The parallelization is compatible with tree-level parallelization and storage-level parallelization. Disclosed technologies can be applied to other fields requiring scalable enumeration, such as database and network applications.},
doi = {},
journal = {},
number = ,
volume = ,
place = {United States},
year = {2021},
month = {3}
}

Works referenced in this record:

Small file aggregation in a parallel computing system
patent, September 2014


System and methods for processing a multidimensional database
patent, October 2007


Fault tolerant multi-stage data replication with relaxed coherency guarantees
patent, January 2010


Conditional writes at distributed storage services
patent, February 2017


Cooperative storage of shared files in a parallel computing system with dynamic block size
patent, November 2015


Storing files in a parallel computing system based on user-specified parser function
patent, October 2014


Storing files in a parallel computing system based on user or application specification
patent, March 2016


Oversubscribed storage extents with on-demand page allocation
patent, October 2017


Scalable file management for a shared file system
patent, December 2012


Parallel file system with metadata distributed across partitioned key-value store
patent, September 2017


Computing erasure metadata and data layout prior to storage using a processing platform
patent, November 2018