Grand unified file indexing
Abstract
Systems and methods are disclosed for a unified file index for a file system. In one example, a Grand Unified File Index (GUFI) includes a tree replicating the directory hierarchy of one or more primary filesystems, and individual metadata stores for each directory. The GUFI tree permits fast traversal, efficient user space access controls, and simple tree directed operations such as renames, moves, or permission changes. In some examples, the individual metadata stores can be implemented as embedded databases on flash storage for speed. In some examples, use of summary tables at the directory or subtree level can eliminate wasteful executions, prune tree traversal, and further improve performance. In various examples, efficient operation can be achieved from laptop to supercomputer scale, across a wide mix of file distributions and filesystems.
- Inventors:
- Issue Date:
- Research Org.:
- Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
- Sponsoring Org.:
- USDOE National Nuclear Security Administration (NNSA)
- OSTI Identifier:
- 1824127
- Patent Number(s):
- 11048699
- Application Number:
- 16/353,882
- Assignee:
- Triad National Security, LLC (Los Alamos, NM)
- Patent Classifications (CPCs):
-
G - PHYSICS G06 - COMPUTING G06F - ELECTRIC DIGITAL DATA PROCESSING
- DOE Contract Number:
- 89233218CNA000001
- Resource Type:
- Patent
- Resource Relation:
- Patent File Date: 03/14/2019
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 97 MATHEMATICS AND COMPUTING
Citation Formats
Grider, Gary A., Manno, Dominic Anthony, Poole, Wendy Kinton, Bonnie, David J., and Inman, Jeffrey Thornton. Grand unified file indexing. United States: N. p., 2021.
Web.
Grider, Gary A., Manno, Dominic Anthony, Poole, Wendy Kinton, Bonnie, David J., & Inman, Jeffrey Thornton. Grand unified file indexing. United States.
Grider, Gary A., Manno, Dominic Anthony, Poole, Wendy Kinton, Bonnie, David J., and Inman, Jeffrey Thornton. Tue .
"Grand unified file indexing". United States. https://www.osti.gov/servlets/purl/1824127.
@article{osti_1824127,
title = {Grand unified file indexing},
author = {Grider, Gary A. and Manno, Dominic Anthony and Poole, Wendy Kinton and Bonnie, David J. and Inman, Jeffrey Thornton},
abstractNote = {Systems and methods are disclosed for a unified file index for a file system. In one example, a Grand Unified File Index (GUFI) includes a tree replicating the directory hierarchy of one or more primary filesystems, and individual metadata stores for each directory. The GUFI tree permits fast traversal, efficient user space access controls, and simple tree directed operations such as renames, moves, or permission changes. In some examples, the individual metadata stores can be implemented as embedded databases on flash storage for speed. In some examples, use of summary tables at the directory or subtree level can eliminate wasteful executions, prune tree traversal, and further improve performance. In various examples, efficient operation can be achieved from laptop to supercomputer scale, across a wide mix of file distributions and filesystems.},
doi = {},
journal = {},
number = ,
volume = ,
place = {United States},
year = {2021},
month = {6}
}
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
Partitioned key-value store with atomic memory operations
patent, February 2017
- Bent, John M.; Faibish, Sorin; Grider, Gary
- US Patent Document 9,563,426
Storage of sparse files using parallel log-structured file system
patent, November 2017
- Bent, John M.; Faibish, Sorin; Grider, Gary
- US Patent Document 9,811,545
Storing files in a parallel computing system using list-based index to identify replica files
patent, July 2015
- Faibish, Sorin; Bent, John M.; Tzelnic, Percy
- US Patent Document 9,087,075
Computing erasure metadata and data layout prior to storage using a processing platform
patent, November 2018
- Bent, John M.; Faibish, Sorin; Ting, Dennis
- US Patent Document 10,135,924
Architecture and method for a burst buffer using flash technology
patent, March 2016
- Tzelnic, Percy; Faibish, Sorin; Gupta, Uday
- US Patent Document 9,286,261
File system management with enhanced searchability
patent, April 2013
- Gillett, Jr., Richard B.; Berger, Michael; Nicklin, Jonathan C.
- US Patent Document 8,417,746
Storing files in a parallel computing system based on user-specified parser function
patent, October 2014
- Faibish, Sorin; Bent, John M.; Tzelnic, Percy
- US Patent Document 8,868,576
Parallel log structured file system collective buffering to achieve a compact representation of scientific and/or dimensional data
patent, September 2015
- Grider, Gary; Poole, Stephen W.
- US Patent Document 9,122,700
Small file aggregation in a parallel computing system
patent, September 2014
- Faibish, Sorin; Bent, John M.; Tzelnic, Percy
- US Patent Document 8,825,652
Parallel File System Processing
patent-application, August 2011
- Hughes, Jeffrey Douglas; Smith, Sean MacQueen; Petersen, Kalen Albert
- US Patent Application 12/704285; 20110196899
Distributed metadata servers in a file system with separate metadata servers for file metadata and directory metadata
patent, November 2018
- Bent, John M.; Faibish, Sorin; Grider, Gary
- US Patent Document 10,140,304
Key-value store with internal key-value storage interface
patent, January 2018
- Bent, John M.; Faibish, Sorin; Ting, Dennis P. J.
- US Patent Document 9,870,168
Method and apparatus for offloading compute resources to a flash co-processing appliance
patent, October 2015
- Tzelnic, Percy; Faibish, Sorin; Gupta, Uday
- US Patent Document 9,158,540
Filesystem Hierarchical Capacity Quantity and Aggregate Metrics
patent-application, December 2016
- Passey, Aaron; O'Neill, Kevin; Godman, Peter
- US Patent Application 14/859114; 20160371296
Methods and apparatus for capture and storage of semantic information with sub-files in a parallel computing system
patent, February 2015
- Faibish, Sorin; Bent, John M.; Tzelnic, Percy
- US Patent Document 8,949,255
Distributed metadata servers for cluster file systems using shared low latency persistent key-value metadata store
patent, December 2017
- Bent, John M.; Faibish, Sorin; Pedone, Jr., James M.
- US Patent Document 9,852,146
Parallel sort with a ranged, partitioned key-value store in a high perfomance computing environment
patent, January 2016
- Bent, John M.; Faibish, Sorin; Grider, Gary
- US Patent Document 9,245,048
Parallel file system with metadata distributed across partitioned key-value store
patent, September 2017
- Bent, John M.; Faibish, Sorin; Grider, Gary
- US Patent Document 9,767,107