Metadata compression

Title: Metadata compression

Abstract

A computer-executable method, system, and computer program product for managing a Parallel-Logged File System (PLFS) on a data storage system, wherein the data storage system is in communication with an application that updates the data storage system with checkpoint data, the computer-executable method, system, and computer program product comprising receiving a first checkpoint including data from the application; analyzing the data in the first checkpoint to determine whether a pattern exists in the checkpoint data; and upon a positive determination, adding the pattern into PLFS.

Inventors:: Bent, John M.; Faibish, Sorin; Zhang, Zhenhua; He, Jun; Torres, Aaron; Grider, Gary

Issue Date:: 2020-02-11

Research Org.:: EMC IP Holding Company LLC, Hopkinton, MA (United States); Triad National Security, LLC, Los Alamos, NM (United States)

Sponsoring Org.:: USDOE National Nuclear Security Administration (NNSA)

OSTI Identifier:: 1632548

Patent Number(s):: 10558618

Application Number:: 14/041,872

Assignee:: EMC IP Holding Company LLC (Hopkinton, MA); Triad National Security, LLC (Los Alamos, NM)

Patent Classifications (CPCs):: G - PHYSICS G06 - COMPUTING G06F - ELECTRIC DIGITAL DATA PROCESSING

Show more

G - PHYSICS G06 - COMPUTING G06F - ELECTRIC DIGITAL DATA PROCESSING
G06F16/1744 - {using compression, e.g. sparse files}
G06F16/13 - File access structures, e.g. distributed indices
G06F16/17 - Details of further file system functions

Show less

DOE Contract Number:: 89233218CNA000001

Resource Type:: Patent

Resource Relation:: Patent File Date: 09/30/2013

Country of Publication:: United States

Language:: English

Citation Formats


                    Bent, John M., Faibish, Sorin, Zhang, Zhenhua, He, Jun, Torres, Aaron, and Grider, Gary. Metadata compression.  United States: N. p., 2020. 
        Web.

Copy to clipboard


                    Bent, John M., Faibish, Sorin, Zhang, Zhenhua, He, Jun, Torres, Aaron, & Grider, Gary. Metadata compression.  United States.

Copy to clipboard


                    Bent, John M., Faibish, Sorin, Zhang, Zhenhua, He, Jun, Torres, Aaron, and Grider, Gary. Tue .  
        "Metadata compression".  United States.  https://www.osti.gov/servlets/purl/1632548.

Copy to clipboard


                    
@article{osti_1632548,

  title        = {Metadata compression},

  author       = {Bent, John M. and Faibish, Sorin and Zhang, Zhenhua and He, Jun and Torres, Aaron and Grider, Gary},

  abstractNote = {A computer-executable method, system, and computer program product for managing a Parallel-Logged File System (PLFS) on a data storage system, wherein the data storage system is in communication with an application that updates the data storage system with checkpoint data, the computer-executable method, system, and computer program product comprising receiving a first checkpoint including data from the application; analyzing the data in the first checkpoint to determine whether a pattern exists in the checkpoint data; and upon a positive determination, adding the pattern into PLFS.},

  doi          = {},

  journal      = {},
number       = ,

  volume       = ,

  place        = {United States},

  year         = {2020},

  month        = {2}

}

Copy to clipboard

Patent:

View Patent

Save / Share:

Export Metadata

Save to My Library

Works referenced in this record:

Architecture and method for a burst buffer using flash technology
patent, March 2016

Tzelnic, Percy; Faibish, Sorin; Gupta, Uday K.
US Patent Document 9,286,261
URL: http://patft.uspto.gov/netacgi/nph-Parser?patentnumber=9286261

Data compression method and apparatus
patent, December 1998

Wightman, Andy T.
US Patent Document 5,850,565
URL: http://patft.uspto.gov/netacgi/nph-Parser?patentnumber=5850565

Similar Records in DOE Patents and OSTI.GOV collections:

Data compression with inline compression metadata

Patent Sadowski, Greg

Techniques for handling data compression in which metadata that indicates which portions of data are compressed are which portions of data are not compressed are disclosed. Segments of a buffer referred to as block groups store compressed blocks of data along with uncompressed blocks of data and hash blocks. If a block group includes a block that is a hash of another block in the block group, then the other block is considered to be compressed. If the block group does not include a block that is a hash of another block in the block group, then the blocks inmore »« less
Full Text Available
Method and system for semantic metadata compression in a two-tier storage system using copy-on-write

Patent Bent, John M.; Faibish, Sorin; Grider, Gary; ...

Example embodiments of the present invention relate to a method, a system, and a computer program product for managing metadata. The method includes determining a common characteristic of metadata entries for a plurality of files in a directory of a file system and computing a representative metadata entry for the plurality of metadata entries according to the common characteristic of the metadata entries. The plurality of metadata entries in the file system then may be represented according to the computed representative metadata entry.
Full Text Available
Distributed metadata servers in a file system with separate metadata servers for file metadata and directory metadata

Patent Bent, John M.; Faibish, Sorin; Grider, Gary

A file system is provided having a plurality of distributed metadata servers with separate metadata servers for file metadata and directory metadata. An exemplary distributed metadata storage system comprises a plurality of compute nodes, wherein at least one compute node comprises a file system client that generates a directory metadata request and/or a file metadata request; a plurality of directory metadata servers to serve directory metadata in response to a given directory metadata request; and a plurality of file metadata servers to serve file metadata in response to a given file metadata request. The plurality of directory metadata servers aremore »« less
Full Text Available
Distributed metadata servers for cluster file systems using shared low latency persistent key-value metadata store

Patent Bent, John M.; Faibish, Sorin; Pedone, Jr., James M.; ...

A cluster file system is provided having a plurality of distributed metadata servers with shared access to one or more shared low latency persistent key-value metadata stores. A metadata server comprises an abstract storage interface comprising a software interface module that communicates with at least one shared persistent key-value metadata store providing a key-value interface for persistent storage of key-value metadata. The software interface module provides the key-value metadata to the at least one shared persistent key-value metadata store in a key-value format. The shared persistent key-value metadata store is accessed by a plurality of metadata servers. A metadata requestmore »« less
Full Text Available
Feature identification or classification using task-specific metadata

Patent Teuton, Jeremy R.; Griswold, Richard L.

Innovations in the identification or classification of features in a data set are described, such as a data set representing measurements taken by a scientific instrument. For example, a task-specific processing component, such as a video encoder, is used to generate task-specific metadata. When the data set includes video frames, metadata can include information regarding motion of image elements between frames, or other differences between frames. A feature of the data set, such as an event, can be identified or classified based on the metadata. For example, an event can be identified when metadata for one or more elements ofmore »« less
Full Text Available

Similar Records

Title: Metadata compression

Abstract

Citation Formats

Architecture and method for a burst buffer using flash technology patent, March 2016

Data compression method and apparatus patent, December 1998

Architecture and method for a burst buffer using flash technology
patent, March 2016

Data compression method and apparatus
patent, December 1998