skip to main content
DOE Patents title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Computing erasure metadata and data layout prior to storage using a processing platform

Abstract

Techniques are provided for computing data and metadata layout prior to storage in a storage system using a processing platform. An exemplary processing platform comprises one or more of a compute node and a burst buffer appliance. The processing platform communicates with a plurality of the compute nodes over a network, wherein a plurality of applications executing on the plurality of compute nodes generate a plurality of data objects; computes erasure metadata for one or more of the data objects on at least one of the compute nodes; and provides the erasure metadata with the corresponding one or more data objects to a storage system. The processing platform optionally determines a full set of the data objects to be stored and queries the storage system to determine an anticipated layout of the full set of the data objects to be stored. The anticipated layout allows special handling, for example, for small files and large files that are identified based on predefined criteria.

Inventors:
; ; ; ; ;
Issue Date:
Research Org.:
Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
1494131
Patent Number(s):
10,135,924
Application Number:
14/751,372
Assignee:
EMC IP Holding Company LLC (Hopkinton, MA); Los Alamos National Security, LLC (Los Alamos, NM) LANL
DOE Contract Number:  
AC52-06NA25396
Resource Type:
Patent
Resource Relation:
Patent File Date: 2015 Jun 26
Country of Publication:
United States
Language:
English
Subject:
96 KNOWLEDGE MANAGEMENT AND PRESERVATION

Citation Formats

Bent, John M., Faibish, Sorin, Ting, Dennis P.J., Teymouri, Sasan, Pedone, Jr., James M., and Grider, Gary. Computing erasure metadata and data layout prior to storage using a processing platform. United States: N. p., 2018. Web.
Bent, John M., Faibish, Sorin, Ting, Dennis P.J., Teymouri, Sasan, Pedone, Jr., James M., & Grider, Gary. Computing erasure metadata and data layout prior to storage using a processing platform. United States.
Bent, John M., Faibish, Sorin, Ting, Dennis P.J., Teymouri, Sasan, Pedone, Jr., James M., and Grider, Gary. Tue . "Computing erasure metadata and data layout prior to storage using a processing platform". United States. https://www.osti.gov/servlets/purl/1494131.
@article{osti_1494131,
title = {Computing erasure metadata and data layout prior to storage using a processing platform},
author = {Bent, John M. and Faibish, Sorin and Ting, Dennis P.J. and Teymouri, Sasan and Pedone, Jr., James M. and Grider, Gary},
abstractNote = {Techniques are provided for computing data and metadata layout prior to storage in a storage system using a processing platform. An exemplary processing platform comprises one or more of a compute node and a burst buffer appliance. The processing platform communicates with a plurality of the compute nodes over a network, wherein a plurality of applications executing on the plurality of compute nodes generate a plurality of data objects; computes erasure metadata for one or more of the data objects on at least one of the compute nodes; and provides the erasure metadata with the corresponding one or more data objects to a storage system. The processing platform optionally determines a full set of the data objects to be stored and queries the storage system to determine an anticipated layout of the full set of the data objects to be stored. The anticipated layout allows special handling, for example, for small files and large files that are identified based on predefined criteria.},
doi = {},
journal = {},
number = ,
volume = ,
place = {United States},
year = {2018},
month = {11}
}

Patent:

Save / Share: