skip to main content
DOE Patents title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Storage of sparse files using parallel log-structured file system

Abstract

A sparse file is stored without holes by storing a data portion of the sparse file using a parallel log-structured file system; and generating an index entry for the data portion, the index entry comprising a logical offset, physical offset and length of the data portion. The holes can be restored to the sparse file upon a reading of the sparse file. The data portion can be stored at a logical end of the sparse file. Additional storage efficiency can optionally be achieved by (i) detecting a write pattern for a plurality of the data portions and generating a single patterned index entry for the plurality of the patterned data portions; and/or (ii) storing the patterned index entries for a plurality of the sparse files in a single directory, wherein each entry in the single directory comprises an identifier of a corresponding sparse file.

Inventors:
; ; ;
Issue Date:
Research Org.:
Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
1407704
Patent Number(s):
9811545
Application Number:
13/921,719
Assignee:
EMC IP Holding Company LLC
Patent Classifications (CPCs):
G - PHYSICS G06 - COMPUTING G06F - ELECTRIC DIGITAL DATA PROCESSING
DOE Contract Number:  
AC52-06NA25396
Resource Type:
Patent
Resource Relation:
Patent File Date: 2013 Jun 19
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING

Citation Formats

Bent, John M., Faibish, Sorin, Grider, Gary, and Torres, Aaron. Storage of sparse files using parallel log-structured file system. United States: N. p., 2017. Web.
Bent, John M., Faibish, Sorin, Grider, Gary, & Torres, Aaron. Storage of sparse files using parallel log-structured file system. United States.
Bent, John M., Faibish, Sorin, Grider, Gary, and Torres, Aaron. Tue . "Storage of sparse files using parallel log-structured file system". United States. https://www.osti.gov/servlets/purl/1407704.
@article{osti_1407704,
title = {Storage of sparse files using parallel log-structured file system},
author = {Bent, John M. and Faibish, Sorin and Grider, Gary and Torres, Aaron},
abstractNote = {A sparse file is stored without holes by storing a data portion of the sparse file using a parallel log-structured file system; and generating an index entry for the data portion, the index entry comprising a logical offset, physical offset and length of the data portion. The holes can be restored to the sparse file upon a reading of the sparse file. The data portion can be stored at a logical end of the sparse file. Additional storage efficiency can optionally be achieved by (i) detecting a write pattern for a plurality of the data portions and generating a single patterned index entry for the plurality of the patterned data portions; and/or (ii) storing the patterned index entries for a plurality of the sparse files in a single directory, wherein each entry in the single directory comprises an identifier of a corresponding sparse file.},
doi = {},
journal = {},
number = ,
volume = ,
place = {United States},
year = {2017},
month = {11}
}

Patent:

Save / Share:

Works referenced in this record:

PLFS: a checkpoint filesystem for parallel applications
conference, January 2009


Storage challenges at Los Alamos National Lab
conference, April 2012


Understanding and Improving Computational Science Storage Access through Continuous Characterization
journal, October 2011


Lessons from characterizing the input/output behavior of parallel scientific applications
journal, June 1998


Pattern-aware file reorganization in MPI-IO
conference, January 2011


Learning to classify parallel input/output access patterns
journal, August 2002


Automatic arima time series modeling for adaptive I/O prefetching
journal, April 2004


Discovering Structure in Unstructured I/O
conference, November 2012