Global to push GA events into
skip to main content

Title: Storage of sparse files using parallel log-structured file system

A sparse file is stored without holes by storing a data portion of the sparse file using a parallel log-structured file system; and generating an index entry for the data portion, the index entry comprising a logical offset, physical offset and length of the data portion. The holes can be restored to the sparse file upon a reading of the sparse file. The data portion can be stored at a logical end of the sparse file. Additional storage efficiency can optionally be achieved by (i) detecting a write pattern for a plurality of the data portions and generating a single patterned index entry for the plurality of the patterned data portions; and/or (ii) storing the patterned index entries for a plurality of the sparse files in a single directory, wherein each entry in the single directory comprises an identifier of a corresponding sparse file.
; ; ;
Issue Date:
OSTI Identifier:
EMC IP Holding Company LLC LANL
Patent Number(s):
Application Number:
Contract Number:
Resource Relation:
Patent File Date: 2013 Jun 19
Research Org:
Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
Sponsoring Org:
Country of Publication:
United States

Works referenced in this record:

PLFS: a checkpoint filesystem for parallel applications
conference, January 2009

Storage challenges at Los Alamos National Lab
conference, April 2012

A Plugin for HDF5 Using PLFS for Improved I/O Performance and Semantic Analysis
conference, November 2012

Understanding and Improving Computational Science Storage Access through Continuous Characterization
journal, October 2011
  • Carns, Philip; Harms, Kevin; Allcock, William
  • ACM Transactions on Storage, Vol. 7, Issue 3, p. 1-26
  • DOI: 10.1145/2027066.2027068

Lessons from characterizing the input/output behavior of parallel scientific applications
journal, June 1998

Pattern-aware file reorganization in MPI-IO
conference, January 2011

Learning to classify parallel input/output access patterns
journal, August 2002
  • Madhyastha, T. M.; Reed, D. A.
  • IEEE Transactions on Parallel and Distributed Systems, Vol. 13, Issue 8, p. 802-813
  • DOI: 10.1109/TPDS.2002.1028437

Markov model prediction of I/O requests for scientific applications
conference, January 2002

Automatic arima time series modeling for adaptive I/O prefetching
journal, April 2004
  • Tran, N.; Reed, D. A.
  • IEEE Transactions on Parallel and Distributed Systems, Vol. 15, Issue 4, p. 362-377
  • DOI: 10.1109/TPDS.2004.1271185

Discovering Structure in Unstructured I/O
conference, November 2012