skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Parallel checksumming of data chunks of a shared data object using a log-structured file system

Abstract

Checksum values are generated and used to verify the data integrity. A client executing in a parallel computing system stores a data chunk to a shared data object on a storage node in the parallel computing system. The client determines a checksum value for the data chunk; and provides the checksum value with the data chunk to the storage node that stores the shared object. The data chunk can be stored on the storage node with the corresponding checksum value as part of the shared object. The storage node may be part of a Parallel Log-Structured File System (PLFS), and the client may comprise, for example, a Log-Structured File System client on a compute node or burst buffer. The checksum value can be evaluated when the data chunk is read from the storage node to verify the integrity of the data that is read.

Inventors:
; ;
Publication Date:
Research Org.:
Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
1320885
Patent Number(s):
9,436,722
Application Number:
13/799,264
Assignee:
Los Alamos National Security, LLC (Los Alamos, NM)
DOE Contract Number:  
AC52-06NA25396
Resource Type:
Patent
Resource Relation:
Patent File Date: 2013 Mar 13
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING

Citation Formats

Bent, John M., Faibish, Sorin, and Grider, Gary. Parallel checksumming of data chunks of a shared data object using a log-structured file system. United States: N. p., 2016. Web.
Bent, John M., Faibish, Sorin, & Grider, Gary. Parallel checksumming of data chunks of a shared data object using a log-structured file system. United States.
Bent, John M., Faibish, Sorin, and Grider, Gary. 2016. "Parallel checksumming of data chunks of a shared data object using a log-structured file system". United States. https://www.osti.gov/servlets/purl/1320885.
@article{osti_1320885,
title = {Parallel checksumming of data chunks of a shared data object using a log-structured file system},
author = {Bent, John M. and Faibish, Sorin and Grider, Gary},
abstractNote = {Checksum values are generated and used to verify the data integrity. A client executing in a parallel computing system stores a data chunk to a shared data object on a storage node in the parallel computing system. The client determines a checksum value for the data chunk; and provides the checksum value with the data chunk to the storage node that stores the shared object. The data chunk can be stored on the storage node with the corresponding checksum value as part of the shared object. The storage node may be part of a Parallel Log-Structured File System (PLFS), and the client may comprise, for example, a Log-Structured File System client on a compute node or burst buffer. The checksum value can be evaluated when the data chunk is read from the storage node to verify the integrity of the data that is read.},
doi = {},
url = {https://www.osti.gov/biblio/1320885}, journal = {},
number = ,
volume = ,
place = {United States},
year = {Tue Sep 06 00:00:00 EDT 2016},
month = {Tue Sep 06 00:00:00 EDT 2016}
}

Works referenced in this record:

Block-appended checksums
patent, October 2005


Detecting read/write conflicts
patent, October 2014