skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Cooperative storage of shared files in a parallel computing system with dynamic block size

Abstract

Improved techniques are provided for parallel writing of data to a shared object in a parallel computing system. A method is provided for storing data generated by a plurality of parallel processes to a shared object in a parallel computing system. The method is performed by at least one of the processes and comprises: dynamically determining a block size for storing the data; exchanging a determined amount of the data with at least one additional process to achieve a block of the data having the dynamically determined block size; and writing the block of the data having the dynamically determined block size to a file system. The determined block size comprises, e.g., a total amount of the data to be stored divided by the number of parallel processes. The file system comprises, for example, a log structured virtual parallel file system, such as a Parallel Log-Structured File System (PLFS).

Inventors:
; ;
Publication Date:
Research Org.:
Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
1225724
Patent Number(s):
9,183,211
Application Number:
13/730,080
Assignee:
Los Alamos National Security, LLC LANL
DOE Contract Number:  
AC52-06NA25396
Resource Type:
Patent
Resource Relation:
Patent File Date: 2012 Dec 28
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING

Citation Formats

Bent, John M., Faibish, Sorin, and Grider, Gary. Cooperative storage of shared files in a parallel computing system with dynamic block size. United States: N. p., 2015. Web.
Bent, John M., Faibish, Sorin, & Grider, Gary. Cooperative storage of shared files in a parallel computing system with dynamic block size. United States.
Bent, John M., Faibish, Sorin, and Grider, Gary. Tue . "Cooperative storage of shared files in a parallel computing system with dynamic block size". United States. doi:. https://www.osti.gov/servlets/purl/1225724.
@article{osti_1225724,
title = {Cooperative storage of shared files in a parallel computing system with dynamic block size},
author = {Bent, John M. and Faibish, Sorin and Grider, Gary},
abstractNote = {Improved techniques are provided for parallel writing of data to a shared object in a parallel computing system. A method is provided for storing data generated by a plurality of parallel processes to a shared object in a parallel computing system. The method is performed by at least one of the processes and comprises: dynamically determining a block size for storing the data; exchanging a determined amount of the data with at least one additional process to achieve a block of the data having the dynamically determined block size; and writing the block of the data having the dynamically determined block size to a file system. The determined block size comprises, e.g., a total amount of the data to be stored divided by the number of parallel processes. The file system comprises, for example, a log structured virtual parallel file system, such as a Parallel Log-Structured File System (PLFS).},
doi = {},
journal = {},
number = ,
volume = ,
place = {United States},
year = {Tue Nov 10 00:00:00 EST 2015},
month = {Tue Nov 10 00:00:00 EST 2015}
}

Patent:

Save / Share:

Works referenced in this record:

PLFS: a checkpoint filesystem for parallel applications
conference, January 2009