Cooperative storage of shared files in a parallel computing system with dynamic block size
Patent
·
OSTI ID:1225724
Improved techniques are provided for parallel writing of data to a shared object in a parallel computing system. A method is provided for storing data generated by a plurality of parallel processes to a shared object in a parallel computing system. The method is performed by at least one of the processes and comprises: dynamically determining a block size for storing the data; exchanging a determined amount of the data with at least one additional process to achieve a block of the data having the dynamically determined block size; and writing the block of the data having the dynamically determined block size to a file system. The determined block size comprises, e.g., a total amount of the data to be stored divided by the number of parallel processes. The file system comprises, for example, a log structured virtual parallel file system, such as a Parallel Log-Structured File System (PLFS).
- Research Organization:
- Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- AC52-06NA25396
- Assignee:
- Los Alamos National Security, LLC
- Patent Number(s):
- 9,183,211
- Application Number:
- 13/730,080
- OSTI ID:
- 1225724
- Country of Publication:
- United States
- Language:
- English
| PLFS: a checkpoint filesystem for parallel applications | conference | January 2009 |
Similar Records
Parallel file system with metadata distributed across partitioned key-value store
c
Parallel checksumming of data chunks of a shared data object using a log-structured file system
Global distributed file append using log-structured file system
Patent
·
2017
·
OSTI ID:1393429
Parallel checksumming of data chunks of a shared data object using a log-structured file system
Patent
·
2016
·
OSTI ID:1320885
Global distributed file append using log-structured file system
Patent
·
2019
·
OSTI ID:1568208