skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Using MPI File Caching to Improve Parallel Write Performance for Large-Scale Scientific Applications

Abstract

Typical large-scale scientific applications periodically write checkpoint files to save the computational state throughout execution. Existing parallel file systems improve such write-only I/O patterns through the use of clientside file caching and write-behind strategies. In distributed environments where files are rarely accessed by more than one client concurrently, file caching has achieved significant success; however, in parallel applications where multiple clients manipulate a shared file, cache coherence control can serialize I/O. We have designed a thread based caching layer for the MPI I/O library, which adds a portable caching system closer to user applications so more information about the application's I/O patterns is available for better coherence control. We demonstrate the impact of our caching solution on parallel write performance with a comprehensive evaluation that includes a set of widely used I/O benchmarks and production application I/O kernels.

Authors:
 [1];  [1];  [1];  [1];  [1];  [2];  [3];  [3]
  1. Northwestern University, Evanston
  2. Sandia National Laboratories (SNL)
  3. ORNL
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Sponsoring Org.:
USDOE Office of Science (SC)
OSTI Identifier:
1054965
DOE Contract Number:
AC05-00OR22725
Resource Type:
Conference
Resource Relation:
Conference: SC07, Reno, NV, USA, 20071110, 20071116
Country of Publication:
United States
Language:
English

Citation Formats

Liao, Wei-keng, Ching, Avery, Coloma, Kenin, Nisar, Arifa, Choudhary, Alok, Chen, Jacqueline H, Sankaran, Ramanan, and Klasky, Scott A. Using MPI File Caching to Improve Parallel Write Performance for Large-Scale Scientific Applications. United States: N. p., 2007. Web. doi:10.1145/1362622.1362634.
Liao, Wei-keng, Ching, Avery, Coloma, Kenin, Nisar, Arifa, Choudhary, Alok, Chen, Jacqueline H, Sankaran, Ramanan, & Klasky, Scott A. Using MPI File Caching to Improve Parallel Write Performance for Large-Scale Scientific Applications. United States. doi:10.1145/1362622.1362634.
Liao, Wei-keng, Ching, Avery, Coloma, Kenin, Nisar, Arifa, Choudhary, Alok, Chen, Jacqueline H, Sankaran, Ramanan, and Klasky, Scott A. Mon . "Using MPI File Caching to Improve Parallel Write Performance for Large-Scale Scientific Applications". United States. doi:10.1145/1362622.1362634.
@article{osti_1054965,
title = {Using MPI File Caching to Improve Parallel Write Performance for Large-Scale Scientific Applications},
author = {Liao, Wei-keng and Ching, Avery and Coloma, Kenin and Nisar, Arifa and Choudhary, Alok and Chen, Jacqueline H and Sankaran, Ramanan and Klasky, Scott A},
abstractNote = {Typical large-scale scientific applications periodically write checkpoint files to save the computational state throughout execution. Existing parallel file systems improve such write-only I/O patterns through the use of clientside file caching and write-behind strategies. In distributed environments where files are rarely accessed by more than one client concurrently, file caching has achieved significant success; however, in parallel applications where multiple clients manipulate a shared file, cache coherence control can serialize I/O. We have designed a thread based caching layer for the MPI I/O library, which adds a portable caching system closer to user applications so more information about the application's I/O patterns is available for better coherence control. We demonstrate the impact of our caching solution on parallel write performance with a comprehensive evaluation that includes a set of widely used I/O benchmarks and production application I/O kernels.},
doi = {10.1145/1362622.1362634},
journal = {},
number = ,
volume = ,
place = {United States},
year = {Mon Jan 01 00:00:00 EST 2007},
month = {Mon Jan 01 00:00:00 EST 2007}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share: