skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Using MPI file caching to improve parallel write performance for large-scale scientific applications

Abstract

Typical large-scale scientific applications periodically write checkpoint files to save the computational state throughout execution. Existing parallel file systems improve such write-only I/O patterns through the use of client-side file caching and write-behind strategies. In distributed environments where files are rarely accessed by more than one client concurrently, file caching has achieved significant success; however, in parallel applications where multiple clients manipulate a shared file, cache coherence control can serialize I/O. We have designed a thread based caching layer for the MPI I/O library, which adds a portable caching system closer to user applications so more information about the application's I/O patterns is available for better coherence control. We demonstrate the impact of our caching solution on parallel write performance with a comprehensive evaluation that includes a set of widely used I/O benchmarks and production application I/O kernels.

Authors:
 [1];  [1];  [2];  [1];  [3]
  1. ORNL
  2. Sandia National Laboratories (SNL)
  3. Northwestern University, Evanston
Publication Date:
Research Org.:
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Center for Computational Sciences
Sponsoring Org.:
USDOE Office of Science (SC)
OSTI Identifier:
931828
DOE Contract Number:  
AC05-00OR22725
Resource Type:
Conference
Resource Relation:
Conference: Super Computing, Reno, NV, USA, 20071110, 20071110
Country of Publication:
United States
Language:
English

Citation Formats

Sankaran, Ramanan, Liao, Wei-Keng, Chen, Jacqueline H, Klasky, Scott A, and Choudhary, Alok. Using MPI file caching to improve parallel write performance for large-scale scientific applications. United States: N. p., 2007. Web.
Sankaran, Ramanan, Liao, Wei-Keng, Chen, Jacqueline H, Klasky, Scott A, & Choudhary, Alok. Using MPI file caching to improve parallel write performance for large-scale scientific applications. United States.
Sankaran, Ramanan, Liao, Wei-Keng, Chen, Jacqueline H, Klasky, Scott A, and Choudhary, Alok. Mon . "Using MPI file caching to improve parallel write performance for large-scale scientific applications". United States. doi:.
@article{osti_931828,
title = {Using MPI file caching to improve parallel write performance for large-scale scientific applications},
author = {Sankaran, Ramanan and Liao, Wei-Keng and Chen, Jacqueline H and Klasky, Scott A and Choudhary, Alok},
abstractNote = {Typical large-scale scientific applications periodically write checkpoint files to save the computational state throughout execution. Existing parallel file systems improve such write-only I/O patterns through the use of client-side file caching and write-behind strategies. In distributed environments where files are rarely accessed by more than one client concurrently, file caching has achieved significant success; however, in parallel applications where multiple clients manipulate a shared file, cache coherence control can serialize I/O. We have designed a thread based caching layer for the MPI I/O library, which adds a portable caching system closer to user applications so more information about the application's I/O patterns is available for better coherence control. We demonstrate the impact of our caching solution on parallel write performance with a comprehensive evaluation that includes a set of widely used I/O benchmarks and production application I/O kernels.},
doi = {},
journal = {},
number = ,
volume = ,
place = {United States},
year = {Mon Jan 01 00:00:00 EST 2007},
month = {Mon Jan 01 00:00:00 EST 2007}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share: