skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Using MPI File Caching to Improve Parallel Write Performance for Large-Scale Scientific Applications

Abstract

Typical large-scale scientific applications periodically write checkpoint files to save the computational state throughout execution. Existing parallel file systems improve such write-only I/O patterns through the use of clientside file caching and write-behind strategies. In distributed environments where files are rarely accessed by more than one client concurrently, file caching has achieved significant success; however, in parallel applications where multiple clients manipulate a shared file, cache coherence control can serialize I/O. We have designed a thread based caching layer for the MPI I/O library, which adds a portable caching system closer to user applications so more information about the application's I/O patterns is available for better coherence control. We demonstrate the impact of our caching solution on parallel write performance with a comprehensive evaluation that includes a set of widely used I/O benchmarks and production application I/O kernels.

Authors:
 [1];  [1];  [1];  [1];  [1];  [2];  [3];  [3]
  1. Northwestern University, Evanston
  2. Sandia National Laboratories (SNL)
  3. ORNL
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Sponsoring Org.:
USDOE Office of Science (SC)
OSTI Identifier:
1054965
DOE Contract Number:
AC05-00OR22725
Resource Type:
Conference
Resource Relation:
Conference: SC07, Reno, NV, USA, 20071110, 20071116
Country of Publication:
United States
Language:
English

Citation Formats

Liao, Wei-keng, Ching, Avery, Coloma, Kenin, Nisar, Arifa, Choudhary, Alok, Chen, Jacqueline H, Sankaran, Ramanan, and Klasky, Scott A. Using MPI File Caching to Improve Parallel Write Performance for Large-Scale Scientific Applications. United States: N. p., 2007. Web. doi:10.1145/1362622.1362634.
Liao, Wei-keng, Ching, Avery, Coloma, Kenin, Nisar, Arifa, Choudhary, Alok, Chen, Jacqueline H, Sankaran, Ramanan, & Klasky, Scott A. Using MPI File Caching to Improve Parallel Write Performance for Large-Scale Scientific Applications. United States. doi:10.1145/1362622.1362634.
Liao, Wei-keng, Ching, Avery, Coloma, Kenin, Nisar, Arifa, Choudhary, Alok, Chen, Jacqueline H, Sankaran, Ramanan, and Klasky, Scott A. Mon . "Using MPI File Caching to Improve Parallel Write Performance for Large-Scale Scientific Applications". United States. doi:10.1145/1362622.1362634.
@article{osti_1054965,
title = {Using MPI File Caching to Improve Parallel Write Performance for Large-Scale Scientific Applications},
author = {Liao, Wei-keng and Ching, Avery and Coloma, Kenin and Nisar, Arifa and Choudhary, Alok and Chen, Jacqueline H and Sankaran, Ramanan and Klasky, Scott A},
abstractNote = {Typical large-scale scientific applications periodically write checkpoint files to save the computational state throughout execution. Existing parallel file systems improve such write-only I/O patterns through the use of clientside file caching and write-behind strategies. In distributed environments where files are rarely accessed by more than one client concurrently, file caching has achieved significant success; however, in parallel applications where multiple clients manipulate a shared file, cache coherence control can serialize I/O. We have designed a thread based caching layer for the MPI I/O library, which adds a portable caching system closer to user applications so more information about the application's I/O patterns is available for better coherence control. We demonstrate the impact of our caching solution on parallel write performance with a comprehensive evaluation that includes a set of widely used I/O benchmarks and production application I/O kernels.},
doi = {10.1145/1362622.1362634},
journal = {},
number = ,
volume = ,
place = {United States},
year = {Mon Jan 01 00:00:00 EST 2007},
month = {Mon Jan 01 00:00:00 EST 2007}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share:
  • Typical large-scale scientific applications periodically write checkpoint files to save the computational state throughout execution. Existing parallel file systems improve such write-only I/O patterns through the use of client-side file caching and write-behind strategies. In distributed environments where files are rarely accessed by more than one client concurrently, file caching has achieved significant success; however, in parallel applications where multiple clients manipulate a shared file, cache coherence control can serialize I/O. We have designed a thread based caching layer for the MPI I/O library, which adds a portable caching system closer to user applications so more information about the application'smore » I/O patterns is available for better coherence control. We demonstrate the impact of our caching solution on parallel write performance with a comprehensive evaluation that includes a set of widely used I/O benchmarks and production application I/O kernels.« less
  • We present some theoretical and experimental results of animportant caching problem that arises frequently in data intensivescientific applications. In such applications, jobs need to processseveral files simultaneously, i.e., a job can only be serviced if all itsneeded files are present in the disk cache. The set of files requested bya job is called a file-bundle. This requirement introduces the need forcache replacement algorithms based on file-bundles rather then individualfiles. We show that traditional caching algorithms such Least RecentlyUsed (LRU), and GreedyDual-Size (GDS), are not optimal in this case sincethey are not sensitive to file-bundles and may hold in the cachenon-relevantmore » combinations of files. In this paper we propose and analyzea new cache replacement algorithm specifically adapted to deal withfile-bundles. We tested the new algorithm using a disk cache simulationmodel under a wide range of parameters such as file requestdistributions, relative cache size, file size distribution,and queuesize. In all these tests, the results show significant improvement overtraditional caching algorithms such as GDS.« less
  • Parallel scientific applications require high-performance I/O support from underlying file systems. A comprehensive understanding of the expected workload is therefore essential for the design of high-performance parallel file systems. We re-examine the workload characteristics in parallel computing environments in the light of recent technology advances and new applications. We analyze application traces from a cluster with hundreds of nodes. On average, each application has only one or two typical request sizes. Large requests from several hundred kilobytes to several megabytes are very common. Although in some applications, small requests account for more than 90% of all requests, almost all ofmore » the I/O data are transferred by large requests. All of these applications show bursty access patterns. More than 65% of write requests have inter-arrival times within one millisecond in most applications. By running the same benchmark on different file models, we also find that the write throughput of using an individual output file for each node exceeds that of using a shared file for all nodes by a factor of 5. This indicates that current file systems are not well optimized for file sharing.« less
  • Abstract not provided.