A case study in application I/O on Linux clusters.
A critical but often ignored component of system performance is the I/O system. Today's applications expect a great deal from underlying storage systems and software, and both high performance distributed storage and high level interfaces have been developed to fill these needs. In this paper they discuss the I/O performance of a parallel scientific application on a Linux cluster, the FLASH astrophysics code. This application relies on three I/O software components to provide high performance parallel I/O on Linux clusters: the Parallel Virtual File System (PVFS), the ROMIO MPI-IO implementation, and the Hierarchical Data Format (HDF5) library. First they discuss the roles played by each of these components in providing an I/O solution. Next they discuss the FLASH I/O benchmark and point out its relevance. Following this they examine the performance of the benchmark, and through instrumentation of both the application and underlying system software code they discover the location of major software bottlenecks. They work around the most inhibiting of these bottlenecks, showing substantial performance improvement. Finally they point out similarities between the inefficiencies found here and those found in message passing systems, indicating that research in the message passing field could be leveraged to solve similar problems in high-level I/O interfaces.
- Research Organization:
- Argonne National Lab., IL (US)
- Sponsoring Organization:
- US Department of Energy (US)
- DOE Contract Number:
- W-31109-ENG-38
- OSTI ID:
- 797881
- Report Number(s):
- ANL/MCS/CP-105443
- Country of Publication:
- United States
- Language:
- English
Similar Records
Scalability and Performance of a Large Linux Cluster
Bridging the gap between parallel file systems and local file systems : a case study with PVFS.