Improving Parallel I/O Performance with Data Layout Awareness
- ORNL
- Illinois Institute of Technology
- Argonne National Laboratory (ANL)
Parallel applications can benefit greatly from massive computational capability, but their performance suffers from large latency of I/O accesses. The poor I/O performance has been attributed as a critical cause of the low sustained performance of parallel computing systems. In this study, we propose a data layout-aware optimization strategy to promote a better integration of the parallel I/O middleware and parallel file systems, two major components of the current parallel I/O systems, and to improve the data access performance. We explore the layout-aware optimization in both independent I/O and collective I/O, two primary forms of I/O in parallel applications. We illustrate that the layout-aware I/O optimization could improve the performance of current parallel I/O strategy effectively. The experimental results verify that the proposed strategy could improve parallel I/O performance by nearly 40% on average. The proposed layout-aware parallel I/O has a promising potential in improving the I/O performance of parallel systems.
- Research Organization:
- Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
- Sponsoring Organization:
- USDOE Office of Science (SC)
- DOE Contract Number:
- DE-AC05-00OR22725
- OSTI ID:
- 989728
- Resource Relation:
- Conference: IEEE International Conference on Cluster Computing 2010, Heraklion, Greece, 20100920, 20100924
- Country of Publication:
- United States
- Language:
- English
Similar Records
Layout-Aware I/O Scheduling for Terabits Data Movement
Layout-aware I/O Scheduling for terabits data movement