Noncontiguous I/O through PVFS.
With the tremendous advances in processor and memory technology, I/O has risen to become the bottleneck in high-performance computing for many applications. The development of parallel file systems has helped to ease the performance gap, but I/O still remains an area needing significant performance improvement. Research has found that noncontiguous I/O access patterns in scientific applications combined with current file system methods to perform these accesses lead to unacceptable performance for large data sets. To enhance performance of noncontiguous I/O, we have created list I/O, a native version of noncontiguous I/O. We have used the Parallel Virtual File System (PVFS) to implement our ideas. Our research and experimentation shows that list I/O outperforms current noncontiguous I/O access methods in most I/O situations and can substantially enhance the performance of real-world scientific applications.
- Research Organization:
- Argonne National Lab., IL (US)
- Sponsoring Organization:
- US Department of Energy (US)
- DOE Contract Number:
- W-31109-ENG-38
- OSTI ID:
- 801647
- Report Number(s):
- ANL/MCS/CP-108408
- Country of Publication:
- United States
- Language:
- English
Similar Records
Optimizing noncontiguous accesses in MPI-IO.
PVFS : a parallel file system for linux clusters