QoS support for end users of I/O-intensive applications using shared storage systems
- Los Alamos National Laboratory
- WAYNE STATE UNIV
- WAYNE STATE UNIV.
I/O-intensive applications are becoming increasingly common on today's high-performance computing systems. While performance of compute-bound applications can be effectively guaranteed with techniques such as space sharing or QoS-aware process scheduling, it remains a challenge to meet QoS requirements for end users of I/O-intensive applications using shared storage systems because it is difficult to differentiate I/O services for different applications with individual quality requirements. Furthermore, it is difficult for end users to accurately specify performance goals to the storage system using I/O-related metrics such as request latency or throughput. As access patterns, request rates, and the system workload change in time, a fixed I/O performance goal, such as bounds on throughput or latency, can be expensive to achieve and may not lead to a meaningful performance guarantees such as bounded program execution time. We propose a scheme supporting end-users QoS goals, specified in terms of program execution time, in shared storage environments. We automatically translate the users performance goals into instantaneous I/O throughput bounds using a machine learning technique, and use dynamically determined service time windows to efficiently meet the throughput bounds. We have implemented this scheme in the PVFS2 parallel file system and have conducted an extensive evaluation. Our results show that this scheme can satisfy realistic end-user QoS requirements by making highly efficient use of the I/O resources. The scheme seeks to balance programs attainment of QoS requirements, and saves as much of the remaining I/O capacity as possible for best-effort programs.
- Research Organization:
- Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- AC52-06NA25396
- OSTI ID:
- 1048794
- Report Number(s):
- LA-UR-11-00409; LA-UR-11-409; TRN: US201217%%145
- Resource Relation:
- Conference: International Conference on Supercomputing 2011 ; June 1, 2011 ; Tucson, AZ
- Country of Publication:
- United States
- Language:
- English
Similar Records
Can I/O Variability Be Reduced on QoS-Less HPC Storage Systems?
Can I/O Variability Be Reduced on QoS-Less HPC Storage Systems?