skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: QoS support for end users of I/O-intensive applications using shared storage systems

Abstract

I/O-intensive applications are becoming increasingly common on today's high-performance computing systems. While performance of compute-bound applications can be effectively guaranteed with techniques such as space sharing or QoS-aware process scheduling, it remains a challenge to meet QoS requirements for end users of I/O-intensive applications using shared storage systems because it is difficult to differentiate I/O services for different applications with individual quality requirements. Furthermore, it is difficult for end users to accurately specify performance goals to the storage system using I/O-related metrics such as request latency or throughput. As access patterns, request rates, and the system workload change in time, a fixed I/O performance goal, such as bounds on throughput or latency, can be expensive to achieve and may not lead to a meaningful performance guarantees such as bounded program execution time. We propose a scheme supporting end-users QoS goals, specified in terms of program execution time, in shared storage environments. We automatically translate the users performance goals into instantaneous I/O throughput bounds using a machine learning technique, and use dynamically determined service time windows to efficiently meet the throughput bounds. We have implemented this scheme in the PVFS2 parallel file system and have conducted an extensive evaluation. Our resultsmore » show that this scheme can satisfy realistic end-user QoS requirements by making highly efficient use of the I/O resources. The scheme seeks to balance programs attainment of QoS requirements, and saves as much of the remaining I/O capacity as possible for best-effort programs.« less

Authors:
 [1];  [2];  [3]
  1. Los Alamos National Laboratory
  2. WAYNE STATE UNIV
  3. WAYNE STATE UNIV.
Publication Date:
Research Org.:
Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
1048794
Report Number(s):
LA-UR-11-00409; LA-UR-11-409
TRN: US201217%%145
DOE Contract Number:  
AC52-06NA25396
Resource Type:
Conference
Resource Relation:
Conference: International Conference on Supercomputing 2011 ; June 1, 2011 ; Tucson, AZ
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICAL METHODS AND COMPUTING; CAPACITY; EVALUATION; LEARNING; METRICS; PERFORMANCE; STORAGE; WINDOWS; COMPUTERS; COMPUTER CODES; PROGRAMMING

Citation Formats

Davis, Marion Kei, Zhang, Xuechen, and Jiang, Song. QoS support for end users of I/O-intensive applications using shared storage systems. United States: N. p., 2011. Web.
Davis, Marion Kei, Zhang, Xuechen, & Jiang, Song. QoS support for end users of I/O-intensive applications using shared storage systems. United States.
Davis, Marion Kei, Zhang, Xuechen, and Jiang, Song. Wed . "QoS support for end users of I/O-intensive applications using shared storage systems". United States. https://www.osti.gov/servlets/purl/1048794.
@article{osti_1048794,
title = {QoS support for end users of I/O-intensive applications using shared storage systems},
author = {Davis, Marion Kei and Zhang, Xuechen and Jiang, Song},
abstractNote = {I/O-intensive applications are becoming increasingly common on today's high-performance computing systems. While performance of compute-bound applications can be effectively guaranteed with techniques such as space sharing or QoS-aware process scheduling, it remains a challenge to meet QoS requirements for end users of I/O-intensive applications using shared storage systems because it is difficult to differentiate I/O services for different applications with individual quality requirements. Furthermore, it is difficult for end users to accurately specify performance goals to the storage system using I/O-related metrics such as request latency or throughput. As access patterns, request rates, and the system workload change in time, a fixed I/O performance goal, such as bounds on throughput or latency, can be expensive to achieve and may not lead to a meaningful performance guarantees such as bounded program execution time. We propose a scheme supporting end-users QoS goals, specified in terms of program execution time, in shared storage environments. We automatically translate the users performance goals into instantaneous I/O throughput bounds using a machine learning technique, and use dynamically determined service time windows to efficiently meet the throughput bounds. We have implemented this scheme in the PVFS2 parallel file system and have conducted an extensive evaluation. Our results show that this scheme can satisfy realistic end-user QoS requirements by making highly efficient use of the I/O resources. The scheme seeks to balance programs attainment of QoS requirements, and saves as much of the remaining I/O capacity as possible for best-effort programs.},
doi = {},
journal = {},
number = ,
volume = ,
place = {United States},
year = {2011},
month = {1}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share: