Managing variability in the IO performance of petascale storage systems.
Significant challenges exist for achieving peak or even consistent levels of performance when using IO systems at scale. They stem from sharing IO system resources across the processes of single large-scale applications and/or multiple simultaneous programs causing internal and external interference, which in turn, causes substantial reductions in IO performance. This paper presents interference effects measurements for two different file systems at multiple supercomputing sites. These measurements motivate developing a 'managed' IO approach using adaptive algorithms varying the IO system workload based on current levels and use areas. An implementation of these methods deployed for the shared, general scratch storage system on Oak Ridge National Laboratory machines achieves higher overall performance and less variability in both a typical usage environment and with artificially introduced levels of 'noise'. The latter serving to clearly delineate and illustrate potential problems arising from shared system usage and the advantages derived from actively managing it.
- Research Organization:
- Sandia National Laboratories (SNL), Albuquerque, NM, and Livermore, CA (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- AC04-94AL85000
- OSTI ID:
- 1031319
- Report Number(s):
- SAND2010-7984C; TRN: US201201%%518
- Resource Relation:
- Conference: Proposed for presentation at a Prep talk at Georgia Tech and actual talk at Supercomputing 2010 held November 11-18, 2010 in Atlanta & New Orleans, LA.
- Country of Publication:
- United States
- Language:
- English
Similar Records
Automatic and Transparent Resource Contention Mitigation for Improving Large-Scale Parallel File System Performance
Automatic and Transparent Resource Contention Mitigation for Improving Large-Scale Parallel File System Performance