Managing variability in the IO performance of petascale storage systems.

Conference · Mon Nov 01 04:00:00 EDT 2010

OSTI ID:1031319

Wolf, Matthew; Zheng, Fang; Klasky, Scott; Schwan, Karsten; Oldfield, Ron A; Liu, Qing; Kordenbrock, Todd

Significant challenges exist for achieving peak or even consistent levels of performance when using IO systems at scale. They stem from sharing IO system resources across the processes of single large-scale applications and/or multiple simultaneous programs causing internal and external interference, which in turn, causes substantial reductions in IO performance. This paper presents interference effects measurements for two different file systems at multiple supercomputing sites. These measurements motivate developing a 'managed' IO approach using adaptive algorithms varying the IO system workload based on current levels and use areas. An implementation of these methods deployed for the shared, general scratch storage system on Oak Ridge National Laboratory machines achieves higher overall performance and less variability in both a typical usage environment and with artificially introduced levels of 'noise'. The latter serving to clearly delineate and illustrate potential problems arising from shared system usage and the advantages derived from actively managing it.

• Research Organization: Sandia National Laboratories
• Sponsoring Organization: USDOE
• DOE Contract Number: AC04-94AL85000
• OSTI ID: 1031319
• Report Number(s): SAND2010-7984C
• Country of Publication: United States
• Language: English

Related Subjects

99 GENERAL AND MISCELLANEOUS
ALGORITHMS
COMPUTERS
DATA BASE MANAGEMENT
IMPLEMENTATION
ORNL
PERFORMANCE
STORAGE