Understanding I/O workload characteristics of a Peta-scale storage system

Journal Article · Tue Nov 11 00:00:00 EST 2014 · Journal of Supercomputing

DOI:https://doi.org/10.1007/s11227-014-1321-8· OSTI ID:1185800

Kim, Youngjae ^[1]; Gunasekaran, Raghul ^[1]

---

1. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). National Center for Computational Sciences

Understanding workload characteristics is critical for optimizing and improving the performance of current systems and software, and architecting new storage systems based on observed workload patterns. In this paper, we characterize the I/O workloads of scientific applications of one of the world's fastest high performance computing (HPC) storage cluster, Spider, at the Oak Ridge Leadership Computing Facility (OLCF). OLCF flagship petascale simulation platform, Titan, and other large HPC clusters, in total over 250 thousands compute cores, depend on Spider for their I/O needs. We characterize the system utilization, the demands of reads and writes, idle time, storage space utilization, and the distribution of read requests to write requests for the Peta-scale Storage Systems. From this study, we develop synthesized workloads, and we show that the read and write I/O bandwidth usage as well as the inter-arrival time of requests can be modeled as a Pareto distribution. We also study the I/O load imbalance problems using I/O performance data collected from the Spider storage system.

• Research Organization: Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF)
• Sponsoring Organization: USDOE Office of Science (SC)
• Grant/Contract Number: AC05-00OR22725
• OSTI ID: 1185800
• Journal Information: Journal of Supercomputing, Vol. 71, Issue 3; ISSN 0920-8542
• Publisher: Springer
• Country of Publication: United States
• Language: English

Related Subjects

storage systems
I/O
load imbalance
read request
request size
workload characterization