Understanding I/O workload characteristics of a Peta-scale storage system
Understanding workload characteristics is critical for optimizing and improving the performance of current systems and software, and architecting new storage systems based on observed workload patterns. In this paper, we characterize the I/O workloads of scientific applications of one of the world s fastest high performance computing (HPC) storage cluster, Spider, at the Oak Ridge Leadership Computing Facility (OLCF). OLCF flagship petascale simulation platform, Titan, and other large HPC clusters, in total over 250 thousands compute cores, depend on Spider for their I/O needs. We characterize the system utilization, the demands of reads and writes, idle time, storage space utilization, and the distribution of read requests to write requests for the Peta-scale Storage Systems. From this study, we develop synthesized workloads, and we show that the read and write I/O bandwidth usage as well as the inter-arrival time of requests can be modeled as a Pareto distribution. We also study the I/O load imbalance problems using I/O performance data collected from the Spider storage system.
- Publication Date:
- OSTI Identifier:
- DOE Contract Number:
- Resource Type:
- Journal Article
- Resource Relation:
- Journal Name: Journal of Supercomputing; Journal Volume: 71; Journal Issue: 3
- Research Org:
- Oak Ridge National Laboratory (ORNL); Oak Ridge Leadership Computing Facility (OLCF)
- Sponsoring Org:
- Country of Publication:
- United States
- Storage Systems; I/O Workload Characterization
Enter terms in the toolbar above to search the full text of this document for pages containing specific keywords.