skip to main content

SciTech ConnectSciTech Connect

Title: Understanding I/O workload characteristics of a Peta-scale storage system

Understanding workload characteristics is critical for optimizing and improving the performance of current systems and software, and architecting new storage systems based on observed workload patterns. In this paper, we characterize the I/O workloads of scientific applications of one of the world s fastest high performance computing (HPC) storage cluster, Spider, at the Oak Ridge Leadership Computing Facility (OLCF). OLCF flagship petascale simulation platform, Titan, and other large HPC clusters, in total over 250 thousands compute cores, depend on Spider for their I/O needs. We characterize the system utilization, the demands of reads and writes, idle time, storage space utilization, and the distribution of read requests to write requests for the Peta-scale Storage Systems. From this study, we develop synthesized workloads, and we show that the read and write I/O bandwidth usage as well as the inter-arrival time of requests can be modeled as a Pareto distribution. We also study the I/O load imbalance problems using I/O performance data collected from the Spider storage system.
 [1] ;  [1]
  1. ORNL
Publication Date:
OSTI Identifier:
DOE Contract Number:
Resource Type:
Journal Article
Resource Relation:
Journal Name: Journal of Supercomputing; Journal Volume: 71; Journal Issue: 3
Research Org:
Oak Ridge National Laboratory (ORNL); Oak Ridge Leadership Computing Facility (OLCF)
Sponsoring Org:
Country of Publication:
United States
Storage Systems; I/O Workload Characterization