skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Workload Characterization of a Leadership Class Storage Cluster

Abstract

Understanding workload characteristics is critical for optimizing and improving the performance of current systems and software, and architecting new storage systems based on observed workload patterns. In this paper, we characterize the scientific workloads of the world s fastest HPC (High Performance Computing) storage cluster, Spider, at the Oak Ridge Leadership Computing Facility (OLCF). Spider provides an aggregate bandwidth of over 240 GB/s with over 10 petabytes of RAID 6 formatted capacity. OLCFs flagship petascale simulation platform, Jaguar, and other large HPC clusters, in total over 250 thousands compute cores, depend on Spider for their I/O needs. We characterize the system utilization, the demands of reads and writes, idle time, and the distribution of read requests to write requests for the storage system observed over a period of 6 months. From this study we develop synthesized workloads and we show that the read and write I/O bandwidth usage as well as the inter-arrival time of requests can be modeled as a Pareto distribution.

Authors:
 [1];  [1];  [1];  [1];  [1];  [1]
  1. ORNL
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States); Center for Computational Sciences
Sponsoring Org.:
USDOE Office of Science (SC)
OSTI Identifier:
993463
DOE Contract Number:  
DE-AC05-00OR22725
Resource Type:
Conference
Resource Relation:
Conference: Petascale Data Storage Workshop, New Orleans, LA, USA, 20101115, 20101115
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICAL METHODS AND COMPUTING; MEMORY DEVICES; INFORMATION SYSTEMS; PERFORMANCE; COMPUTER CODES; SUPERCOMPUTERS; Workload Characterization; Large-scale Storage Systems

Citation Formats

Kim, Youngjae, Gunasekaran, Raghul, Shipman, Galen M, Dillow, David A, Zhang, Zhe, and Settlemyer, Bradley W. Workload Characterization of a Leadership Class Storage Cluster. United States: N. p., 2010. Web.
Kim, Youngjae, Gunasekaran, Raghul, Shipman, Galen M, Dillow, David A, Zhang, Zhe, & Settlemyer, Bradley W. Workload Characterization of a Leadership Class Storage Cluster. United States.
Kim, Youngjae, Gunasekaran, Raghul, Shipman, Galen M, Dillow, David A, Zhang, Zhe, and Settlemyer, Bradley W. Fri . "Workload Characterization of a Leadership Class Storage Cluster". United States.
@article{osti_993463,
title = {Workload Characterization of a Leadership Class Storage Cluster},
author = {Kim, Youngjae and Gunasekaran, Raghul and Shipman, Galen M and Dillow, David A and Zhang, Zhe and Settlemyer, Bradley W},
abstractNote = {Understanding workload characteristics is critical for optimizing and improving the performance of current systems and software, and architecting new storage systems based on observed workload patterns. In this paper, we characterize the scientific workloads of the world s fastest HPC (High Performance Computing) storage cluster, Spider, at the Oak Ridge Leadership Computing Facility (OLCF). Spider provides an aggregate bandwidth of over 240 GB/s with over 10 petabytes of RAID 6 formatted capacity. OLCFs flagship petascale simulation platform, Jaguar, and other large HPC clusters, in total over 250 thousands compute cores, depend on Spider for their I/O needs. We characterize the system utilization, the demands of reads and writes, idle time, and the distribution of read requests to write requests for the storage system observed over a period of 6 months. From this study we develop synthesized workloads and we show that the read and write I/O bandwidth usage as well as the inter-arrival time of requests can be modeled as a Pareto distribution.},
doi = {},
journal = {},
number = ,
volume = ,
place = {United States},
year = {2010},
month = {1}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share: