skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Modeling I/O Performance Variability Using Conditional Variational Auto Encoders

Abstract

Storage system performance modeling is crucial for efficient use of heterogeneous shared resources on leadership class computers. Variability in application performance, particularly variability arising from concurrent applications sharing I/O resources, is a major hurdle in the development of accurate performance models. We adopt a deep learning approach based on conditional variational auto encoders (CVAE) for I/O performance modeling, and use it to quantify performance variability. We illustrate our approach using the data collected on Edison, a production supercomputing system at the National Energy Research Scientific Computing Center (NERSC). The CVAE approach is investigated by comparing it to a previously proposed sensitivity-based Gaussian process (GP) model. We find that the CVAE model performs slightly better than the GP model in cases where training and testing data come from different applications, since CVAE can inherently leverage the whole data from multiple applications whereas GP partitions the data and builds separate models for each partition. Hence, the CVAE offers an alternative modeling approach that does not need preprocessing; it has enough flexibility to handle data from a wide variety of applications without changing the inference approach.

Authors:
; ; ; ; ; ;
Publication Date:
Research Org.:
Argonne National Lab. (ANL), Argonne, IL (United States)
Sponsoring Org.:
USDOE Office of Science - Office of Advanced Scientific Computing Research (ASCR)
OSTI Identifier:
1817862
DOE Contract Number:  
AC02-06CH11357
Resource Type:
Conference
Resource Relation:
Conference: 20th IEEE Cluster, 09/10/18 - 09/13/18, Belfast, GB
Country of Publication:
United States
Language:
English
Subject:
I/O perfornance variability; parallel filesystems; probabilistic machine learning; variational autoencoders

Citation Formats

Madireddy, Sandeep, Balaprakash, Prasanna, Carns, Philip, Latham, Robert, Ross, Robert, Snyder, Shane, and Wild, Stefan M. Modeling I/O Performance Variability Using Conditional Variational Auto Encoders. United States: N. p., 2018. Web. doi:10.1109/CLUSTER.2018.00022.
Madireddy, Sandeep, Balaprakash, Prasanna, Carns, Philip, Latham, Robert, Ross, Robert, Snyder, Shane, & Wild, Stefan M. Modeling I/O Performance Variability Using Conditional Variational Auto Encoders. United States. https://doi.org/10.1109/CLUSTER.2018.00022
Madireddy, Sandeep, Balaprakash, Prasanna, Carns, Philip, Latham, Robert, Ross, Robert, Snyder, Shane, and Wild, Stefan M. 2018. "Modeling I/O Performance Variability Using Conditional Variational Auto Encoders". United States. https://doi.org/10.1109/CLUSTER.2018.00022. https://www.osti.gov/servlets/purl/1817862.
@article{osti_1817862,
title = {Modeling I/O Performance Variability Using Conditional Variational Auto Encoders},
author = {Madireddy, Sandeep and Balaprakash, Prasanna and Carns, Philip and Latham, Robert and Ross, Robert and Snyder, Shane and Wild, Stefan M.},
abstractNote = {Storage system performance modeling is crucial for efficient use of heterogeneous shared resources on leadership class computers. Variability in application performance, particularly variability arising from concurrent applications sharing I/O resources, is a major hurdle in the development of accurate performance models. We adopt a deep learning approach based on conditional variational auto encoders (CVAE) for I/O performance modeling, and use it to quantify performance variability. We illustrate our approach using the data collected on Edison, a production supercomputing system at the National Energy Research Scientific Computing Center (NERSC). The CVAE approach is investigated by comparing it to a previously proposed sensitivity-based Gaussian process (GP) model. We find that the CVAE model performs slightly better than the GP model in cases where training and testing data come from different applications, since CVAE can inherently leverage the whole data from multiple applications whereas GP partitions the data and builds separate models for each partition. Hence, the CVAE offers an alternative modeling approach that does not need preprocessing; it has enough flexibility to handle data from a wide variety of applications without changing the inference approach.},
doi = {10.1109/CLUSTER.2018.00022},
url = {https://www.osti.gov/biblio/1817862}, journal = {},
number = ,
volume = ,
place = {United States},
year = {2018},
month = {1}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share: