skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Interpreting Write Performance of Supercomputer I/O Systems with Regression Models

Abstract

This work seeks to advance the state of the art in HPC I/O performance analysis and interpretation. In particular, we demonstrate effective techniques to: (1) model output performance in the presence of I/O interference from production loads; (2) build features from write patterns and key parameters of the system architecture and configurations; (3) employ suitable machine learning algorithms to improve model accuracy. We train models with five popular regression algorithms and conduct experiments on two distinct production HPC platforms. We find that the lasso and random forest models predict output performance with high accuracy on both of the target systems. We also explore use of the models to guide adaptation in I/O middleware systems, and show potential for improvements of at least 15% from model-guided adaptation on 70% of samples, and improvements up to 10× on some samples for both of the target systems.

Authors:
 [1];  [2];  [3];  [4];  [3];  [5]; ORCiD logo [1];  [6]; ORCiD logo [1]
  1. ORNL
  2. Carnegie Mellon University (CMU)
  3. Argonne National Laboratory (ANL)
  4. Duke University
  5. Sandia National Laboratories (SNL)
  6. Micron Technology Inc
Publication Date:
Research Org.:
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
1809965
DOE Contract Number:  
AC05-00OR22725
Resource Type:
Conference
Resource Relation:
Conference: IEEE International Parallel and Distributed Processing Symposium (IPDPS) - Portland, Oregon, United States of America - 5/17/2021 8:00:00 AM-5/21/2021 8:00:00 AM
Country of Publication:
United States
Language:
English

Citation Formats

Xie, Bing, Tan, Zilong, Carns, Philip, Chase, Jeffrey, Harms, Kevin, Lofstead, Gerald, Oral, Sarp, Vazhkudai, Sudharshan S., and Wang, Feiyi. Interpreting Write Performance of Supercomputer I/O Systems with Regression Models. United States: N. p., 2021. Web. doi:10.1109/IPDPS49936.2021.00064.
Xie, Bing, Tan, Zilong, Carns, Philip, Chase, Jeffrey, Harms, Kevin, Lofstead, Gerald, Oral, Sarp, Vazhkudai, Sudharshan S., & Wang, Feiyi. Interpreting Write Performance of Supercomputer I/O Systems with Regression Models. United States. https://doi.org/10.1109/IPDPS49936.2021.00064
Xie, Bing, Tan, Zilong, Carns, Philip, Chase, Jeffrey, Harms, Kevin, Lofstead, Gerald, Oral, Sarp, Vazhkudai, Sudharshan S., and Wang, Feiyi. 2021. "Interpreting Write Performance of Supercomputer I/O Systems with Regression Models". United States. https://doi.org/10.1109/IPDPS49936.2021.00064. https://www.osti.gov/servlets/purl/1809965.
@article{osti_1809965,
title = {Interpreting Write Performance of Supercomputer I/O Systems with Regression Models},
author = {Xie, Bing and Tan, Zilong and Carns, Philip and Chase, Jeffrey and Harms, Kevin and Lofstead, Gerald and Oral, Sarp and Vazhkudai, Sudharshan S. and Wang, Feiyi},
abstractNote = {This work seeks to advance the state of the art in HPC I/O performance analysis and interpretation. In particular, we demonstrate effective techniques to: (1) model output performance in the presence of I/O interference from production loads; (2) build features from write patterns and key parameters of the system architecture and configurations; (3) employ suitable machine learning algorithms to improve model accuracy. We train models with five popular regression algorithms and conduct experiments on two distinct production HPC platforms. We find that the lasso and random forest models predict output performance with high accuracy on both of the target systems. We also explore use of the models to guide adaptation in I/O middleware systems, and show potential for improvements of at least 15% from model-guided adaptation on 70% of samples, and improvements up to 10× on some samples for both of the target systems.},
doi = {10.1109/IPDPS49936.2021.00064},
url = {https://www.osti.gov/biblio/1809965}, journal = {},
number = ,
volume = ,
place = {United States},
year = {Sat May 01 00:00:00 EDT 2021},
month = {Sat May 01 00:00:00 EDT 2021}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share: