SCTuner: An Auto-tuner Addressing Dynamic I/O Needs on Supercomputer I/O Subsystems

Tang, Houjun; Xie, Bing; Byna, Suren; Carns, Philip; Koziol, Quincey; Kannan, Sudarsun; Lofstead, Jay; Oral, Sarp

Title: SCTuner: An Auto-tuner Addressing Dynamic I/O Needs on Supercomputer I/O Subsystems

Full Record
Other Related Research

Authors:

Tang, Houjun ^[1]; Xie, Bing ^[2]; Byna, Suren ^[1]; Carns, Philip ^[3]; Koziol, Quincey ^[1]; Kannan, Sudarsun ^[4]; Lofstead, Jay ^[5];

^[2]

Lawrence Berkeley Laboratory, CA
ORNL
Argonne National Laboratory (ANL)
Rutgers University
Sandia National Laboratories (SNL)

Publication Date:: 2021-11-01

Research Org.:: Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)

Sponsoring Org.:: USDOE

OSTI Identifier:: 1836439

DOE Contract Number:: AC05-00OR22725

Resource Type:: Conference

Resource Relation:: Conference: 6TH INTERNATIONAL PARALLEL DATA SYSTEMS WORKSHOP (PDSW) - Saint Louis, Missouri, United States of America - 11/15/2021 10:00:00 AM-11/20/2021 10:00:00 AM

Country of Publication:: United States

Language:: English

Citation Formats


                    Tang, Houjun, Xie, Bing, Byna, Suren, Carns, Philip, Koziol, Quincey, Kannan, Sudarsun, Lofstead, Jay, and Oral, Sarp. SCTuner: An Auto-tuner Addressing Dynamic I/O Needs on Supercomputer I/O Subsystems.  United States: N. p., 2021. 
        Web.

Copy to clipboard


                    Tang, Houjun, Xie, Bing, Byna, Suren, Carns, Philip, Koziol, Quincey, Kannan, Sudarsun, Lofstead, Jay, & Oral, Sarp. SCTuner: An Auto-tuner Addressing Dynamic I/O Needs on Supercomputer I/O Subsystems.  United States.

Copy to clipboard


                    Tang, Houjun, Xie, Bing, Byna, Suren, Carns, Philip, Koziol, Quincey, Kannan, Sudarsun, Lofstead, Jay, and Oral, Sarp. 2021.  
        "SCTuner: An Auto-tuner Addressing Dynamic I/O Needs on Supercomputer I/O Subsystems".  United States.  https://www.osti.gov/servlets/purl/1836439.

Copy to clipboard


                    
@article{osti_1836439,

  title        = {SCTuner: An Auto-tuner Addressing Dynamic I/O Needs on Supercomputer I/O Subsystems},

  author       = {Tang, Houjun and Xie, Bing and Byna, Suren and Carns, Philip and Koziol, Quincey and Kannan, Sudarsun and Lofstead, Jay and Oral, Sarp},

  abstractNote = {},

  doi          = {},

  url          = {https://www.osti.gov/biblio/1836439},
  journal      = {},
number       = ,

  volume       = ,

  place        = {United States},

  year         = {2021},

  month        = {11}

}

Copy to clipboard

Conference:

View Conference (0.67 MB)

Other availability

Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share:

Export Metadata

Save to My Library

Similar records in OSTI.GOV collections:

Addressing Scientific I/O Needs for Current and Future Architectures.

Conference Oldfield, Ron

Abstract not provided.
Full Text Available
Analysis and Modeling of the End-to-End I/O Performance on OLCF's Titan Supercomputer

Conference Wan, Lipeng; Wolf, Matthew; Wang, Feiyi; ...

With the increase of scale and complexity seen in a variety of leadership-class scientific computation and simulation applications, it has become more important to understand their I/O performance characteristics. The user-observed performance is a combination of properties of how the application is using the HPC facility, as well as how others' use of the facility causes variability in the static machine capabilities. Our work leverages statistical analysis of I/O performance data gathered with fine time resolution over a full week from Titan supercomputer. Based on observed properties of the distribution of I/O latencies, we build a three-state hidden Markov modelmore »« less
https://doi.org/10.1109/HPCC-SmartCity-DSS.2017.1

Full Text Available
End-to-end I/O portfolio for the summit supercomputing ecosystem

Conference Oral, Sarp; Vazhkudai, Sudharshan; Wang, Feiyi; ...

The I/O subsystem for the Summit supercomputer, No. 1 on the Top500 list, and its ecosystem of analysis platforms is composed of two distinct layers, namely the in-system layer and the center-wide parallel file system layer (PFS), Spider 3. The in-system layer uses node-local SSDs and provides 26.7 TB/s for reads, 9.7 TB/s for writes, and 4.6 billion IOPS to Summit. The Spider 3 PFS layer uses IBM's Spectrum Scale™ and provides 2.5 TB/s and 2.6 million IOPS to Summit and other systems. While deploying them as two distinct layers was operationally efficient, it also presented usability challenges in termsmore »« less
https://doi.org/10.1145/3295500.3356157

Full Text Available
Interpreting Write Performance of Supercomputer I/O Systems with Regression Models

Conference Xie, Bing; Tan, Zilong; Carns, Philip; ...

This work seeks to advance the state of the art in HPC I/O performance analysis and interpretation. In particular, we demonstrate effective techniques to: (1) model output performance in the presence of I/O interference from production loads; (2) build features from write patterns and key parameters of the system architecture and configurations; (3) employ suitable machine learning algorithms to improve model accuracy. We train models with five popular regression algorithms and conduct experiments on two distinct production HPC platforms. We find that the lasso and random forest models predict output performance with high accuracy on both of the target systems.more »« less
Full Text Available
Using Balanced Data Placement to Address I/O Contention in Production Environments

Conference Neuwirth, Sarah; Wang, Feiyi; Oral, H; ...
https://doi.org/10.1109/SBAC-PAD.2016.10

Full Text Available

Similar Records