Access Patterns and Performance Behaviors of Multi-layer Supercomputer I/O Subsystems under Production Load

Conference · Wed Jun 01 04:00:00 EDT 2022

DOI:https://doi.org/10.1145/3502181.3531461· OSTI ID:1885368

Bez, Jean Luca ^[1]; Karimi, Ahmad Maroof ^[2]; Paul, Arnab ^[2]; Xie, Bing ^[2]; Byna, Suren ^[3]; Carns, Philip ^[4]; Oral, Sarp ^[2]; Wang, Feiyi ^[2]; Hanley, Jesse ^[2]

---

1. Lawrence Berkeley National Laboratory (LBNL)
2. ORNL
3. Lawrence Berkeley Laboratory, CA
4. Argonne National Laboratory (ANL)

Scientific computing workloads at HPC facilities have been shifting from traditional numerical simulations to AI/ML applications for training and inference while processing and producing ever-increasing amounts of scientific data. To address the growing need for increased storage capacity, lower access latency, and higher bandwidth, emerging technologies such as non-volatile memory are integrated into supercomputer I/O subsystems. With these emerging trends, we need a better understanding of the multilayer supercomputer I/O systems and ways to use these subsystems efficiently. In this work, we study the I/O access patterns and performance characteristics of two representative supercomputer I/O subsystems. Through an extensive analysis of year-long I/O logs on each system, we report new observations in I/O reads and writes, unbalanced use of storage system layers, and new trends in user behaviors at the HPC I/O middleware stack.

• Research Organization: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
• Sponsoring Organization: USDOE
• DOE Contract Number: AC05-00OR22725
• OSTI ID: 1885368
• Country of Publication: United States
• Language: English