Parallel I/O Evaluation Techniques and Emerging HPC Workloads: A Perspective

Conference · Fri Oct 01 00:00:00 EDT 2021

DOI:https://doi.org/10.1109/Cluster48925.2021.00100· OSTI ID:1973311

Neuwirth, Sarah ^[1]; Paul, Arnab ^[2]

---

1. Goethe-University Frankfurt, Germany
2. ORNL

Emerging workloads such as artificial intelligence, big data analytics and complex multi-step workflows alongside future exascale applications are anticipated future HPC workloads, which will result in a more diverse I/O system workload and even less predictable I/O behavior and access patterns. Along with the ever increasing gap between the compute and storage performance capabilities, the in-depth understanding of extreme-scale I/O behavior and the I/O performance modeling and prediction are essential tools of the large-scale I/O evaluation process for addressing the needs of extreme-scale hybrid workloads. In this survey article, we focus on the state-of-the-art of the I/O behavior and performance analysis process for HPC systems in a 5-year time window and identify future research challenges.

• Research Organization: Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
• Sponsoring Organization: USDOE
• DOE Contract Number: AC05-00OR22725
• OSTI ID: 1973311
• Resource Relation: Conference: IEEE CLUSTER: Workshop on Re-envisioning Extreme-Scale I/O for Emerging Hybrid HPC Workloads - Portland, Oregon, United States of America - 9/7/2021 4:00:00 AM-9/10/2021 4:00:00 AM
• Country of Publication: United States
• Language: English