Modeling Large-Scale Slim Fly Networks Using Parallel Discrete-Event Simulation

Journal Article · 30 August 2018 · ACM Transactions on Modeling and Computer Simulation

DOI: https://doi.org/10.1145/3203406 · OSTI ID:1488539

Wolfe, Noah ^[1]; Mubarak, Misbah ^[2]; Carothers, Christopher D. ^[1]; Ross, Robert B. ^[2]; Carns, Philip H. ^[2]

---

1. Rensselaer Polytechnic Inst., Troy, NY (United States)
2. Argonne National Lab. (ANL), Lemont, IL (United States)

As supercomputers approach exascale performance, the increased number of processors translates to an increased demand on the underlying network interconnect. We present that the slim fly network topology, a new low-diameter, low-latency, and low-cost interconnection network, is gaining interest as one possible solution for next-generation supercomputing interconnect systems. In this article, we present a high-fidelity slim fly packet-level model leveraging the Rensselaer Optimistic Simulation System (ROSS) and Co-Design of Exascale Storage (CODES) frameworks. We validate the model with published work before scaling the network size up to an unprecedented 1 million compute nodes and confirming that the slim fly observes peak network throughput at extreme scale. In addition to synthetic workloads, we evaluate large-scale slim fly models with real communication workloads from applications in the Design Forward program with over 110,000 MPI processes. We show strong scaling of the slim fly model on an Intel cluster achieving a peak network packet transfer rate of 2.3 million packets per second and processing over 7 billion discrete events using 128 MPI tasks. Enabled by the strong performance capabilities of the model, we perform a detailed application trace and routing protocol performance study. Lastly, through analysis of metrics such as packet latency, hop count, and congestion, we find that the slim fly network is able to leverage simple minimal routing and achieve the same performance as more complex adaptive routing for tested DOE benchmark applications.

• Research Organization: Argonne National Laboratory (ANL), Argonne, IL (United States)
• Sponsoring Organization: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21); Air Force Research Laboratory (AFRL)
• Grant/Contract Number: AC02-06CH11357
• OSTI ID: 1488539
• Journal Information: ACM Transactions on Modeling and Computer Simulation, Journal Name: ACM Transactions on Modeling and Computer Simulation Journal Issue: 4 Vol. 28; ISSN 1049-3301
• Publisher: Association for Computing MachineryCopyright Statement
• Country of Publication: United States
• Language: English