Enabling parallel simulation of large-scale HPC network systems

doi:https://doi.org/10.1109/TPDS.2016.2543725

Title: Enabling parallel simulation of large-scale HPC network systems

Abstract

Here, with the increasing complexity of today’s high-performance computing (HPC) architectures, simulation has become an indispensable tool for exploring the design space of HPC systems—in particular, networks. In order to make effective design decisions, simulations of these systems must possess the following properties: (1) have high accuracy and fidelity, (2) produce results in a timely manner, and (3) be able to analyze a broad range of network workloads. Most state-of-the-art HPC network simulation frameworks, however, are constrained in one or more of these areas. In this work, we present a simulation framework for modeling two important classes of networks used in today’s IBM and Cray supercomputers: torus and dragonfly networks. We use the Co-Design of Multi-layer Exascale Storage Architecture (CODES) simulation framework to simulate these network topologies at a flit-level detail using the Rensselaer Optimistic Simulation System (ROSS) for parallel discrete-event simulation. Our simulation framework meets all the requirements of a practical network simulation and can assist network designers in design space exploration. First, it uses validated and detailed flit-level network models to provide an accurate and high-fidelity network simulation. Second, instead of relying on serial time-stepped or traditional conservative discrete-event simulations that limit simulation scalability and efficiency, we usemore »« less

Authors:

^[1]; Carothers, Christopher D. ^[2]; Ross, Robert B. ^[1]; Carns, Philip ^[1]

Argonne National Lab. (ANL), Lemont, IL (United States)
Rensselaer Polytechnic Inst., Troy, NY (United States)

Publication Date:: 2016-04-07

Research Org.:: Argonne National Lab. (ANL), Argonne, IL (United States)

Sponsoring Org.:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)

OSTI Identifier:: 1366454

Grant/Contract Number:: AC02-06CH11357

Resource Type:: Journal Article: Accepted Manuscript

Journal Name:: IEEE Transactions on Parallel and Distributed Systems

Additional Journal Information:: Journal Volume: 28; Journal Issue: 1; Journal ID: ISSN 1045-9219

Publisher:: IEEE

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING; interconnect networks; massively parallel discrete-event simulation; trace-based simulation

Citation Formats


                    Mubarak, Misbah, Carothers, Christopher D., Ross, Robert B., and Carns, Philip. Enabling parallel simulation of large-scale HPC network systems.  United States: N. p., 2016. 
        Web.  doi:10.1109/TPDS.2016.2543725.

Copy to clipboard


                    Mubarak, Misbah, Carothers, Christopher D., Ross, Robert B., & Carns, Philip. Enabling parallel simulation of large-scale HPC network systems.  United States.  https://doi.org/10.1109/TPDS.2016.2543725

Copy to clipboard


                    Mubarak, Misbah, Carothers, Christopher D., Ross, Robert B., and Carns, Philip. Thu .  
        "Enabling parallel simulation of large-scale HPC network systems".  United States.  https://doi.org/10.1109/TPDS.2016.2543725.  https://www.osti.gov/servlets/purl/1366454.

Copy to clipboard


                    
@article{osti_1366454,

  title        = {Enabling parallel simulation of large-scale HPC network systems},

  author       = {Mubarak, Misbah and Carothers, Christopher D. and Ross, Robert B. and Carns, Philip},

  abstractNote = {Here, with the increasing complexity of today’s high-performance computing (HPC) architectures, simulation has become an indispensable tool for exploring the design space of HPC systems—in particular, networks. In order to make effective design decisions, simulations of these systems must possess the following properties: (1) have high accuracy and fidelity, (2) produce results in a timely manner, and (3) be able to analyze a broad range of network workloads. Most state-of-the-art HPC network simulation frameworks, however, are constrained in one or more of these areas. In this work, we present a simulation framework for modeling two important classes of networks used in today’s IBM and Cray supercomputers: torus and dragonfly networks. We use the Co-Design of Multi-layer Exascale Storage Architecture (CODES) simulation framework to simulate these network topologies at a flit-level detail using the Rensselaer Optimistic Simulation System (ROSS) for parallel discrete-event simulation. Our simulation framework meets all the requirements of a practical network simulation and can assist network designers in design space exploration. First, it uses validated and detailed flit-level network models to provide an accurate and high-fidelity network simulation. Second, instead of relying on serial time-stepped or traditional conservative discrete-event simulations that limit simulation scalability and efficiency, we use the optimistic event-scheduling capability of ROSS to achieve efficient and scalable HPC network simulations on today’s high-performance cluster systems. Third, our models give network designers a choice in simulating a broad range of network workloads, including HPC application workloads using detailed network traces, an ability that is rarely offered in parallel with high-fidelity network simulations},

  doi          = {10.1109/TPDS.2016.2543725},

  url          = {https://www.osti.gov/biblio/1366454},
  journal      = {IEEE Transactions on Parallel and Distributed Systems},

  issn         = {1045-9219},

  number       = 1,

  volume       = 28,

  place        = {United States},

  year         = {2016},

  month        = {4}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1109/TPDS.2016.2543725

Other availability

Search WorldCat to find libraries that may hold this journal

Citation Metrics:

Cited by: 27 works

Citation information provided by
Web of Science

Save / Share:

Export Metadata

Save to My Library

Works referencing / citing this record:

Interference between I/O and MPI Traffic on Fat-tree Networks
conference, August 2018

Brown, Kevin A.; Jain, Nikhil; Matsuoka, Satoshi
ICPP 2018: 47th International Conference on Parallel Processing, Proceedings of the 47th International Conference on Parallel Processing
https://doi.org/10.1145/3225058.3225144

Reconfiguration of the Multi-channel Communication System with Hierarchical Structure and Distributed Passive Switching
book, June 2019

Hajder, Piotr; Rauch, Łukasz; Rodrigues, João M. F.
Computational Science – ICCS 2019: 19th International Conference, Faro, Portugal, June 12–14, 2019, Proceedings, Part II, p. 502-516
https://doi.org/10.1007/978-3-030-22741-8_36

AccaSim: a customizable workload management simulator for job dispatching research in HPC systems
journal, February 2019

Galleguillos, Cristian; Kiziltan, Zeynep; Netti, Alessio
Cluster Computing, Vol. 23, Issue 1
https://doi.org/10.1007/s10586-019-02905-5

MAHA: Migration-based Adaptive Heuristic Algorithm for Large-scale Network Simulations
journal, September 2019

Ibrahim, Muhammad; Iqbal, Muhammad Azhar; Aleem, Muhammad
Cluster Computing, Vol. 23, Issue 2
https://doi.org/10.1007/s10586-019-02991-5

Comparative Analysis of Parallel Brain Activity Mapping Algorithms for High Resolution Brain Models
journal, September 2019

Molina-Machado, Cristhian D.; Cuartas, Ernesto; Martínez-Vargas, Juan D.
TecnoLógicas, Vol. 22, Issue 46
https://doi.org/10.22430/22565337.1344

Similar records in OSTI.GOV collections:

Modeling Large-Scale Slim Fly Networks Using Parallel Discrete-Event Simulation

Journal Article Wolfe, Noah ; Mubarak, Misbah ; Carothers, Christopher D. ; ... - ACM Transactions on Modeling and Computer Simulation

As supercomputers approach exascale performance, the increased number of processors translates to an increased demand on the underlying network interconnect. We present that the slim fly network topology, a new low-diameter, low-latency, and low-cost interconnection network, is gaining interest as one possible solution for next-generation supercomputing interconnect systems. In this article, we present a high-fidelity slim fly packet-level model leveraging the Rensselaer Optimistic Simulation System (ROSS) and Co-Design of Exascale Storage (CODES) frameworks. We validate the model with published work before scaling the network size up to an unprecedented 1 million compute nodes and confirming that the slim fly observesmore »« less
Cited by 2
https://doi.org/10.1145/3203406

Full Text Available
Visual Data-Analytics of Large-Scale Parallel Discrete-Event Simulations

Conference Ross, Caitlin ; Carothers, Christopher D. ; Mubarak, Misbah ; ...

Parallel discrete-event simulation (PDES) is an important tool in the codesign of extreme-scale systems because PDES provides a cost-effective way to evaluate designs of highperformance computing systems. Optimistic synchronization algorithms for PDES, such as Time Warp, allow events to be processed without global synchronization among the processing elements. A rollback mechanism is provided when events are processed out of timestamp order. Although optimistic synchronization protocols enable the scalability of large-scale PDES, the performance of the simulations must be tuned to reduce the number of rollbacks and provide an improved simulation runtime. To enable efficient large-scale optimistic simulations, one has tomore »« less
https://doi.org/10.1109/PMBS.2016.014

Full Text Available
Durango: Scalable Synthetic Workload Generation for Extreme-Scale Application Performance Modeling and Simulation

Conference Carothers, Christopher D. ; Meredith, Jeremy S. ; Blanco, Marc ; ...

Performance modeling of extreme-scale applications on accurate representations of potential architectures is critical for designing next generation supercomputing systems because it is impractical to construct prototype systems at scale with new network hardware in order to explore designs and policies. However, these simulations often rely on static application traces that can be difficult to work with because of their size and lack of flexibility to extend or scale up without rerunning the original application. To address this problem, we have created a new technique for generating scalable, flexible workloads from real applications, we have implemented a prototype, called Durango, thatmore »« less
https://doi.org/10.1145/3064911.3064923

Full Text Available
Fit Fly: A Case Study of Interconnect Innovation through Parallel Simulation

Conference McGlohon, Neil ; Wolfe, Noah ; Mubarak, Misbah ; ...

To meet the demand for exascale-level performance from high-performance computing (HPC) interconnects, many system architects are turning to simulation results for accurate and reliable predictions of the performance of prospective technologies. Testing full-scale networks with a variety of benchmarking tools, including synthetic workloads and application traces, can give crucial insight into what ideas are most promising without needing to physically construct a test network. While flexible, however, this approach is extremely compute time intensive. We address this time complexity challenge through the use of large-scale, optimistic parallel simulation that ultimately leads to faster HPC network architecture innovations. In this papermore »« less
https://doi.org/10.1145/3316480.3325515
Modeling a Million-Node Slim Fly Network Using Parallel Discrete-Event Simulation

Conference Wolfe, Noah ; Carothers, Christopher ; Mubarak, Misbah ; ...

As supercomputers close in on exascale performance, the increased number of processors and processing power translates to an increased demand on the underlying network interconnect. The Slim Fly network topology, a new lowdiameter and low-latency interconnection network, is gaining interest as one possible solution for next-generation supercomputing interconnect systems. In this paper, we present a high-fidelity Slim Fly it-level model leveraging the Rensselaer Optimistic Simulation System (ROSS) and Co-Design of Exascale Storage (CODES) frameworks. We validate our Slim Fly model with the Kathareios et al. Slim Fly model results provided at moderately sized network scales. We further scale the modelmore »« less
https://doi.org/10.1145/2901378.2901389

Full Text Available

Similar Records

Title: Enabling parallel simulation of large-scale HPC network systems

Abstract

Citation Formats

Interference between I/O and MPI Traffic on Fat-tree Networks conference, August 2018

Reconfiguration of the Multi-channel Communication System with Hierarchical Structure and Distributed Passive Switching book, June 2019

AccaSim: a customizable workload management simulator for job dispatching research in HPC systems journal, February 2019

MAHA: Migration-based Adaptive Heuristic Algorithm for Large-scale Network Simulations journal, September 2019

Comparative Analysis of Parallel Brain Activity Mapping Algorithms for High Resolution Brain Models journal, September 2019

Interference between I/O and MPI Traffic on Fat-tree Networks
conference, August 2018

Reconfiguration of the Multi-channel Communication System with Hierarchical Structure and Distributed Passive Switching
book, June 2019

AccaSim: a customizable workload management simulator for job dispatching research in HPC systems
journal, February 2019

MAHA: Migration-based Adaptive Heuristic Algorithm for Large-scale Network Simulations
journal, September 2019

Comparative Analysis of Parallel Brain Activity Mapping Algorithms for High Resolution Brain Models
journal, September 2019