skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Durango: Scalable Synthetic Workloard Generation for Extreme-Scale Application Performance Modeling & Simulation

Authors:
; ; ; ; ; ;
Publication Date:
Research Org.:
Argonne National Lab. (ANL), Argonne, IL (United States)
Sponsoring Org.:
USDOE Office of Science - Office of Advanced Scientific Computing Research
OSTI Identifier:
1364656
DOE Contract Number:
AC02-06CH11357
Resource Type:
Conference
Resource Relation:
Conference: 2017 ACM SIGSIM Conference on Principles of Advanced Discrete Simulation, 05/24/17 - 05/26/17, Nanyang Technological University, SG
Country of Publication:
United States
Language:
English

Citation Formats

Carothers, Christopher D., Meredith, Jeremy S., Blanco, Mark, Vetter, Jeffery, Mubarak, Misbah, LaPre, Justin, and Moore, Shirley V.. Durango: Scalable Synthetic Workloard Generation for Extreme-Scale Application Performance Modeling & Simulation. United States: N. p., 2017. Web. doi:10.1145/3064911.3064923.
Carothers, Christopher D., Meredith, Jeremy S., Blanco, Mark, Vetter, Jeffery, Mubarak, Misbah, LaPre, Justin, & Moore, Shirley V.. Durango: Scalable Synthetic Workloard Generation for Extreme-Scale Application Performance Modeling & Simulation. United States. doi:10.1145/3064911.3064923.
Carothers, Christopher D., Meredith, Jeremy S., Blanco, Mark, Vetter, Jeffery, Mubarak, Misbah, LaPre, Justin, and Moore, Shirley V.. Wed . "Durango: Scalable Synthetic Workloard Generation for Extreme-Scale Application Performance Modeling & Simulation". United States. doi:10.1145/3064911.3064923. https://www.osti.gov/servlets/purl/1364656.
@article{osti_1364656,
title = {Durango: Scalable Synthetic Workloard Generation for Extreme-Scale Application Performance Modeling & Simulation},
author = {Carothers, Christopher D. and Meredith, Jeremy S. and Blanco, Mark and Vetter, Jeffery and Mubarak, Misbah and LaPre, Justin and Moore, Shirley V.},
abstractNote = {},
doi = {10.1145/3064911.3064923},
journal = {},
number = ,
volume = ,
place = {United States},
year = {Wed May 24 00:00:00 EDT 2017},
month = {Wed May 24 00:00:00 EDT 2017}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share:
  • Performance modeling of extreme-scale applications on accurate representations of potential architectures is critical for designing next generation supercomputing systems because it is impractical to construct prototype systems at scale with new network hardware in order to explore designs and policies. However, these simulations often rely on static application traces that can be difficult to work with because of their size and lack of flexibility to extend or scale up without rerunning the original application. To address this problem, we have created a new technique for generating scalable, flexible workloads from real applications, we have implemented a prototype, called Durango, thatmore » combines a proven analytical performance modeling language, Aspen, with the massively parallel HPC network modeling capabilities of the CODES framework.Our models are compact, parameterized and representative of real applications with computation events. They are not resource intensive to create and are portable across simulator environments. We demonstrate the utility of Durango by simulating the LULESH application in the CODES simulation environment on several topologies and show that Durango is practical to use for simulation without loss of fidelity, as quantified by simulation metrics. During our validation of Durango's generated communication model of LULESH, we found that the original LULESH miniapp code had a latent bug where the MPI_Waitall operation was used incorrectly. This finding underscores the potential need for a tool such as Durango, beyond its benefits for flexible workload generation and modeling.Additionally, we demonstrate the efficacy of Durango's direct integration approach, which links Aspen into CODES as part of the running network simulation model. Here, Aspen generates the application-level computation timing events, which in turn drive the start of a network communication phase. Results show that Durango's performance scales well when executing both torus and dragonfly network models on up to 4K Blue Gene/Q nodes using 32K MPI ranks, Durango also avoids the overheads and complexities associated with extreme-scale trace files.« less
  • Abstract not provided.
  • Abstract not provided.
  • The path to extreme scale high-performance computing (HPC) poses several challenges related to power, performance, resilience, productivity, programmability, data movement, and data management. Investigating the performance of parallel applications at scale on future architectures and the performance impact of different architectural choices is an important component of HPC hardware/software co-design. Simulations using models of future HPC systems and communication traces from applications running on existing HPC systems can offer an insight into the performance of future architectures. This work targets technology developed for scalable application tracing of communication events. It focuses on extreme-scale simulation of HPC applications and their communicationmore » behavior via lightweight parallel discrete event simulation for performance estimation and evaluation. Instead of simply replaying a trace within a simulator, this work promotes the generation of a benchmark from traces. This benchmark is subsequently exposed to simulation using models to reflect the performance characteristics of future-generation HPC systems. This technique provides a number of benefits, such as eliminating the data intensive trace replay and enabling simulations at different scales. The presented work features novel software co-design aspects, combining the ScalaTrace tool to generate scalable trace files, the ScalaBenchGen tool to generate the benchmark, and the xSim tool to assess the benchmark characteristics within a simulator.« less