skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Efficient Parallel Execution of Event-Driven Electromagnetic Hybrid Models

Abstract

New discrete-event formulations of physics simulation models are emerging that can outperform traditional time-stepped models, especially in simulations containing multiple timescales. Detailed simulation of the Earth's magnetosphere, for example, requires execution of sub-models that operate at timescales that differ by orders of magnitude. In contrast to time-stepped simulation which requires tightly coupled updates to almost the entire system state at regular time intervals, the new discrete event simulation (DES) approaches help evolve the states of sub-models on relatively independent timescales. However, in contrast to relative ease of parallelization of time-stepped codes, the parallelization of DES-based models raises challenges with respect to their scalability and performance. One of the key challenges is to improve the computation granularity to offset synchronization and communication overheads within and across processors. Our previous work on parallelization was limited in scalability and runtime performance due to such challenges. Here we report on optimizations we performed on DES-based plasma simulation models to improve parallel execution performance. The mapping of the model to simulation processes is optimized via aggregation techniques, and the parallel runtime engine is optimized for communication and memory efficiency. The net result is the capability to simulate hybrid particle-in-cell (PIC) models with over 2 billionmore » ion particles using 512 processors on supercomputing platforms.« less

Authors:
 [1];  [2];  [1]
  1. ORNL
  2. SciberQuest Inc.
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States); Center for Computational Sciences
Sponsoring Org.:
USDOE Laboratory Directed Research and Development (LDRD) Program
OSTI Identifier:
934798
DOE Contract Number:
DE-AC05-00OR22725
Resource Type:
Journal Article
Resource Relation:
Journal Name: International Journal for Multiscale Computational Engineering; Journal Volume: 5; Journal Issue: 1
Country of Publication:
United States
Language:
English
Subject:
97; 99 GENERAL AND MISCELLANEOUS//MATHEMATICS, COMPUTING, AND INFORMATION SCIENCE; ELECTROMAGNETIC RADIATION; PARALLEL PROCESSING; MATHEMATICAL MODELS; COMPUTER CALCULATIONS; PLASMA; SUPERCOMPUTERS

Citation Formats

Perumalla, Kalyan S, Karimabadi, Dr. Homa, and Fujimoto, Richard. Efficient Parallel Execution of Event-Driven Electromagnetic Hybrid Models. United States: N. p., 2007. Web. doi:10.1615/IntJMultCompEng.v5.i1.40.
Perumalla, Kalyan S, Karimabadi, Dr. Homa, & Fujimoto, Richard. Efficient Parallel Execution of Event-Driven Electromagnetic Hybrid Models. United States. doi:10.1615/IntJMultCompEng.v5.i1.40.
Perumalla, Kalyan S, Karimabadi, Dr. Homa, and Fujimoto, Richard. Mon . "Efficient Parallel Execution of Event-Driven Electromagnetic Hybrid Models". United States. doi:10.1615/IntJMultCompEng.v5.i1.40.
@article{osti_934798,
title = {Efficient Parallel Execution of Event-Driven Electromagnetic Hybrid Models},
author = {Perumalla, Kalyan S and Karimabadi, Dr. Homa and Fujimoto, Richard},
abstractNote = {New discrete-event formulations of physics simulation models are emerging that can outperform traditional time-stepped models, especially in simulations containing multiple timescales. Detailed simulation of the Earth's magnetosphere, for example, requires execution of sub-models that operate at timescales that differ by orders of magnitude. In contrast to time-stepped simulation which requires tightly coupled updates to almost the entire system state at regular time intervals, the new discrete event simulation (DES) approaches help evolve the states of sub-models on relatively independent timescales. However, in contrast to relative ease of parallelization of time-stepped codes, the parallelization of DES-based models raises challenges with respect to their scalability and performance. One of the key challenges is to improve the computation granularity to offset synchronization and communication overheads within and across processors. Our previous work on parallelization was limited in scalability and runtime performance due to such challenges. Here we report on optimizations we performed on DES-based plasma simulation models to improve parallel execution performance. The mapping of the model to simulation processes is optimized via aggregation techniques, and the parallel runtime engine is optimized for communication and memory efficiency. The net result is the capability to simulate hybrid particle-in-cell (PIC) models with over 2 billion ion particles using 512 processors on supercomputing platforms.},
doi = {10.1615/IntJMultCompEng.v5.i1.40},
journal = {International Journal for Multiscale Computational Engineering},
number = 1,
volume = 5,
place = {United States},
year = {Mon Jan 01 00:00:00 EST 2007},
month = {Mon Jan 01 00:00:00 EST 2007}
}
  • Vehicular traffic simulations are useful in applications such as emergency planning and traffic management. High speed of traffic simulations translates to speed of response and level of resilience in those applications. Discrete event formulation of traffic flow at the level of individual vehicles affords both the flexibility of simulating complex scenarios of vehicular flow behavior as well as rapid simulation time advances. However, efficient parallel/distributed execution of the models becomes challenging due to synchronization overheads. Here, a parallel traffic simulation approach is presented that is aimed at reducing the time for simulating emergency vehicular traffic scenarios. Our approach resolves themore » challenges that arise in parallel execution of microscopic, vehicular-level models of traffic. We apply a reverse computation-based optimistic execution approach to address the parallel synchronization problem. This is achieved by formulating a reversible version of a discrete event model of vehicular traffic, and by utilizing this reversible model in an optimistic execution setting. Three unique aspects of this effort are: (1) exploration of optimistic simulation applied to vehicular traffic simulation (2) addressing reverse computation challenges specific to optimistic vehicular traffic simulation (3) achieving absolute (as opposed to self-relative) speedup with a sequential speed close to that of a fast, de facto standard sequential simulator for emergency traffic. The design and development of the parallel simulation system is presented, along with a performance study that demonstrates excellent sequential performance as well as parallel performance. The benefits of optimistic execution are demonstrated, including a speed up of nearly 20 on 32 processors observed on a vehicular network of over 65,000 intersections and over 13 million vehicles.« less
  • In complex phenomena such as epidemiological outbreaks, the intensity of inherent feedback effects and the significant role of transients in the dynamics make simulation the only effective method for proactive, reactive or post-facto analysis. The spatial scale, runtime speed, and behavioral detail needed in detailed simulations of epidemic outbreaks make it necessary to use large-scale parallel processing. Here, an optimistic parallel execution of a new discrete event formulation of a reaction-diffusion simulation model of epidemic propagation is presented to facilitate in dramatically increasing the fidelity and speed by which epidemiological simulations can be performed. Rollback support needed during optimistic parallelmore » execution is achieved by combining reverse computation with a small amount of incremental state saving. Parallel speedup of over 5,500 and other runtime performance metrics of the system are observed with weak-scaling execution on a small (8,192-core) Blue Gene / P system, while scalability with a weak-scaling speedup of over 10,000 is demonstrated on 65,536 cores of a large Cray XT5 system. Scenarios representing large population sizes exceeding several hundreds of millions of individuals in the largest cases are successfully exercised to verify model scalability.« less
  • This article gives a brief introduction to BIDDLE. The authors' aim here is to argue that the basic principle of BIDDLE are quite straightforward and that they can, indeed, design an architecture to directly execute Lisp in parallel. As mentioned in the introduction, important and interesting issues like side effect handling, object storage, environment maintenance etc. are not dealt with in great enough details.
  • This report contains a summary of the work done for the Dynax Project from 9/1/2012 to 8/31/2015. Much of the information presented is discussed in further detail in other STI annual and quarterly reports. Additionally, a NCE report was submitted covering work done from the period of 9/1/2015 to 12/31/2015.
  • In plasmas with strong field aligned currents, the most unstable mode is not always at parallel propagation, but may be at intermediate and very oblique angles. 2D particle simulations are performed in order to examine the interaction between the plasma waves at various angles and the electron and ion distributions in low {beta} collisionless plasmas with strong electron drifts. The parallel Buneman instability is known to arise in this situation, but the simulations demonstrate that the very oblique lower hybrid (LH) waves, until recently considered unimportant, may actually play a role just as significant as the waves at parallel propagation.more » The LH waves are energized by a current-driven linear instability, which may be seen as the oblique limit of the Buneman or ion-acoustic instability. The simulations resolve strong LH turbulence, substantial perpendicular ion tail heating and parallel electron heating. The combined action of parallel and oblique modes results in more complete electron relaxation than may be produced by the parallel Buneman instability alone.« less