skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Modern gyrokinetic particle-in-cell simulation of fusion plasmas on top supercomputers

Abstract

The Gyrokinetic Toroidal Code at Princeton (GTC-P) is a highly scalable and portable particle-in-cell (PIC) code. It solves the 5D Vlasov-Poisson equation featuring efficient utilization of modern parallel computer architectures at the petascale and beyond. Motivated by the goal of developing a modern code capable of dealing with the physics challenge of increasing problem size with sufficient resolution, new thread-level optimizations have been introduced as well as a key additional domain decomposition. GTC-P's multiple levels of parallelism, including inter-node 2D domain decomposition and particle decomposition, as well as intra-node shared memory partition and vectorization have enabled pushing the scalability of the PIC method to extreme computational scales. In this paper, we describe the methods developed to build a highly parallelized PIC code across a broad range of supercomputer designs. This particularly includes implementations on heterogeneous systems using NVIDIA GPU accelerators and Intel Xeon Phi (MIC) co-processors and performance comparisons with state-of-the-art homogeneous HPC systems such as Blue Gene/Q. New discovery science capabilities in the magnetic fusion energy application domain are enabled, including investigations of Ion-Temperature-Gradient (ITG) driven turbulence simulations with unprecedented spatial resolution and long temporal duration. Performance studies with realistic fusion experimental parameters are carried out on multiple supercomputingmore » systems spanning a wide range of cache capacities, cache-sharing configurations, memory bandwidth, interconnects and network topologies. Furthermore, these performance comparisons using a realistic discovery-science-capable domain application code provide valuable insights on optimization techniques across one of the broadest sets of current high-end computing platforms worldwide.« less

Authors:
 [1];  [2];  [3];  [4];  [5];  [4];  [4]
  1. Princeton Univ., Princeton, NJ (United States)
  2. Princeton Plasma Physics Lab. (PPPL), Princeton, NJ (United States)
  3. Princeton Univ., Princeton, NJ (United States); Princeton Plasma Physics Lab. (PPPL), Princeton, NJ (United States)
  4. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
  5. Pennsylvania State Univ., University Park, PA (United States)
Publication Date:
Research Org.:
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)
OSTI Identifier:
1398471
Grant/Contract Number:  
AC02-05CH11231
Resource Type:
Journal Article: Accepted Manuscript
Journal Name:
International Journal of High Performance Computing Applications
Additional Journal Information:
Journal Name: International Journal of High Performance Computing Applications; Journal ID: ISSN 1094-3420
Publisher:
SAGE
Country of Publication:
United States
Language:
English
Subject:
70 PLASMA PHYSICS AND FUSION TECHNOLOGY; 97 MATHEMATICS AND COMPUTING; Particle-in-cell methods; Vlasov–Poisson equations; NVIDIA GPU; Intel Xeon Phi (MIC); heterogeneous systems; fusion plasma simulations; extreme scale

Citation Formats

Wang, Bei, Ethier, Stephane, Tang, William, Ibrahim, Khaled Z., Madduri, Kamesh, Williams, Samuel, and Oliker, Leonid. Modern gyrokinetic particle-in-cell simulation of fusion plasmas on top supercomputers. United States: N. p., 2017. Web. doi:10.1177/1094342017712059.
Wang, Bei, Ethier, Stephane, Tang, William, Ibrahim, Khaled Z., Madduri, Kamesh, Williams, Samuel, & Oliker, Leonid. Modern gyrokinetic particle-in-cell simulation of fusion plasmas on top supercomputers. United States. doi:10.1177/1094342017712059.
Wang, Bei, Ethier, Stephane, Tang, William, Ibrahim, Khaled Z., Madduri, Kamesh, Williams, Samuel, and Oliker, Leonid. Thu . "Modern gyrokinetic particle-in-cell simulation of fusion plasmas on top supercomputers". United States. doi:10.1177/1094342017712059. https://www.osti.gov/servlets/purl/1398471.
@article{osti_1398471,
title = {Modern gyrokinetic particle-in-cell simulation of fusion plasmas on top supercomputers},
author = {Wang, Bei and Ethier, Stephane and Tang, William and Ibrahim, Khaled Z. and Madduri, Kamesh and Williams, Samuel and Oliker, Leonid},
abstractNote = {The Gyrokinetic Toroidal Code at Princeton (GTC-P) is a highly scalable and portable particle-in-cell (PIC) code. It solves the 5D Vlasov-Poisson equation featuring efficient utilization of modern parallel computer architectures at the petascale and beyond. Motivated by the goal of developing a modern code capable of dealing with the physics challenge of increasing problem size with sufficient resolution, new thread-level optimizations have been introduced as well as a key additional domain decomposition. GTC-P's multiple levels of parallelism, including inter-node 2D domain decomposition and particle decomposition, as well as intra-node shared memory partition and vectorization have enabled pushing the scalability of the PIC method to extreme computational scales. In this paper, we describe the methods developed to build a highly parallelized PIC code across a broad range of supercomputer designs. This particularly includes implementations on heterogeneous systems using NVIDIA GPU accelerators and Intel Xeon Phi (MIC) co-processors and performance comparisons with state-of-the-art homogeneous HPC systems such as Blue Gene/Q. New discovery science capabilities in the magnetic fusion energy application domain are enabled, including investigations of Ion-Temperature-Gradient (ITG) driven turbulence simulations with unprecedented spatial resolution and long temporal duration. Performance studies with realistic fusion experimental parameters are carried out on multiple supercomputing systems spanning a wide range of cache capacities, cache-sharing configurations, memory bandwidth, interconnects and network topologies. Furthermore, these performance comparisons using a realistic discovery-science-capable domain application code provide valuable insights on optimization techniques across one of the broadest sets of current high-end computing platforms worldwide.},
doi = {10.1177/1094342017712059},
journal = {International Journal of High Performance Computing Applications},
number = ,
volume = ,
place = {United States},
year = {Thu Jun 29 00:00:00 EDT 2017},
month = {Thu Jun 29 00:00:00 EDT 2017}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Save / Share: