skip to main content


Title: Modern gyrokinetic particle-in-cell simulation of fusion plasmas on top supercomputers

The Gyrokinetic Toroidal Code at Princeton (GTC-P) is a highly scalable and portable particle-in-cell (PIC) code. It solves the 5D Vlasov-Poisson equation featuring efficient utilization of modern parallel computer architectures at the petascale and beyond. Motivated by the goal of developing a modern code capable of dealing with the physics challenge of increasing problem size with sufficient resolution, new thread-level optimizations have been introduced as well as a key additional domain decomposition. GTC-P's multiple levels of parallelism, including inter-node 2D domain decomposition and particle decomposition, as well as intra-node shared memory partition and vectorization have enabled pushing the scalability of the PIC method to extreme computational scales. In this paper, we describe the methods developed to build a highly parallelized PIC code across a broad range of supercomputer designs. This particularly includes implementations on heterogeneous systems using NVIDIA GPU accelerators and Intel Xeon Phi (MIC) co-processors and performance comparisons with state-of-the-art homogeneous HPC systems such as Blue Gene/Q. New discovery science capabilities in the magnetic fusion energy application domain are enabled, including investigations of Ion-Temperature-Gradient (ITG) driven turbulence simulations with unprecedented spatial resolution and long temporal duration. Performance studies with realistic fusion experimental parameters are carried out on multiple supercomputingmore » systems spanning a wide range of cache capacities, cache-sharing configurations, memory bandwidth, interconnects and network topologies. Furthermore, these performance comparisons using a realistic discovery-science-capable domain application code provide valuable insights on optimization techniques across one of the broadest sets of current high-end computing platforms worldwide.« less
 [1] ;  [2] ;  [3] ;  [4] ;  [5] ;  [4] ;  [4]
  1. Princeton Univ., Princeton, NJ (United States)
  2. Princeton Plasma Physics Lab. (PPPL), Princeton, NJ (United States)
  3. Princeton Univ., Princeton, NJ (United States); Princeton Plasma Physics Lab. (PPPL), Princeton, NJ (United States)
  4. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
  5. Pennsylvania State Univ., University Park, PA (United States)
Publication Date:
Grant/Contract Number:
Accepted Manuscript
Journal Name:
International Journal of High Performance Computing Applications
Additional Journal Information:
Journal Name: International Journal of High Performance Computing Applications; Journal ID: ISSN 1094-3420
Research Org:
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Sponsoring Org:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)
Country of Publication:
United States
70 PLASMA PHYSICS AND FUSION TECHNOLOGY; 97 MATHEMATICS AND COMPUTING; Particle-in-cell methods; Vlasov–Poisson equations; NVIDIA GPU; Intel Xeon Phi (MIC); heterogeneous systems; fusion plasma simulations; extreme scale
OSTI Identifier: