Search Menu
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information
Search Scholarly Publications

Title: Modern gyrokinetic particle-in-cell simulation of fusion plasmas on top supercomputers

Journal Article · · International Journal of High Performance Computing Applications
 [1];  [2];  [3];  [4];  [5];  [4];  [4]
  1. Princeton Univ., Princeton, NJ (United States)
  2. Princeton Plasma Physics Lab. (PPPL), Princeton, NJ (United States)
  3. Princeton Univ., Princeton, NJ (United States); Princeton Plasma Physics Lab. (PPPL), Princeton, NJ (United States)
  4. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
  5. Pennsylvania State Univ., University Park, PA (United States)
+ Show Author Affiliations

The Gyrokinetic Toroidal Code at Princeton (GTC-P) is a highly scalable and portable particle-in-cell (PIC) code. It solves the 5D Vlasov-Poisson equation featuring efficient utilization of modern parallel computer architectures at the petascale and beyond. Motivated by the goal of developing a modern code capable of dealing with the physics challenge of increasing problem size with sufficient resolution, new thread-level optimizations have been introduced as well as a key additional domain decomposition. GTC-P's multiple levels of parallelism, including inter-node 2D domain decomposition and particle decomposition, as well as intra-node shared memory partition and vectorization have enabled pushing the scalability of the PIC method to extreme computational scales. In this paper, we describe the methods developed to build a highly parallelized PIC code across a broad range of supercomputer designs. This particularly includes implementations on heterogeneous systems using NVIDIA GPU accelerators and Intel Xeon Phi (MIC) co-processors and performance comparisons with state-of-the-art homogeneous HPC systems such as Blue Gene/Q. New discovery science capabilities in the magnetic fusion energy application domain are enabled, including investigations of Ion-Temperature-Gradient (ITG) driven turbulence simulations with unprecedented spatial resolution and long temporal duration. Performance studies with realistic fusion experimental parameters are carried out on multiple supercomputing systems spanning a wide range of cache capacities, cache-sharing configurations, memory bandwidth, interconnects and network topologies. Furthermore, these performance comparisons using a realistic discovery-science-capable domain application code provide valuable insights on optimization techniques across one of the broadest sets of current high-end computing platforms worldwide.

Research Organization:
Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)
Sponsoring Organization:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
Grant/Contract Number:
AC02-05CH11231
OSTI ID:
1398471
Journal Information:
International Journal of High Performance Computing Applications, Vol. 33, Issue 1; ISSN 1094-3420
Publisher:
SAGECopyright Statement
Country of Publication:
United States
Language:
English
Citation Metrics:
Cited by: 8 works
Citation information provided by
Web of Science

References (25)

A generalized weight-based particle-in-cell simulation scheme journal March 2011
A unified Monte Carlo interpretation of particle simulations and applications to non‐neutral plasmas journal April 1994
A Particle-in-cell Method with Adaptive Phase-space Remapping for Kinetic Plasmas journal January 2011
Gyrokinetic toroidal simulations on leading multi- and manycore HPC systems
  • Madduri, Kamesh; Ibrahim, Khaled Z.; Williams, Samuel
  • Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '11 https://doi.org/10.1145/2063384.2063415
conference January 2011
Long global gyrokinetic simulations: Source terms and particle noise control journal May 2008
Analysis and optimization of gyrokinetic toroidal simulations on homogenous and heterogenous platforms journal July 2013
An Adaptive, High-Order Phase-Space Remapping for the Two Dimensional Vlasov--Poisson Equations journal January 2012
Coarse-graining phase space in δf particle-in-cell simulations journal August 2007
A global collisionless PIC code in magnetic coordinates journal September 2007
Scientific Discovery in Fusion Plasma Turbulence Simulations at Extreme Scale journal September 2014
Plasma size and power scaling of ion temperature gradient driven turbulence journal February 2014
Gyrokinetic simulation of ion temperature gradient driven turbulence in 3D toroidal geometry journal September 1993
Gyrokinetic particle simulation model journal September 1987
Intel® Xeon Phi™ Coprocessor Architecture and Tools: The Guide for Application Developers book January 2013
Turbulent Transport Reduction by Zonal Flows: Massively Parallel Simulations journal September 1998
Large-scale gyrokinetic particle simulation of microturbulence in magnetically confined fusion plasmas journal January 2008
Plasma Physics via Computer Simulation book January 1991
System Size Effects on Gyrokinetic Turbulence journal October 2010
Comparisons and physics basis of tokamak transport models and turbulence simulations journal March 2000
Gyrokinetic particle-in-cell simulations of plasma microturbulence on advanced computing platforms journal January 2005
Full-f gyrokinetic particle simulation of centrally heated global ITG turbulence from magnetic axis to edge pedestal top in a realistic tokamak geometry journal September 2009
Size Scaling of Turbulent Transport in Magnetically Confined Plasmas journal April 2002
Kinetic turbulence simulations at extreme scale on leadership-class systems
  • Wang, Bei; Ethier, Stephane; Tang, William
  • Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '13 https://doi.org/10.1145/2503210.2503258
conference January 2013
Memory-efficient optimization of Gyrokinetic particle-to-grid interpolation for multicore processors
  • Madduri, Kamesh; Williams, Samuel; Ethier, Stéphane
  • Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis - SC '09 https://doi.org/10.1145/1654059.1654108
conference January 2009
Heterogeneous Programming and Optimization of Gyrokinetic Toroidal Code and Large-Scale Performance Test on TH-1A book January 2013

Figures / Tables (10)

Figure 1(p. 6)figure Figure 1
Figure 2(p. 7)figure Figure 2
Table 1(p. 9)table Table 1
Table 2(p. 13)table Table 2
Figure 3(p. 14)figure Figure 3
Table 3(p. 15)table Table 3
Figure 4(p. 16)figure Figure 4
Figure 5(p. 17)figure Figure 5
Table 4(p. 18)table Table 4
Figure 6(p. 19)figure Figure 6

Similar Records

Related Subjects

70 PLASMA PHYSICS AND FUSION TECHNOLOGY
97 MATHEMATICS AND COMPUTING
Particle-in-cell methods
Vlasov–Poisson equations
NVIDIA GPU
Intel Xeon Phi (MIC)
heterogeneous systems
fusion plasma simulations
extreme scale