Search Menu
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information
Search Scholarly Publications

Title: Modern gyrokinetic particle-in-cell simulation of fusion plasmas on top supercomputers

Abstract

The Gyrokinetic Toroidal Code at Princeton (GTC-P) is a highly scalable and portable particle-in-cell (PIC) code. It solves the 5D Vlasov-Poisson equation featuring efficient utilization of modern parallel computer architectures at the petascale and beyond. Motivated by the goal of developing a modern code capable of dealing with the physics challenge of increasing problem size with sufficient resolution, new thread-level optimizations have been introduced as well as a key additional domain decomposition. GTC-P's multiple levels of parallelism, including inter-node 2D domain decomposition and particle decomposition, as well as intra-node shared memory partition and vectorization have enabled pushing the scalability of the PIC method to extreme computational scales. In this paper, we describe the methods developed to build a highly parallelized PIC code across a broad range of supercomputer designs. This particularly includes implementations on heterogeneous systems using NVIDIA GPU accelerators and Intel Xeon Phi (MIC) co-processors and performance comparisons with state-of-the-art homogeneous HPC systems such as Blue Gene/Q. New discovery science capabilities in the magnetic fusion energy application domain are enabled, including investigations of Ion-Temperature-Gradient (ITG) driven turbulence simulations with unprecedented spatial resolution and long temporal duration. Performance studies with realistic fusion experimental parameters are carried out on multiple supercomputingmore » systems spanning a wide range of cache capacities, cache-sharing configurations, memory bandwidth, interconnects and network topologies. Furthermore, these performance comparisons using a realistic discovery-science-capable domain application code provide valuable insights on optimization techniques across one of the broadest sets of current high-end computing platforms worldwide.« less

Authors:
 [1];  [2];  [3];  [4];  [5];  [4];  [4]
+ Show Author Affiliations
  1. Princeton Univ., Princeton, NJ (United States)
  2. Princeton Plasma Physics Lab. (PPPL), Princeton, NJ (United States)
  3. Princeton Univ., Princeton, NJ (United States); Princeton Plasma Physics Lab. (PPPL), Princeton, NJ (United States)
  4. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
  5. Pennsylvania State Univ., University Park, PA (United States)
Publication Date:
Research Org.:
Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
OSTI Identifier:
1398471
Grant/Contract Number:  
AC02-05CH11231
Resource Type:
Accepted Manuscript
Journal Name:
International Journal of High Performance Computing Applications
Additional Journal Information:
Journal Volume: 33; Journal Issue: 1; Journal ID: ISSN 1094-3420
Publisher:
SAGE
Country of Publication:
United States
Language:
English
Subject:
70 PLASMA PHYSICS AND FUSION TECHNOLOGY; 97 MATHEMATICS AND COMPUTING; Particle-in-cell methods; Vlasov–Poisson equations; NVIDIA GPU; Intel Xeon Phi (MIC); heterogeneous systems; fusion plasma simulations; extreme scale

Citation Formats

Wang, Bei, Ethier, Stephane, Tang, William, Ibrahim, Khaled Z., Madduri, Kamesh, Williams, Samuel, and Oliker, Leonid. Modern gyrokinetic particle-in-cell simulation of fusion plasmas on top supercomputers. United States: N. p., 2017. Web. doi:10.1177/1094342017712059.
Copy to clipboard
Wang, Bei, Ethier, Stephane, Tang, William, Ibrahim, Khaled Z., Madduri, Kamesh, Williams, Samuel, & Oliker, Leonid. Modern gyrokinetic particle-in-cell simulation of fusion plasmas on top supercomputers. United States. https://doi.org/10.1177/1094342017712059
Copy to clipboard
Wang, Bei, Ethier, Stephane, Tang, William, Ibrahim, Khaled Z., Madduri, Kamesh, Williams, Samuel, and Oliker, Leonid. Thu . "Modern gyrokinetic particle-in-cell simulation of fusion plasmas on top supercomputers". United States. https://doi.org/10.1177/1094342017712059. https://www.osti.gov/servlets/purl/1398471.
Copy to clipboard
@article{osti_1398471,
title = {Modern gyrokinetic particle-in-cell simulation of fusion plasmas on top supercomputers},
author = {Wang, Bei and Ethier, Stephane and Tang, William and Ibrahim, Khaled Z. and Madduri, Kamesh and Williams, Samuel and Oliker, Leonid},
abstractNote = {The Gyrokinetic Toroidal Code at Princeton (GTC-P) is a highly scalable and portable particle-in-cell (PIC) code. It solves the 5D Vlasov-Poisson equation featuring efficient utilization of modern parallel computer architectures at the petascale and beyond. Motivated by the goal of developing a modern code capable of dealing with the physics challenge of increasing problem size with sufficient resolution, new thread-level optimizations have been introduced as well as a key additional domain decomposition. GTC-P's multiple levels of parallelism, including inter-node 2D domain decomposition and particle decomposition, as well as intra-node shared memory partition and vectorization have enabled pushing the scalability of the PIC method to extreme computational scales. In this paper, we describe the methods developed to build a highly parallelized PIC code across a broad range of supercomputer designs. This particularly includes implementations on heterogeneous systems using NVIDIA GPU accelerators and Intel Xeon Phi (MIC) co-processors and performance comparisons with state-of-the-art homogeneous HPC systems such as Blue Gene/Q. New discovery science capabilities in the magnetic fusion energy application domain are enabled, including investigations of Ion-Temperature-Gradient (ITG) driven turbulence simulations with unprecedented spatial resolution and long temporal duration. Performance studies with realistic fusion experimental parameters are carried out on multiple supercomputing systems spanning a wide range of cache capacities, cache-sharing configurations, memory bandwidth, interconnects and network topologies. Furthermore, these performance comparisons using a realistic discovery-science-capable domain application code provide valuable insights on optimization techniques across one of the broadest sets of current high-end computing platforms worldwide.},
doi = {10.1177/1094342017712059},
journal = {International Journal of High Performance Computing Applications},
number = 1,
volume = 33,
place = {United States},
year = {Thu Jun 29 00:00:00 EDT 2017},
month = {Thu Jun 29 00:00:00 EDT 2017}
}
Copy to clipboard
Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
https://doi.org/10.1177/1094342017712059
Copyright Statement
Other availability
Search WorldCat Search WorldCat to find libraries that may hold this journal
Citation Metrics:
Cited by: 8 works
Citation information provided by
Web of Science

Figures / Tables:

Figure 1 Figure 1: An illustration of 3D toroidal grid.
All figures and tables (10 total)
Save / Share:
Export Metadata
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.

Works referenced in this record:

A generalized weight-based particle-in-cell simulation scheme
journal, March 2011

A unified Monte Carlo interpretation of particle simulations and applications to non‐neutral plasmas
journal, April 1994

A Particle-in-cell Method with Adaptive Phase-space Remapping for Kinetic Plasmas
journal, January 2011

  • Wang, B.; Miller, G. H.; Colella, P.
  • SIAM Journal on Scientific Computing, Vol. 33, Issue 6
  • DOI: 10.1137/100811805

Gyrokinetic toroidal simulations on leading multi- and manycore HPC systems
conference, January 2011

  • Madduri, Kamesh; Ibrahim, Khaled Z.; Williams, Samuel
  • Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '11
  • DOI: 10.1145/2063384.2063415

Long global gyrokinetic simulations: Source terms and particle noise control
journal, May 2008

  • McMillan, B. F.; Jolliet, S.; Tran, T. M.
  • Physics of Plasmas, Vol. 15, Issue 5
  • DOI: 10.1063/1.2921792

Analysis and optimization of gyrokinetic toroidal simulations on homogenous and heterogenous platforms
journal, July 2013

  • Ibrahim, Khaled Z.; Madduri, Kamesh; Williams, Samuel
  • The International Journal of High Performance Computing Applications, Vol. 27, Issue 4
  • DOI: 10.1177/1094342013492446

An Adaptive, High-Order Phase-Space Remapping for the Two Dimensional Vlasov--Poisson Equations
journal, January 2012

  • Wang, Bei; Miller, Greg; Colella, Phil
  • SIAM Journal on Scientific Computing, Vol. 34, Issue 6
  • DOI: 10.1137/120872954

Coarse-graining phase space in δf particle-in-cell simulations
journal, August 2007

  • Chen, Yang; Parker, Scott E.
  • Physics of Plasmas, Vol. 14, Issue 8
  • DOI: 10.1063/1.2751603

A global collisionless PIC code in magnetic coordinates
journal, September 2007

Scientific Discovery in Fusion Plasma Turbulence Simulations at Extreme Scale
journal, September 2014

  • Tang, William; Wang, Bei; Ethier, Stephane
  • Computing in Science & Engineering, Vol. 16, Issue 5
  • DOI: 10.1109/MCSE.2014.54

Plasma size and power scaling of ion temperature gradient driven turbulence
journal, February 2014

  • Idomura, Yasuhiro; Nakata, Motoki
  • Physics of Plasmas, Vol. 21, Issue 2
  • DOI: 10.1063/1.4867379

Gyrokinetic simulation of ion temperature gradient driven turbulence in 3D toroidal geometry
journal, September 1993

Gyrokinetic particle simulation model
journal, September 1987

Turbulent Transport Reduction by Zonal Flows: Massively Parallel Simulations
journal, September 1998

Large-scale gyrokinetic particle simulation of microturbulence in magnetically confined fusion plasmas
journal, January 2008

  • Ethier, S.; Tang, W. M.; Walkup, R.
  • IBM Journal of Research and Development, Vol. 52, Issue 1.2
  • DOI: 10.1147/rd.521.0105

Plasma Physics via Computer Simulation
book, January 1991

Comparisons and physics basis of tokamak transport models and turbulence simulations
journal, March 2000

  • Dimits, A. M.; Bateman, G.; Beer, M. A.
  • Physics of Plasmas, Vol. 7, Issue 3
  • DOI: 10.1063/1.873896

Gyrokinetic particle-in-cell simulations of plasma microturbulence on advanced computing platforms
journal, January 2005

Size Scaling of Turbulent Transport in Magnetically Confined Plasmas
journal, April 2002

Kinetic turbulence simulations at extreme scale on leadership-class systems
conference, January 2013

  • Wang, Bei; Ethier, Stephane; Tang, William
  • Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '13
  • DOI: 10.1145/2503210.2503258

Memory-efficient optimization of Gyrokinetic particle-to-grid interpolation for multicore processors
conference, January 2009

  • Madduri, Kamesh; Williams, Samuel; Ethier, Stéphane
  • Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis - SC '09
  • DOI: 10.1145/1654059.1654108

Heterogeneous Programming and Optimization of Gyrokinetic Toroidal Code and Large-Scale Performance Test on TH-1A
book, January 2013

    Sort by:
    [ × clear filter / sort ]

    Figures / Tables found in this record:

    Figure 1 (p. 6)figure
    Figure 2 (p. 7)figure
    Table 1 (p. 9)table
    Table 2 (p. 13)table
    Figure 3 (p. 14)figure
    Table 3 (p. 15)table
    Figure 4 (p. 16)figure
    Figure 5 (p. 17)figure
    Table 4 (p. 18)table
    Figure 6 (p. 19)figure
      [ × clear filter / sort ]
      Figures/Tables have been extracted from DOE-funded journal article accepted manuscripts.

      Similar Records in DOE PAGES and OSTI.GOV collections: