Extreme Scale Plasma Turbulence Simulations on Top Supercomputers Worldwide

Tang, William; Wang, Bei; Ethier, Stephane; Kwasniewski, Grzegorz; Hoefler, Torsten; Ibrahim, Khaled Z.; Madduri, Kamesh; Williams, Samuel; Oliker, Leonid; Rosales-Fernandez, Carlos; Williams, Tim

doi:10.1109/SC.2016.42

Title: Extreme Scale Plasma Turbulence Simulations on Top Supercomputers Worldwide

Full Record
Other Related Research

Abstract

The goal of the extreme scale plasma turbulence studies described in this paper is to expedite the delivery of reliable predictions on confinement physics in large magnetic fusion systems by using world-class supercomputers to carry out simulations with unprecedented resolution and temporal duration. This has involved architecture-dependent optimizations of performance scaling and addressing code portability and energy issues, with the metrics for multi-platform comparisons being 'time-to-solution' and 'energy-to-solution'. Realistic results addressing how confinement losses caused by plasma turbulence scale from present-day devices to the much larger $25 billion international ITER fusion facility have been enabled by innovative advances in the GTC-P code including (i) implementation of one-sided communication from MPI 3.0 standard; (ii) creative optimization techniques on Xeon Phi processors; and (iii) development of a novel performance model for the key kernels of the PIC code. Our results show that modeling data movement is sufficient to predict performance on modern supercomputer platforms.

Authors:

Tang, William ^[1]; Wang, Bei ^[2]; Ethier, Stephane ^[3]; Kwasniewski, Grzegorz ^[4]; Hoefler, Torsten ^[4]; Ibrahim, Khaled Z. ^[5]; Madduri, Kamesh ^[6]; Williams, Samuel ^[5]; Oliker, Leonid ^[5]; Rosales-Fernandez, Carlos ^[7]; Williams, Tim ^[8]

Princeton Univ., NJ (United States). Princeton Inst. for Computational Science and Engineering; Princeton Plasma Physics Lab. (PPPL), Princeton, NJ (United States)
Princeton Univ., NJ (United States). Princeton Inst. for Computational Science and Engineering
Princeton Plasma Physics Lab. (PPPL), Princeton, NJ (United States)
ETH Zurich (Switzerland). Computer Science Dept.
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). Computational Research Division
Pennsylvania State Univ., University Park, PA (United States). Computer Science and Engineering Dept.
Univ. of Texas, Austin, TX (United States). Texas Advanced Computing Center
Argonne National Lab. (ANL), Argonne, IL (United States). Argonne Leadership Comuting Facility

Publication Date:: Tue Nov 01 00:00:00 EDT 2016

Research Org.:: Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)

Sponsoring Org.:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)

OSTI Identifier:: 1379775

Grant/Contract Number:: AC02-05CH11231; AC02-06CH11357; AC02-09CH11466; AC02-00OR22725

Resource Type:: Accepted Manuscript

Journal Name:: International Conference for High Performance Computing, Networking, Storage and Analysis (Online)

Additional Journal Information:: Journal Name: International Conference for High Performance Computing, Networking, Storage and Analysis (Online); Journal Volume: 2017; Conference: International Conference for High Performance Computing, Networking, Storage and Analysis, Salt Lake City, UT (United States), 13-18 Nov 2016; Journal ID: ISSN 2167-4337

Publisher:: IEEE

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING

Citation Formats


                    Tang, William, Wang, Bei, Ethier, Stephane, Kwasniewski, Grzegorz, Hoefler, Torsten, Ibrahim, Khaled Z., Madduri, Kamesh, Williams, Samuel, Oliker, Leonid, Rosales-Fernandez, Carlos, and Williams, Tim. Extreme Scale Plasma Turbulence Simulations on Top Supercomputers Worldwide.  United States: N. p., 2016. 
Web.  doi:10.1109/SC.2016.42.

Copy to clipboard


                    Tang, William, Wang, Bei, Ethier, Stephane, Kwasniewski, Grzegorz, Hoefler, Torsten, Ibrahim, Khaled Z., Madduri, Kamesh, Williams, Samuel, Oliker, Leonid, Rosales-Fernandez, Carlos, & Williams, Tim. Extreme Scale Plasma Turbulence Simulations on Top Supercomputers Worldwide.  United States.  https://doi.org/10.1109/SC.2016.42

Copy to clipboard


                    Tang, William, Wang, Bei, Ethier, Stephane, Kwasniewski, Grzegorz, Hoefler, Torsten, Ibrahim, Khaled Z., Madduri, Kamesh, Williams, Samuel, Oliker, Leonid, Rosales-Fernandez, Carlos, and Williams, Tim. Tue .  
"Extreme Scale Plasma Turbulence Simulations on Top Supercomputers Worldwide".  United States.  https://doi.org/10.1109/SC.2016.42.  https://www.osti.gov/servlets/purl/1379775.

Copy to clipboard


                    
@article{osti_1379775,

  title        = {Extreme Scale Plasma Turbulence Simulations on Top Supercomputers Worldwide},

  author       = {Tang, William and Wang, Bei and Ethier, Stephane and Kwasniewski, Grzegorz and Hoefler, Torsten and Ibrahim, Khaled Z. and Madduri, Kamesh and Williams, Samuel and Oliker, Leonid and Rosales-Fernandez, Carlos and Williams, Tim},

  abstractNote = {The goal of the extreme scale plasma turbulence studies described in this paper is to expedite the delivery of reliable predictions on confinement physics in large magnetic fusion systems by using world-class supercomputers to carry out simulations with unprecedented resolution and temporal duration. This has involved architecture-dependent optimizations of performance scaling and addressing code portability and energy issues, with the metrics for multi-platform comparisons being 'time-to-solution' and 'energy-to-solution'. Realistic results addressing how confinement losses caused by plasma turbulence scale from present-day devices to the much larger $25 billion international ITER fusion facility have been enabled by innovative advances in the GTC-P code including (i) implementation of one-sided communication from MPI 3.0 standard; (ii) creative optimization techniques on Xeon Phi processors; and (iii) development of a novel performance model for the key kernels of the PIC code. Our results show that modeling data movement is sufficient to predict performance on modern supercomputer platforms.},

  doi          = {10.1109/SC.2016.42},

  journal      = {International Conference for High Performance Computing, Networking, Storage and Analysis (Online)},

  number       = ,

  volume       = 2017,

  place        = {United States},

  year         = {Tue Nov 01 00:00:00 EDT 2016},

  month        = {Tue Nov 01 00:00:00 EDT 2016}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1109/SC.2016.42

Other availability

Search WorldCat to find libraries that may hold this journal

Save / Share:

Export Metadata

Save to My Library

Similar Records in DOE PAGES and OSTI.GOV collections:

Modern gyrokinetic particle-in-cell simulation of fusion plasmas on top supercomputers

Journal Article Wang, Bei ; Ethier, Stephane ; Tang, William ; ... - International Journal of High Performance Computing Applications

The Gyrokinetic Toroidal Code at Princeton (GTC-P) is a highly scalable and portable particle-in-cell (PIC) code. It solves the 5D Vlasov-Poisson equation featuring efficient utilization of modern parallel computer architectures at the petascale and beyond. Motivated by the goal of developing a modern code capable of dealing with the physics challenge of increasing problem size with sufficient resolution, new thread-level optimizations have been introduced as well as a key additional domain decomposition. GTC-P's multiple levels of parallelism, including inter-node 2D domain decomposition and particle decomposition, as well as intra-node shared memory partition and vectorization have enabled pushing the scalability ofmore »« less
Cited by 8
https://doi.org/10.1177/1094342017712059

Full Text Available
Memory-efficient optimization of Gyrokinetic particle-to-grid interpolation for multicore processors

Conference Madduri, Kamesh ; Williams, Samuel ; Ethier, Stephane ; ...

We present multicore parallelization strategies for the particle-to-grid interpolation step in the Gyrokinetic Toroidal Code (GTC), a 3D particle-in-cell (PIC) application to study turbulent transport in magnetic-confinement fusion devices. Particle-grid interpolation is a known performance bottleneck in several PIC applications. In GTC, this step involves particles depositing charges to a 3D toroidal mesh, and multiple particles may contribute to the charge at a grid point. We design new parallel algorithms for the GTC charge deposition kernel, and analyze their performance on three leading multicore platforms. We implement thirteen different variants for this kernel and identify the best-performing ones given typicalmore »« less
https://doi.org/10.1145/1654059.1654108

Full Text Available
Kinetic turbulence simulations at extreme scale on leadership-class systems

Conference Wang, Bei ; Ethier, Stephane ; Tang, William ; ...

Reliable predictive simulation capability addressing confinement properties in magnetically confined fusion plasmas is critically-important for ITER, a 20 billion dollar international burning plasma device under construction in France. The complex study of kinetic turbulence, which can severely limit the energy confinement and impact the economic viability of fusion systems, requires simulations at extreme scale for such an unprecedented device size. Our newly optimized, global, ab initio particle-in-cell code solving the nonlinear equations underlying gyrokinetic theory achieves excellent performance with respect to "time to solution" at the full capacity of the IBM Blue Gene/Q on 786,432 cores of Mira at ALCFmore »« less
https://doi.org/10.1145/2503210.2503258

Full Text Available
Roofline Analysis in the Intel® Advisor to Deliver Optimized Performance for applications on Intel® Xeon Phi™ Processor

Conference Koskela, Tuomas S. ; Lobet, Mathieu ; Deslippe, Jack ; ...

In this session we show, in two case studies, how the roofline feature of Intel Advisor has been utilized to optimize the performance of kernels of the XGC1 and PICSAR codes in preparation for Intel Knights Landing architecture. The impact of the implemented optimizations and the benefits of using the automatic roofline feature of Intel Advisor to study performance of large applications will be presented. This demonstrates an effective optimization strategy that has enabled these science applications to achieve up to 4.6 times speed-up and prepare for future exascale architectures. # Goal/Relevance of Session The roofline model [1,2] is amore »« less
Full Text Available
Performance Portability of HPC Discovery Science Software: Fusion Energy Turbulence Simulations at Extreme Scale

Journal Article Tang, William ; Want, Bei ; Ethier, Stephane ; ... - Supercomputing frontiers and innovations

As HPC R&D moves forward on a variety of “path to exascale” architectures today, an associated objective is to demonstrate performance portability of discovery-science-capable software. Important application domains, such as Magnetic Fusion Energy (MFE), have improved modelling of increasingly complex physical systems -- especially with respect to reducing “time-to-solution” as well as “energy to solution.” The emergence of new insights on confinement scaling in MFE systems has been aided significantly by efficient software capable of harnessing powerful supercomputers to carry out simulations with unprecedented resolution and temporal duration to address increasing problem sizes. Specifically, highly scalable particle-in-cell (PIC) programing methodologymore »« less
https://doi.org/10.14529/jsfi170105

Full Text Available

Similar Records