Analysis and optimization of gyrokinetic toroidal simulations on homogenous and heterogenous platforms

Ibrahim, Khaled Z.; Madduri, Kamesh; Williams, Samuel; Wang, Bei; Ethier, Stephane; Oliker, Leonid

doi:10.1177/1094342013492446

Analysis and optimization of gyrokinetic toroidal simulations on homogenous and heterogenous platforms

Journal Article · 18 July 2013 · International Journal of High Performance Computing Applications

DOI:https://doi.org/10.1177/1094342013492446· OSTI ID:1407202

Ibrahim, Khaled Z. ^[1]; Madduri, Kamesh ^[2]; Williams, Samuel ^[1]; Wang, Bei ^[3]; Ethier, Stephane ^[4]; Oliker, Leonid ^[1]

Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Pennsylvania State Univ., University Park, PA (United States)
Princeton Univ., NJ (United States). Princeton Inst. of Computational Science and Engineering
Princeton Plasma Physics Lab. (PPPL), Princeton, NJ (United States)

The Gyrokinetic Toroidal Code (GTC) uses the particle-in-cell method to efficiently simulate plasma microturbulence. This paper presents novel analysis and optimization techniques to enhance the performance of GTC on large-scale machines. We introduce cell access analysis to better manage locality vs. synchronization tradeoffs on CPU and GPU-based architectures. Finally, our optimized hybrid parallel implementation of GTC uses MPI, OpenMP, and NVIDIA CUDA, achieves up to a 2× speedup over the reference Fortran version on multiple parallel systems, and scales efficiently to tens of thousands of cores.

View Accepted Manuscript (DOE)

Research Organization:: Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)

Grant/Contract Number:: AC02-05CH11231; AC02-09CH11466; AC02-06CH11357; AC05-00OR22725

OSTI ID:: 1407202

Journal Information:: International Journal of High Performance Computing Applications, Journal Name: International Journal of High Performance Computing Applications Journal Issue: 4 Vol. 27; ISSN 1094-3420

Publisher:: SAGECopyright Statement

Country of Publication:: United States

Language:: English

References (20)

On parallel PIC versatility and the structure of parallel PIC approaches Carmona, Edward A.; Chandler, Leon J. Concurrency: Practice and Experience, Vol. 9, Issue 12 https://doi.org/10.1002/(SICI)1096-9128(199712)9:12<1377::AID-CPE284>3.0.CO;2-Q	journal	December 1997
Accelerating a Particle-in-Cell Simulation Using a Hybrid Counting Sort Bowers, K. J. Journal of Computational Physics, Vol. 173, Issue 2 https://doi.org/10.1006/jcph.2001.6851	journal	November 2001
Gyrokinetic particle simulation model Lee, W. W. Journal of Computational Physics, Vol. 72, Issue 1 https://doi.org/10.1016/0021-9991(87)90080-5	journal	September 1987
Method to increase the simulation speed of particle-in-cell (PIC) code Shon, C. H.; Lee, H. J.; Lee, J. K. Computer Physics Communications, Vol. 141, Issue 3 https://doi.org/10.1016/S0010-4655(01)00417-9	journal	December 2001
UPIC: A framework for massively parallel particle-in-cell codes Decyk, Viktor K. Computer Physics Communications, Vol. 177, Issue 1-2 https://doi.org/10.1016/j.cpc.2007.02.092	journal	July 2007
Adaptable Particle-in-Cell algorithms for graphical processing units Decyk, Viktor K.; Singh, Tajendra V. Computer Physics Communications, Vol. 182, Issue 3 https://doi.org/10.1016/j.cpc.2010.11.009	journal	March 2011
VORPAL: a versatile plasma simulation code Nieter, Chet; Cary, John R. Journal of Computational Physics, Vol. 196, Issue 2 https://doi.org/10.1016/j.jcp.2003.11.004	journal	May 2004
A finite element Poisson solver for gyrokinetic particle simulations in a global field aligned mesh Nishimura, Y.; Lin, Z.; Lewandowski, J. L. V. Journal of Computational Physics, Vol. 214, Issue 2 https://doi.org/10.1016/j.jcp.2005.10.011	journal	May 2006
QUICKPIC: A highly efficient particle-in-cell code for modeling wakefield acceleration in plasmas Huang, C.; Decyk, V. K.; Ren, C. Journal of Computational Physics, Vol. 217, Issue 2 https://doi.org/10.1016/j.jcp.2006.01.039	journal	September 2006
Fast parallel Particle-To-Grid interpolation for plasma PIC simulations on the GPU Stantchev, George; Dorland, William; Gumerov, Nail Journal of Parallel and Distributed Computing, Vol. 68, Issue 10 https://doi.org/10.1016/j.jpdc.2008.05.009	journal	October 2008
Cosmological N-Body Simulations Bertschinger, Edmund; Gelb, James M. Computers in Physics, Vol. 5, Issue 2 https://doi.org/10.1063/1.4822978	journal	January 1991
Method for solving the gyrokinetic Poisson equation in general geometry Lin, Z.; Lee, W. W. Physical Review E, Vol. 52, Issue 5 https://doi.org/10.1103/PhysRevE.52.5646	journal	November 1995
Scientific Computations on Modern Parallel Vector Systems Oliker, L.; Canning, A.; Carter, J. Proceedings of the ACM/IEEE SC2004 Conference https://doi.org/10.1109/SC.2004.54	conference	January 2004
0.374 Pflop/s trillion-particle kinetic modeling of laser plasma interaction on roadrunner Bowers, K. J.; Albright, B. J.; Bergen, B. 2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1109/SC.2008.5222734	conference	November 2008
Turbulent Transport Reduction by Zonal Flows: Massively Parallel Simulations Lin, Z. Science, Vol. 281, Issue 5384 https://doi.org/10.1126/science.281.5384.1835	journal	September 1998
OhHelp: a scalable domain-decomposing dynamic load balancing for particle-in-cell simulations Nakashima, Hiroshi; Miyake, Yohei; Usui, Hideyuki Proceedings of the 23rd international conference on Conference on Supercomputing - ICS '09 https://doi.org/10.1145/1542275.1542293	conference	January 2009
Memory-efficient optimization of Gyrokinetic particle-to-grid interpolation for multicore processors Madduri, Kamesh; Williams, Samuel; Ethier, Stéphane Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis - SC '09 https://doi.org/10.1145/1654059.1654108	conference	January 2009
Gyrokinetic toroidal simulations on leading multi- and manycore HPC systems Madduri, Kamesh; Ibrahim, Khaled Z.; Williams, Samuel Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '11 https://doi.org/10.1145/2063384.2063415	conference	January 2011
Particle-in-cell simulation codes in High Performance Fortran Akarsu, Erol; Dincer, Kivanc; Haupt, Tomasz Proceedings of the 1996 ACM/IEEE conference on Supercomputing (CDROM) - Supercomputing '96 https://doi.org/10.1145/369028.369108	conference	January 1996
Large-scale gyrokinetic particle simulation of microturbulence in magnetically confined fusion plasmas Ethier, S.; Tang, W. M.; Walkup, R. IBM Journal of Research and Development, Vol. 52, Issue 1.2 https://doi.org/10.1147/rd.521.0105	journal	January 2008

Similar Records

Gyrokinetic toroidal simulations on leading multi- and manycore HPC systems

Conference · 2010 · OSTI ID:1407109

Hands-on Performance Tuning of 3D Finite Difference Earthquake Simulation on GPU Fermi Chipset

Journal Article · 2012 · Procedia Computer Science · OSTI ID:1567289

Gyrokinetic particle-in-cell optimization on emerging multi- and manycore platforms

Journal Article · 2011 · Parallel Computing · OSTI ID:1407105

Related Subjects

97 MATHEMATICS AND COMPUTING

Analysis and optimization of gyrokinetic toroidal simulations on homogenous and heterogenous platforms

Citation Formats

References (20)

Similar Records

Related Subjects