EMPIRE-PIC: A Performance Portable Unstructured Particle-in-Cell Code

Journal Article · 31 May 2021 · Communications in Computational Physics

DOI:https://doi.org/10.4208/cicp.oa-2020-0261· OSTI ID:1822232

Bettencourt, Matthew T. ^[1]; Brown, Dominic A.S. ^[2]; Cartwright, Keith L. ^[1]; Cyr, Eric C. ^[1]; Glusa, Christian A. ^[1]; Lin, Paul T. ^[3]; Moore, Stan G. ^[1]; McGregor, Duncan A. O. ^[1]; Pawlowski, Roger P. ^[1]; Phillips, Edward G. ^[1]; Roberts, Nathan V. ^[1]; Wright, Steven A. ^[4]; Maheswaran, Satheesh ^[5]; Jones, John P. ^[6]; Jarvis, Stephen A. ^[7]

---

1. Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
2. Univ. of Warwick, Coventry (United Kingdom)
3. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
4. Univ. of York (United Kingdom)
5. Diamond Light Source Ltd., Didcot (United Kingdom)
6. Atomic Weapons Establishment, Aldermaston (United Kingdom)
7. Univ. of Birmingham (United Kingdom)

In this study we introduce EMPIRE-PIC, a finite element method particle-in-cell (FEM-PIC) application developed at Sandia National Laboratories. The code has been developed in C++ using the Trilinos library and the Kokkos Performance Portability Framework to enable running on multiple modern compute architectures while only requiring maintenance of a single codebase. EMPIRE-PIC is capable of solving both electrostatic and electromagnetic problems in two- and three-dimensions to second-order accuracy in space and time. In this paper we validate the code against three benchmark problems — a simple electron orbit, an electrostatic Langmuir wave, and a transverse electromagnetic wave propagating through a plasma. We demonstrate the performance of EMPIRE-PIC on four different architectures: Intel Haswell CPUs, Intel's Xeon Phi Knights Landing, ARM Thunder-X2 CPUs, and NVIDIA Tesla V100 GPUs attached to IBM POWER9 processors. This analysis demonstrates scalability of the code up to more than two thousand GPUs, and greater than one hundred thousand CPUs.

• Research Organization: Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States)
• Sponsoring Organization: USDOE National Nuclear Security Administration (NNSA)
• Grant/Contract Number: NA0003525
• OSTI ID: 1822232
• Report Number(s): SAND--2021-2806J; {694653,"Journal ID: ISSN 1815-2406"}
• Journal Information: Communications in Computational Physics, Journal Name: Communications in Computational Physics Journal Issue: 4 Vol. 30; ISSN 1815-2406
• Publisher: Global Science Press
• Country of Publication: United States
• Language: English

Related Subjects

97
PIC
electrostatics
electromagnetics
HPC
performance portability