Modeling of a chain of three plasma accelerator stages with the WarpX electromagnetic PIC code on GPUs

Vay, J. -L.; Huebl, A.; Almgren, A.; Amorim, L. D.; Bell, J.; Fedeli, L.; Ge, L.; Gott, K.; Grote, D. P.; Hogan, M.; Jambunathan, R.; Lehe, R.; Myers, A.; Ng, C.; Rowan, M.; Shapoval, O.; Thévenet, M.; Vincenti, H.; Yang, E.; Zaïm, N.; Zhang, W.; Zhao, Y.; Zoni, E.

doi:10.1063/5.0028512

Modeling of a chain of three plasma accelerator stages with the WarpX electromagnetic PIC code on GPUs

Journal Article · Tue Feb 09 04:00:00 EST 2021 · Physics of Plasmas

DOI:https://doi.org/10.1063/5.0028512· OSTI ID:2567563

; ; Almgren, A.; ; Bell, J.; Fedeli, L.; Ge, L.; Gott, K.; ; Hogan, M.; Jambunathan, R.; Lehe, R.; Myers, A.; Ng, C.; Rowan, M.; Shapoval, O.; Thévenet, M.; Vincenti, H.; Yang, E.; Zaïm, N. more »

The fully electromagnetic particle-in-cell code WarpX is being developed by a team of the U.S. DOE Exascale Computing Project (with additional non-U.S. collaborators on part of the code) to enable the modeling of chains of tens to hundreds of plasma accelerator stages on exascale supercomputers, for future collider designs. The code is combining the latest algorithmic advances (e.g., Lorentz boosted frame and pseudo-spectral Maxwell solvers) with mesh refinement and runs on the latest computer processing unit and graphical processing unit (GPU) architectures. In this paper, we summarize the strategy that was adopted to port WarpX to GPUs, report on the weak parallel scaling of the pseudo-spectral electromagnetic solver, and then present solutions for decreasing the time spent in data exchanges from guard regions between subdomains. In Sec. IV, we demonstrate the simulations of a chain of three consecutive multi-GeV laser-driven plasma accelerator stages.

View Journal Article

Sponsoring Organization:: USDOE

Grant/Contract Number:: AC02-05CH11231; AC02-76SF00515; AC52-07NA27344

OSTI ID:: 2567563

Alternate ID(s):: OSTI ID: 1765030
OSTI ID: 1782089
OSTI ID: 1787210

Journal Information:: Physics of Plasmas, Journal Name: Physics of Plasmas Journal Issue: 2 Vol. 28; ISSN 1070-664X

Publisher:: American Institute of PhysicsCopyright Statement

Country of Publication:: United States

Language:: English

References (29)

Numerical Cherenkov instabilities in electromagnetic particle codes Godfrey, Brendan B. Journal of Computational Physics, Vol. 15, Issue 4 https://doi.org/10.1016/0021-9991(74)90076-X	journal	August 1974
Asymmetric PML for the absorption of waves. Application to mesh refinement in electromagnetic Particle-In-Cell plasma simulations Vay, J. -L.; Adam, J. -C.; Héron, A. Computer Physics Communications, Vol. 164, Issue 1-3 https://doi.org/10.1016/j.cpc.2004.06.026	journal	December 2004
Detailed analysis of the effects of stencil spatial variations with arbitrary high-order finite-difference Maxwell solver Vincenti, H.; Vay, J. -L. Computer Physics Communications, Vol. 200 https://doi.org/10.1016/j.cpc.2015.11.009	journal	March 2016
Ultrahigh-order Maxwell solver with extreme scalability for electromagnetic PIC simulations of plasmas Vincenti, Henri; Vay, Jean-Luc Computer Physics Communications, Vol. 228 https://doi.org/10.1016/j.cpc.2018.03.018	journal	July 2018
A generalized massively parallel ultra-high order FFT-based Maxwell solver Kallala, Haithem; Vay, Jean-Luc; Vincenti, Henri Computer Physics Communications, Vol. 244 https://doi.org/10.1016/j.cpc.2019.07.009	journal	November 2019
Numerical methods for instability mitigation in the modeling of laser wakefield accelerators in a Lorentz-boosted frame Vay, J. -L.; Geddes, C. G. R.; Cormier-Michel, E. Journal of Computational Physics, Vol. 230, Issue 15 https://doi.org/10.1016/j.jcp.2011.04.003	journal	July 2011
A domain decomposition method for pseudo-spectral electromagnetic simulations of plasmas Vay, Jean-Luc; Haber, Irving; Godfrey, Brendan B. Journal of Computational Physics, Vol. 243 https://doi.org/10.1016/j.jcp.2013.03.010	journal	June 2013
Numerical stability analysis of the pseudo-spectral analytical time-domain PIC algorithm Godfrey, Brendan B.; Vay, Jean-Luc; Haber, Irving Journal of Computational Physics, Vol. 258 https://doi.org/10.1016/j.jcp.2013.10.053	journal	February 2014
Suppressing the numerical Cherenkov instability in FDTD PIC codes Godfrey, Brendan B.; Vay, Jean-Luc Journal of Computational Physics, Vol. 267 https://doi.org/10.1016/j.jcp.2014.02.022	journal	June 2014
Kokkos: Enabling manycore performance portability through polymorphic memory access patterns Carter Edwards, H.; Trott, Christian R.; Sunderland, Daniel Journal of Parallel and Distributed Computing, Vol. 74, Issue 12 https://doi.org/10.1016/j.jpdc.2014.07.003	journal	December 2014
Warp-X: A new exascale computing platform for beam–plasma simulations Vay, J. -L.; Almgren, A.; Bell, J. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, Vol. 909 https://doi.org/10.1016/j.nima.2018.01.035	journal	November 2018
Three-dimensional electromagnetic relativistic particle-in-cell code VLPL (Virtual Laser Plasma Lab) Pukhov, A. Journal of Plasma Physics, Vol. 61, Issue 3 https://doi.org/10.1017/S0022377899007515	journal	April 1999
Modeling of 10 GeV-1 TeV laser-plasma accelerators using Lorentz boosted simulations Vay, J. -L.; Geddes, C. G. R.; Esarey, E. Physics of Plasmas, Vol. 18, Issue 12 https://doi.org/10.1063/1.3663841	journal	December 2011
Stable discrete representation of relativistically drifting plasmas Kirchen, M.; Lehe, R.; Godfrey, B. B. Physics of Plasmas, Vol. 23, Issue 10 https://doi.org/10.1063/1.4964770	journal	October 2016
Accurate modeling of plasma acceleration with arbitrary order pseudo-spectral particle-in-cell methods Jalas, S.; Dornmair, I.; Lehe, R. Physics of Plasmas, Vol. 24, Issue 3 https://doi.org/10.1063/1.4978569	journal	March 2017
Toward the modeling of chains of plasma accelerator stages with WarpX Vay, J. -L.; Almgren, A.; Amorim, L. D. Journal of Physics: Conference Series, Vol. 1596 https://doi.org/10.1088/1742-6596/1596/1/012059	journal	July 2020
Elimination of numerical Cherenkov instability in flowing-plasma particle-in-cell simulations by using Galilean coordinates Lehe, Remi; Kirchen, Manuel; Godfrey, Brendan B. Physical Review E, Vol. 94, Issue 5 https://doi.org/10.1103/PhysRevE.94.053305	journal	November 2016
Pseudospectral Maxwell solvers for an accurate modeling of Doppler harmonic generation on plasma mirrors with particle-in-cell codes Blaclard, G.; Vincenti, H.; Lehe, R. Physical Review E, Vol. 96, Issue 3 https://doi.org/10.1103/PhysRevE.96.033305	journal	September 2017
Spatial Properties of High-Order Harmonic Beams from Plasma Mirrors: A Ptychographic Study Leblanc, A.; Monchocé, S.; Vincenti, H. Physical Review Letters, Vol. 119, Issue 15 https://doi.org/10.1103/PhysRevLett.119.155001	journal	October 2017
Laser Electron Accelerator Tajima, T.; Dawson, J. M. Physical Review Letters, Vol. 43, Issue 4 https://doi.org/10.1103/PhysRevLett.43.267	journal	July 1979
Noninvariance of Space- and Time-Scale Ranges under a Lorentz Transformation and the Implications for the Study of Relativistic Interactions Vay, J. -L. Physical Review Letters, Vol. 98, Issue 13 https://doi.org/10.1103/PhysRevLett.98.130405	journal	March 2007
Particle-in-cell simulation of plasma-based amplification using a moving window Yoffe, S. R.; Lehe, R.; Ersfeld, B. Physical Review Research, Vol. 2, Issue 1 https://doi.org/10.1103/PhysRevResearch.2.013227	journal	February 2020
Numerical growth of emittance in simulations of laser-wakefield acceleration Lehe, R.; Lifschitz, A.; Thaury, C. Physical Review Special Topics - Accelerators and Beams, Vol. 16, Issue 2 https://doi.org/10.1103/PhysRevSTAB.16.021301	journal	February 2013
Generalized algorithm for control of numerical dispersion in explicit time-domain electromagnetic simulations Cowan, Benjamin M.; Bruhwiler, David L.; Cary, John R. Physical Review Special Topics - Accelerators and Beams, Vol. 16, Issue 4 https://doi.org/10.1103/PhysRevSTAB.16.041303	journal	April 2013
Physics of laser-driven plasma-based electron accelerators Esarey, E.; Schroeder, C. B.; Leemans, W. P. Reviews of Modern Physics, Vol. 81, Issue 3 https://doi.org/10.1103/RevModPhys.81.1229	journal	August 2009
A high-accuracy realization of the Yee algorithm using non-standard finite differences Cole, J. B. IEEE Transactions on Microwave Theory and Techniques, Vol. 45, Issue 6 https://doi.org/10.1109/22.588615	journal	June 1997
Numerical solution of initial boundary value problems involving maxwell's equations in isotropic media No authors listed IEEE Transactions on Antennas and Propagation, Vol. 14, Issue 3 https://doi.org/10.1109/TAP.1966.1138693	journal	May 1966
High-accuracy Yee algorithm based on nonstandard finite differences: new developments and verifications Cole, J. B. IEEE Transactions on Antennas and Propagation, Vol. 50, Issue 9 https://doi.org/10.1109/TAP.2002.801268	journal	September 2002
AMReX: a framework for block-structured adaptive mesh refinement Zhang, Weiqun; Almgren, Ann; Beckner, Vince Journal of Open Source Software, Vol. 4, Issue 37 https://doi.org/10.21105/joss.01370	journal	May 2019

Similar Records

WarpX v0.1.0

Software · Tue Aug 21 20:00:00 EDT 2018 · OSTI ID:code-20020

Modeling of a chain of three plasma accelerator stages with the WarpX electromagnetic PIC code on GPUs

Citation Formats

References (29)

Similar Records

Related Subjects