skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

This content will become publicly available on November 1, 2020

Title: A generalized massively parallel ultra-high order FFT-based Maxwell solver

Abstract

Dispersion-free ultra-high order FFT-based Maxwell solvers have recently proven to be paramount to a large range of applications, including the high-fidelity modeling of high-intensity laser–matter interactions with Particle-In-Cell (PIC) codes. To enable a massively parallel scaling of these solvers, a novel parallelization technique was recently proposed, which consists in splitting the simulation domain into several processor sub-domains, with guard regions appended at each sub-domain boundary. Maxwell's equations are advanced independently on each sub-domain using local shared-memory FFTs (instead of a single distributed global FFT). This implies small truncation errors at sub-domain boundaries, the amplitude of which depends on guard regions sizes and order of the Maxwell solver. For moderate guard region sizes, this ’local’ technique proved to be highly scalable on up to a million cores and notably enabled the 3D modeling of so-called plasma mirrors, for which 8 guard cells only were enough to prevent truncation error growth. Yet, for other applications, the required number of guard cells might be much higher, which would severely limit the parallel efficiency of this technique due to the large volume of guard cells to be exchanged between sub-domains. In this context, we propose a novel parallelization technique that ensures very good scalingmore » of FFT-based solvers with an arbitrarily high number of guard cells. Our ’hybrid’ technique consists in performing distributed FFTs on local groups of processors with guard regions now appended to boundaries of each group of processors. It uses a dual domain decomposition method for the Maxwell solver and other parts of the PIC cycle to keep the simulation load-balanced. This ’hybrid’ technique was implemented in the open source exascale library PICSAR. Benchmarks show that for a large number of guard cells (>16), the ’hybrid’ technique offers up to ×3 speed-up and ×8 memory savings compared to the ’local’ one.« less

Authors:
; ;
Publication Date:
Research Org.:
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Sponsoring Org.:
USDOE Office of Science (SC), High Energy Physics (HEP) (SC-25)
OSTI Identifier:
1580963
Alternate Identifier(s):
OSTI ID: 1566215
Grant/Contract Number:  
AC02-05CH11231
Resource Type:
Accepted Manuscript
Journal Name:
Computer Physics Communications
Additional Journal Information:
Journal Volume: 244; Journal Issue: C; Journal ID: ISSN 0010-4655
Publisher:
Elsevier
Country of Publication:
United States
Language:
English

Citation Formats

Kallala, Haithem, Vay, Jean-Luc, and Vincenti, Henri. A generalized massively parallel ultra-high order FFT-based Maxwell solver. United States: N. p., 2019. Web. doi:10.1016/j.cpc.2019.07.009.
Kallala, Haithem, Vay, Jean-Luc, & Vincenti, Henri. A generalized massively parallel ultra-high order FFT-based Maxwell solver. United States. doi:10.1016/j.cpc.2019.07.009.
Kallala, Haithem, Vay, Jean-Luc, and Vincenti, Henri. Fri . "A generalized massively parallel ultra-high order FFT-based Maxwell solver". United States. doi:10.1016/j.cpc.2019.07.009.
@article{osti_1580963,
title = {A generalized massively parallel ultra-high order FFT-based Maxwell solver},
author = {Kallala, Haithem and Vay, Jean-Luc and Vincenti, Henri},
abstractNote = {Dispersion-free ultra-high order FFT-based Maxwell solvers have recently proven to be paramount to a large range of applications, including the high-fidelity modeling of high-intensity laser–matter interactions with Particle-In-Cell (PIC) codes. To enable a massively parallel scaling of these solvers, a novel parallelization technique was recently proposed, which consists in splitting the simulation domain into several processor sub-domains, with guard regions appended at each sub-domain boundary. Maxwell's equations are advanced independently on each sub-domain using local shared-memory FFTs (instead of a single distributed global FFT). This implies small truncation errors at sub-domain boundaries, the amplitude of which depends on guard regions sizes and order of the Maxwell solver. For moderate guard region sizes, this ’local’ technique proved to be highly scalable on up to a million cores and notably enabled the 3D modeling of so-called plasma mirrors, for which 8 guard cells only were enough to prevent truncation error growth. Yet, for other applications, the required number of guard cells might be much higher, which would severely limit the parallel efficiency of this technique due to the large volume of guard cells to be exchanged between sub-domains. In this context, we propose a novel parallelization technique that ensures very good scaling of FFT-based solvers with an arbitrarily high number of guard cells. Our ’hybrid’ technique consists in performing distributed FFTs on local groups of processors with guard regions now appended to boundaries of each group of processors. It uses a dual domain decomposition method for the Maxwell solver and other parts of the PIC cycle to keep the simulation load-balanced. This ’hybrid’ technique was implemented in the open source exascale library PICSAR. Benchmarks show that for a large number of guard cells (>16), the ’hybrid’ technique offers up to ×3 speed-up and ×8 memory savings compared to the ’local’ one.},
doi = {10.1016/j.cpc.2019.07.009},
journal = {Computer Physics Communications},
number = C,
volume = 244,
place = {United States},
year = {2019},
month = {11}
}

Journal Article:
Free Publicly Available Full Text
This content will become publicly available on November 1, 2020
Publisher's Version of Record

Citation Metrics:
Cited by: 1 work
Citation information provided by
Web of Science

Save / Share: