skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: An MPI/OpenACC implementation of a high-order electromagnetics solver with GPUDirect communication

Journal Article · · International Journal of High Performance Computing Applications
 [1];  [2];  [3];  [4];  [4];  [5];  [3]
  1. Department of Physics, Cornell University, Ithaca, NY, USA; Mathematics and Computer Science, Argonne National Laboratory, Lemont, IL, USA
  2. KTH Royal Institute of Technology, Stockholm, Sweden
  3. Mathematics and Computer Science, Argonne National Laboratory, Lemont, IL, USA
  4. Cray’s Suercomputing Cener of Excellence, Oak Ridge National Laboratory, Oak Ridge, TN, USA
  5. Mathematics and Computer Science, Argonne National Laboratory, Lemont, IL, USA; Department of Computer Science, Univerisity of Illinois at Urbana–Champaign, Champaign, IL, USA; Department of Mechanical Engineering, Univerisity of Illinois at Urbana–Champaign, Champaign, IL, USA
+ Show Author Affiliations

We present performance results and an analysis of a message passing interface (MPI)/OpenACC implementation of an electromagnetic solver based on a spectral-element discontinuous Galerkin discretization of the time-dependent Maxwell equations. The OpenACC implementation covers all solution routines, including a highly tuned element-by-element operator evaluation and a GPUDirect gather–scatter kernel to effect nearest neighbor flux exchanges. Modifications are designed to make effective use of vectorization, streaming, and data management. Performance results using up to 16,384 graphics processing units of the Cray XK7 supercomputer Titan show more than 2.5× speedup over central processing unit-only performance on the same number of nodes (262,144 MPI ranks) for problem sizes of up to 6.9 billion grid points. We discuss performance-enhancement strategies and the overall potential of GPU-based computing for this class of problems.

Research Organization:
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF)
Sponsoring Organization:
USDOE Office of Science (SC)
DOE Contract Number:
AC05-00OR22725; AC02-06CH11357
OSTI ID:
1565523
Journal Information:
International Journal of High Performance Computing Applications, Vol. 30, Issue 3; ISSN 1094-3420
Publisher:
SAGE
Country of Publication:
United States
Language:
English

References (4)

High-Order Methods for Incompressible Fluid Flow book January 2009
Nodal discontinuous Galerkin methods on graphics processors journal November 2009
OpenACC acceleration of the Nek5000 spectral element code journal March 2015
An Efficient High-Order Time Integration Method for Spectral-Element Discontinuous Galerkin Simulations in Electromagnetics journal June 2013

Similar Records

OpenACC acceleration of an unstructured CFD solver based on a reconstructed discontinuous Galerkin method for compressible flows
Journal Article · Mon Feb 09 00:00:00 EST 2015 · International Journal for Numerical Methods in Fluids · OSTI ID:1565523
+2 more
Nekbone performance on GPUs with OpenACC and CUDA Fortran implementations
Journal Article · Mon Jul 18 00:00:00 EDT 2016 · Journal of Supercomputing · OSTI ID:1565523
+3 more
Massively parallel and linear-scaling algorithm for second-order Moller–Plesset perturbation theory applied to the study of supramolecular wires
Journal Article · Wed Nov 16 00:00:00 EST 2016 · Computer Physics Communications · OSTI ID:1565523
+9 more

Related Subjects

Computer Science