GPU acceleration and performance of the particle-beam-dynamics code Elegant

King, J. R.; Pogorelov, I. V.; Amyx, K. M.; Borland, M.; Soliday, R.

doi:10.1016/j.cpc.2018.09.022

Title: GPU acceleration and performance of the particle-beam-dynamics code Elegant

Abstract

Elegant is an accelerator physics and particle-beam dynamics code widely used for modeling and design of a variety of high-energy particle accelerators and accelerator-based systems. We discuss a recently developed version of the code that can take advantage of CUDA-enabled graphics processing units (GPUs) to achieve significantly improved performance for a large class of simulations that are important in practice. The GPU version is largely defined by a framework that simplifies implementations of the fundamental kernel types that are used by Elegant: particle operations, reductions, particle loss, histograms, array convolutions and random number generation. Accelerated performance on the Titan Cray XK-7 supercomputer is approximately 6–10 times better with the GPU than all the CPU cores associated with the same node count. In addition to performance, the maintainability of the GPU-accelerated version of the code was considered a key design objective. Accuracy with respect to the CPU implementation is also a core consideration. Finally, four different methods are used to ensure that the accelerated code faithfully reproduces the CPU results.

Authors:

^[1]; Pogorelov, I. V. ^[1]; Amyx, K. M. ^[1]; Borland, M. ^[2]; Soliday, R. ^[2]

Tech-X Corporation, Boulder, CO (United States)
Argonne National Lab. (ANL), Argonne, IL (United States)

Publication Date:: Tue Oct 16 00:00:00 EDT 2018

Research Org.:: Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF); Tech-X Corporation, Boulder, CO (United States)

Sponsoring Org.:: USDOE Office of Science (SC), Basic Energy Sciences (BES)

OSTI Identifier:: 1482835

Alternate Identifier(s):: OSTI ID: 1635832

Grant/Contract Number:: SC0004585; AC05-00OR22725

Resource Type:: Accepted Manuscript

Journal Name:: Computer Physics Communications

Additional Journal Information:: Journal Volume: 235; Journal ID: ISSN 0010-4655

Publisher:: Elsevier

Country of Publication:: United States

Language:: English

Subject:: 43 PARTICLE ACCELERATORS; 97 MATHEMATICS AND COMPUTING; particle-accelerator simulation; GPU acceleration

Citation Formats


                    King, J. R., Pogorelov, I. V., Amyx, K. M., Borland, M., and Soliday, R. GPU acceleration and performance of the particle-beam-dynamics code Elegant.  United States: N. p., 2018. 
Web.  doi:10.1016/j.cpc.2018.09.022.

Copy to clipboard


                    King, J. R., Pogorelov, I. V., Amyx, K. M., Borland, M., & Soliday, R. GPU acceleration and performance of the particle-beam-dynamics code Elegant.  United States.  https://doi.org/10.1016/j.cpc.2018.09.022

Copy to clipboard


                    King, J. R., Pogorelov, I. V., Amyx, K. M., Borland, M., and Soliday, R. Tue .  
"GPU acceleration and performance of the particle-beam-dynamics code Elegant".  United States.  https://doi.org/10.1016/j.cpc.2018.09.022.  https://www.osti.gov/servlets/purl/1482835.

Copy to clipboard


                    
@article{osti_1482835,

  title        = {GPU acceleration and performance of the particle-beam-dynamics code Elegant},

  author       = {King, J. R. and Pogorelov, I. V. and Amyx, K. M. and Borland, M. and Soliday, R.},

  abstractNote = {Elegant is an accelerator physics and particle-beam dynamics code widely used for modeling and design of a variety of high-energy particle accelerators and accelerator-based systems. We discuss a recently developed version of the code that can take advantage of CUDA-enabled graphics processing units (GPUs) to achieve significantly improved performance for a large class of simulations that are important in practice. The GPU version is largely defined by a framework that simplifies implementations of the fundamental kernel types that are used by Elegant: particle operations, reductions, particle loss, histograms, array convolutions and random number generation. Accelerated performance on the Titan Cray XK-7 supercomputer is approximately 6–10 times better with the GPU than all the CPU cores associated with the same node count. In addition to performance, the maintainability of the GPU-accelerated version of the code was considered a key design objective. Accuracy with respect to the CPU implementation is also a core consideration. Finally, four different methods are used to ensure that the accelerated code faithfully reproduces the CPU results.},

  doi          = {10.1016/j.cpc.2018.09.022},

  journal      = {Computer Physics Communications},

  number       = ,

  volume       = 235,

  place        = {United States},

  year         = {Tue Oct 16 00:00:00 EDT 2018},

  month        = {Tue Oct 16 00:00:00 EDT 2018}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (Publisher)

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1016/j.cpc.2018.09.022

Other availability

Search WorldCat to find libraries that may hold this journal

Citation Metrics:

Cited by: 1 work

Citation information provided by
Web of Science

Figures / Tables:

Listing 1: The Elegant CPU exactDrift function.

All figures and tables (13 total)

Save / Share:

Export Metadata

Save to My Library

Figures / Tables found in this record:

Figures/Tables have been extracted from DOE-funded journal article accepted manuscripts.

Similar Records in DOE PAGES and OSTI.GOV collections:

GPU acceleration of a petascale application for turbulent mixing at high Schmidt number using OpenMP 4.5

Journal Article Clay, M. P. ; Buaria, D. ; Yeung, P. K. ; ... - Computer Physics Communications

This paper reports on the successful implementation of a massively parallel GPU-accelerated algorithm for the direct numerical simulation of turbulent mixing at high Schmidt number. The work stems from a recent development (Comput. Phys. Commun., vol. 219, 2017, 313–328), in which a low-communication algorithm was shown to attain high degrees of scalability on the Cray XE6 architecture when overlapping communication and computation via dedicated communication threads. An even higher level of performance has now been achieved using OpenMP 4.5 on the Cray XK7 architecture, where on each node the 16 integer cores of an AMD Interlagos processor share a singlemore »« less
https://doi.org/10.1016/j.cpc.2018.02.020
Collaborating CPU and GPU for large-scale high-order CFD simulations with complex grids on the TianHe-1A supercomputer

Journal Article Xu, Chuanfu ; Deng, Xiaogang ; Zhang, Lilun ; ... - Journal of Computational Physics

Programming and optimizing complex, real-world CFD codes on current many-core accelerated HPC systems is very challenging, especially when collaborating CPUs and accelerators to fully tap the potential of heterogeneous systems. In this paper, with a tri-level hybrid and heterogeneous programming model using MPI + OpenMP + CUDA, we port and optimize our high-order multi-block structured CFD software HOSTA on the GPU-accelerated TianHe-1A supercomputer. HOSTA adopts two self-developed high-order compact definite difference schemes WCNS and HDCS that can simulate flows with complex geometries. We present a dual-level parallelization scheme for efficient multi-block computation on GPUs and perform particular kernel optimizations formore »« less
https://doi.org/10.1016/J.JCP.2014.08.024
Nek5000 with OpenACC

Journal Article

Nek5000 is a computational fluid dynamics code based on the spectral element method used for the simulation of incompressible flows. We follow up on an earlier study which ported the simplified version of Nek5000 to a GPU-accelerated system by presenting the hybrid CPU/GPU implementation of the full Nek5000 code using OpenACC. The matrix-matrix multiplication, the Nek5000 gather-scatter operator and a preconditioned Conjugate Gradient solver have implemented using OpenACC for multi-GPU systems.We report an speed-up of 1.3 on single node of a Cray XK6 when using OpenACC directives in Nek5000. On 512 nodes of the Titan supercomputer, the speed-up can bemore »« less
https://doi.org/10.1007/978-3-319-15976-8_4
miniGMG

Software Williams, Sameul

miniGMG is a compact geometric multigrid (MG benchmark designed to proxy the performance characteristics of the solves found in adaptive mesh refinement multigrid (AMR MG) applications. It solves the equation a*alpha*u - b div beta grad u = f on a large cubical domain using cell-centered values and periodic boundaries. a and b are scalar constants, alpha and beta are space varying constants, f is the right hand side, and u is the solution. The cubical domain is divided into cubical subdomains which are distributed across the supercomputer. The righthand side f is generated from a continuous function for u.more »« less
Nekbone performance on GPUs with OpenACC and CUDA Fortran implementations

Journal Article Gong, Jing ; Markidis, Stefano ; Laure, Erwin ; ... - Journal of Supercomputing

We present a hybrid GPU implementation and performance analysis of Nekbone, which represents one of the core kernels of the incompressible Navier–Stokes solver Nek5000. The implementation is based on OpenACC and CUDA Fortran for local parallelization of the compute-intensive matrix–matrix multiplication part, which significantly minimizes the modification of the existing CPU code while extending the simulation capability of the code to GPU architectures. Our discussion includes the GPU results of OpenACC interoperating with CUDA Fortran and the gather–scatter operations with GPUDirect communication. We demonstrate performance of up to 552 Tflops on 16, 384 GPUs of the OLCF Cray XK7 Titan.
https://doi.org/10.1007/s11227-016-1744-5

Similar Records