OpenMP application experiences: Porting to accelerated nodes

Bak, Seonmyeong; Bertoni, Colleen; Boehm, Swen; Budiardja, Reuben; Chapman, Barbara M.; Doerfert, Johannes; Eisenbach, Markus; Finkel, Hal; Hernandez, Oscar; Huber, Joseph; Iwasaki, Shintaro; Kale, Vivek; Kent, Paul R.C.; Kwack, JaeHyuk; Lin, Meifeng; Luszczek, Piotr; Luo, Ye; Pham, Buu; Pophale, Swaroop; Ravikumar, Kiran; Sarkar, Vivek; Scogland, Thomas; Tian, Shilei; Yeung, P. K.

doi:10.1016/j.parco.2021.102856

OpenMP application experiences: Porting to accelerated nodes

Journal Article · Sat Oct 23 04:00:00 EDT 2021 · Parallel Computing

DOI:https://doi.org/10.1016/j.parco.2021.102856· OSTI ID:1829281

Bak, Seonmyeong ^[1]; Bertoni, Colleen ^[2]; ^[3]; ^[3]; Chapman, Barbara M. ^[4]; Doerfert, Johannes ^[2]; ^[3]; Finkel, Hal ^[2]; Hernandez, Oscar ^[3]; ^[3]; Iwasaki, Shintaro ^[2]; ^[4]; ^[3]; Kwack, JaeHyuk ^[2]; Lin, Meifeng ^[4]; Luszczek, Piotr ^[5]; Luo, Ye ^[2]; Pham, Buu ^[6]; ^[3]; Ravikumar, Kiran ^[1] more »

Georgia Inst. of Technology, Atlanta, GA (United States)
Argonne National Lab. (ANL), Argonne, IL (United States)
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Brookhaven National Lab. (BNL), Upton, NY (United States)
Univ. of Tennessee, Knoxville, TN (United States)
Iowa State Univ., Ames, IA (United States)
Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
Stony Brook Univ., NY (United States)

As recent enhancements to the OpenMP specification become available in its implementations, there is a need to share the results of experimentation in order to better understand the OpenMP implementation’s behavior in practice, to identify pitfalls, and to learn how the implementations can be effectively deployed in scientific codes. We report on experiences gained and practices adopted when using OpenMP to port a variety of ECP applications, mini-apps and libraries based on different computational motifs to accelerator-based leadership-class high-performance supercomputer systems at the United States Department of Energy. Additionally, we identify important challenges and open problems related to the deployment of OpenMP. Through our report of experiences, we find that OpenMP implementations are successful on current supercomputing platforms and that OpenMP is a promising programming model to use for applications to be run on emerging and future platforms with accelerated nodes.

View Accepted Manuscript (DOE)

Research Organization:: Brookhaven National Laboratory (BNL), Upton, NY (United States)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA); National Science Foundation (NSF)

Grant/Contract Number:: SC0012704; AC05-00OR22725; AC02-06CH11357

OSTI ID:: 1829281

Alternate ID(s):: OSTI ID: 1829922
OSTI ID: 1832702
OSTI ID: 1870066

Report Number(s):: BNL--222351-2021-JAAM

Journal Information:: Parallel Computing, Journal Name: Parallel Computing Vol. 109; ISSN 0167-8191

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

References (22)

General atomic and molecular electronic structure system Schmidt, Michael W.; Baldridge, Kim K.; Boatz, Jerry A. Journal of Computational Chemistry, Vol. 14, Issue 11, p. 1347-1363 https://doi.org/10.1002/jcc.540141112	journal	November 1993
Performance of the RI-MP2 Fortran Kernel of GAMESS on GPUs via Directive-Based Offloading with Math Libraries Kwack, JaeHyuk; Bertoni, Colleen; Pham, Buu Accelerator Programming Using Directives https://doi.org/10.1007/978-3-030-49943-3_5	book	January 2020
On the calculation of the energy of a Bloch wave in a metal Korringa, J. Physica, Vol. 13, Issue 6-7 https://doi.org/10.1016/0031-8914(47)90013-X	journal	August 1947
GenASiS Basics: Object-oriented utilitarian functionality for large-scale physics simulations Cardall, Christian Y.; Budiardja, Reuben D. Computer Physics Communications, Vol. 196 https://doi.org/10.1016/j.cpc.2015.06.001	journal	November 2015
GPU acceleration of the Locally Selfconsistent Multiple Scattering code for first principles calculation of the ground state and statistical physics of materials Eisenbach, Markus; Larkin, Jeff; Lutjens, Justin Computer Physics Communications, Vol. 211 https://doi.org/10.1016/j.cpc.2016.07.013	journal	February 2017
GenASiS Basics: Object-oriented utilitarian functionality for large-scale physics simulations (Version 2) Cardall, Christian Y.; Budiardja, Reuben D. Computer Physics Communications, Vol. 214 https://doi.org/10.1016/j.cpc.2016.12.019	journal	May 2017
GenASiS Mathematics : Object-oriented manifolds, operations, and solvers for large-scale physics simulations Cardall, Christian Y.; Budiardja, Reuben D. Computer Physics Communications, Vol. 222 https://doi.org/10.1016/j.cpc.2017.10.004	journal	January 2018
GenASiS Basics: Object-oriented utilitarian functionality for large-scale physics simulations (Version 3) Budiardja, Reuben D.; Cardall, Christian Y. Computer Physics Communications, Vol. 244 https://doi.org/10.1016/j.cpc.2019.05.014	journal	November 2019
Hybrid Distributed/Shared Memory Model for the RI-MP2 Method in the Fragment Molecular Orbital Framework Pham, Buu Q.; Gordon, Mark S. Journal of Chemical Theory and Computation, Vol. 15, Issue 10 https://doi.org/10.1021/acs.jctc.9b00409	journal	September 2019
QMCPACK: Advances in the development, efficiency, and application of auxiliary field and real-space variational and diffusion quantum Monte Carlo Kent, P. R. C.; Annaberdiyev, Abdulgani; Benali, Anouar The Journal of Chemical Physics, Vol. 152, Issue 17 https://doi.org/10.1063/5.0004860	journal	May 2020
Extreme events in computational turbulence Yeung, P. K.; Zhai, X. M.; Sreenivasan, Katepalli R. Proceedings of the National Academy of Sciences, Vol. 112, Issue 41 https://doi.org/10.1073/pnas.1517368112	journal	September 2015
Stochasticity and Efficiency in Simplified Models of Core-Collapse Supernova Explosions Cardall, Christian Y.; Budiardja, Reuben D. The Astrophysical Journal, Vol. 813, Issue 1 https://doi.org/10.1088/2041-8205/813/1/L6	journal	October 2015
Inhomogeneous Electron Gas Hohenberg, P.; Kohn, W. Physical Review, Vol. 136, Issue 3B, p. B864-B871 https://doi.org/10.1103/PhysRev.136.B864	journal	November 1964
Self-Consistent Equations Including Exchange and Correlation Effects Kohn, W.; Sham, L. J. Physical Review, Vol. 140, Issue 4A, p. A1133-A1138 https://doi.org/10.1103/PhysRev.140.A1133	journal	November 1965
Solution of the Schrödinger Equation in Periodic Lattices with an Application to Metallic Lithium Kohn, W.; Rostoker, N. Physical Review, Vol. 94, Issue 5 https://doi.org/10.1103/PhysRev.94.1111	journal	June 1954
Order- N Multiple Scattering Approach to Electronic Structure Calculations Wang, Yang; Stocks, G. M.; Shelton, W. A. Physical Review Letters, Vol. 75, Issue 15 https://doi.org/10.1103/PhysRevLett.75.2867	journal	October 1995
OpenMP: an industry standard API for shared-memory programming Dagum, L.; Menon, R. IEEE Computational Science and Engineering, Vol. 5, Issue 1 https://doi.org/10.1109/99.660313	journal	January 1998
BOLT: Optimizing OpenMP Parallel Regions with User-Level Threads Iwasaki, Shintaro; Amer, Abdelhalim; Taura, Kenjiro 2019 28th International Conference on Parallel Architectures and Compilation Techniques (PACT) https://doi.org/10.1109/PACT.2019.00011	conference	September 2019
Embracing a new era of highly efficient and productive quantum Monte Carlo simulations Mathuriya, Amrita; Luo, Ye; Clay, Raymond C. SC '17: The International Conference for High Performance Computing, Networking, Storage and Analysis, Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1145/3126908.3126952	conference	November 2017
Fibers are not (P)Threads: The Case for Loose Coupling of Asynchronous Programming Models and MPI Through Continuations Schuchart, Joseph; Niethammer, Christoph; Gracia, José EuroMPI/USA '20: 27th European MPI Users' Group Meeting https://doi.org/10.1145/3416315.3416320	conference	October 2020
Study of High–Reynolds Number Isotropic Turbulence by Direct Numerical Simulation Ishihara, Takashi; Gotoh, Toshiyuki; Kaneda, Yukio Annual Review of Fluid Mechanics, Vol. 41, Issue 1 https://doi.org/10.1146/annurev.fluid.010908.165203	journal	January 2009
An efficient MPI/OpenMP parallelization of the Hartree–Fock–Roothaan method for the first generation of Intel® Xeon Phi™ processor architecture Mironov, Vladimir; Moskovsky, Alexander; D’Mello, Michael The International Journal of High Performance Computing Applications, Vol. 33, Issue 1 https://doi.org/10.1177/1094342017732628	journal	February 2017

Similar Records

Outcomes of OpenMP Hackathon: OpenMP Application Experiences with the Offloading Model (Part II)

Conference · Wed Sep 01 00:00:00 EDT 2021 · OSTI ID:1823332

Related Subjects

97 MATHEMATICS AND COMPUTING
GAMESS
GESTS
GenASiS
GridQCD
LSMS
OpenMC implementations
QMCPACK
RAJA
SLATE
accelerators
application porting experiences
high performance computing

OpenMP application experiences: Porting to accelerated nodes

Citation Formats

References (22)

Similar Records

Related Subjects