skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Experiences in porting mini-applications to OpenACC and OpenMP on heterogeneous systems

Abstract

This article studies mini-applications—Minisweep, GenASiS, GPP, and FF—that use computational methods commonly encountered in HPC. We have ported these applications to develop OpenACC and OpenMP versions, and evaluated their performance on Titan (Cray XK7 with K20x GPUs), Cori (Cray XC40 with Intel KNL), Summit (IBM AC922 with Volta GPUs), and Cori-GPU (Cray CS-Storm 500NX with Intel Skylake and Volta GPUs). Our goals are for these new ports to be useful to both application and compiler developers, to document and describe the lessons learned and the methodology to create optimized OpenMP and OpenACC versions, and to provide a description of possible migration paths between the two specifications. Cases where specific directives or code patterns result in improved performance for a given architecture are highlighted. Here, we also include discussions of the functionality and maturity of the latest compilers available on the above platforms with respect to OpenACC or OpenMP implementations.

Authors:
ORCiD logo [1]; ORCiD logo [1];  [2];  [2]; ORCiD logo [1]; ORCiD logo [1]
  1. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). National Center for Computational Sciences (NCCS)
  2. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). National Energy Research Scientific Computing Center (NERSC)
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States); Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). National Energy Research Scientific Computing Center (NERSC)
Sponsoring Org.:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
OSTI Identifier:
1649533
Alternate Identifier(s):
OSTI ID: 1615974
Grant/Contract Number:  
AC05-00OR22725; AC02-05CH11231; AC02‐05CH11231; AC05‐00OR22725
Resource Type:
Accepted Manuscript
Journal Name:
Concurrency and Computation. Practice and Experience
Additional Journal Information:
Journal Volume: 32; Journal Issue: 20; Journal ID: ISSN 1532-0626
Publisher:
Wiley
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; OpenMP; OpenACC; performance evaluation; mini-applications

Citation Formats

Vergara Larrea, Verónica G., Budiardja, Reuben D., Gayatri, Rahulkumar, Daley, Christopher, Hernandez, Oscar, and Joubert, Wayne. Experiences in porting mini-applications to OpenACC and OpenMP on heterogeneous systems. United States: N. p., 2020. Web. https://doi.org/10.1002/cpe.5780.
Vergara Larrea, Verónica G., Budiardja, Reuben D., Gayatri, Rahulkumar, Daley, Christopher, Hernandez, Oscar, & Joubert, Wayne. Experiences in porting mini-applications to OpenACC and OpenMP on heterogeneous systems. United States. https://doi.org/10.1002/cpe.5780
Vergara Larrea, Verónica G., Budiardja, Reuben D., Gayatri, Rahulkumar, Daley, Christopher, Hernandez, Oscar, and Joubert, Wayne. Fri . "Experiences in porting mini-applications to OpenACC and OpenMP on heterogeneous systems". United States. https://doi.org/10.1002/cpe.5780. https://www.osti.gov/servlets/purl/1649533.
@article{osti_1649533,
title = {Experiences in porting mini-applications to OpenACC and OpenMP on heterogeneous systems},
author = {Vergara Larrea, Verónica G. and Budiardja, Reuben D. and Gayatri, Rahulkumar and Daley, Christopher and Hernandez, Oscar and Joubert, Wayne},
abstractNote = {This article studies mini-applications—Minisweep, GenASiS, GPP, and FF—that use computational methods commonly encountered in HPC. We have ported these applications to develop OpenACC and OpenMP versions, and evaluated their performance on Titan (Cray XK7 with K20x GPUs), Cori (Cray XC40 with Intel KNL), Summit (IBM AC922 with Volta GPUs), and Cori-GPU (Cray CS-Storm 500NX with Intel Skylake and Volta GPUs). Our goals are for these new ports to be useful to both application and compiler developers, to document and describe the lessons learned and the methodology to create optimized OpenMP and OpenACC versions, and to provide a description of possible migration paths between the two specifications. Cases where specific directives or code patterns result in improved performance for a given architecture are highlighted. Here, we also include discussions of the functionality and maturity of the latest compilers available on the above platforms with respect to OpenACC or OpenMP implementations.},
doi = {10.1002/cpe.5780},
journal = {Concurrency and Computation. Practice and Experience},
number = 20,
volume = 32,
place = {United States},
year = {2020},
month = {4}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Save / Share:

Works referenced in this record:

GenASiS Basics: Object-oriented utilitarian functionality for large-scale physics simulations (Version 3)
journal, November 2019


Genasis: General Astrophysical Simulation System. i. Refinable mesh and Nonrelativistic Hydrodynamics
journal, January 2014

  • Cardall, Christian Y.; Budiardja, Reuben D.; Endeve, Eirik
  • The Astrophysical Journal Supplement Series, Vol. 210, Issue 2
  • DOI: 10.1088/0067-0049/210/2/17

MiniApps derived from production HPC applications using multiple programing models
journal, September 2016

  • Messer, OE Bronson; D’Azevedo, Ed; Hill, Judy
  • The International Journal of High Performance Computing Applications, Vol. 32, Issue 4
  • DOI: 10.1177/1094342016668241

BerkeleyGW: A massively parallel computer package for the calculation of the quasiparticle and optical properties of materials and nanostructures
journal, June 2012

  • Deslippe, Jack; Samsonidze, Georgy; Strubbe, David A.
  • Computer Physics Communications, Vol. 183, Issue 6
  • DOI: 10.1016/j.cpc.2011.12.006

Targeting GPUs with OpenMP directives on Summit: A simple and effective Fortran experience
journal, October 2019


High performance radiation transport simulations: Preparing for TITAN
conference, November 2012

  • Baker, C.; Davidson, G.; Evans, T. M.
  • 2012 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis
  • DOI: 10.1109/SC.2012.64

MPI + OpenACC: Accelerating radiation transport mini-application, minisweep, on heterogeneous systems
journal, March 2019

  • Searles, Robert; Chandrasekaran, Sunita; Joubert, Wayne
  • Computer Physics Communications, Vol. 236
  • DOI: 10.1016/j.cpc.2018.10.007

GenASiS   Basics: Object-oriented utilitarian functionality for large-scale physics simulations
journal, November 2015


Electron self-energy calculation using a general multi-pole approximation
journal, April 2003

  • Soininen, J. A.; Rehr, J. J.; Shirley, Eric L.
  • Journal of Physics: Condensed Matter, Vol. 15, Issue 17
  • DOI: 10.1088/0953-8984/15/17/312

Hybridizing S3D into an Exascale application using OpenACC: An approach for moving to multi-petaflops and beyond
conference, November 2012

  • Levesque, John M.; Sankaran, Ramanan; Grout, Ray
  • 2012 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis
  • DOI: 10.1109/SC.2012.69