Experiences in porting mini-applications to OpenACC and OpenMP on heterogeneous systems

Vergara Larrea, Verónica G.; Budiardja, Reuben D.; Gayatri, Rahulkumar; Daley, Christopher; Hernandez, Oscar; Joubert, Wayne

doi:10.1002/cpe.5780

Title: Experiences in porting mini-applications to OpenACC and OpenMP on heterogeneous systems

Abstract

This article studies mini-applications—Minisweep, GenASiS, GPP, and FF—that use computational methods commonly encountered in HPC. We have ported these applications to develop OpenACC and OpenMP versions, and evaluated their performance on Titan (Cray XK7 with K20x GPUs), Cori (Cray XC40 with Intel KNL), Summit (IBM AC922 with Volta GPUs), and Cori-GPU (Cray CS-Storm 500NX with Intel Skylake and Volta GPUs). Our goals are for these new ports to be useful to both application and compiler developers, to document and describe the lessons learned and the methodology to create optimized OpenMP and OpenACC versions, and to provide a description of possible migration paths between the two specifications. Cases where specific directives or code patterns result in improved performance for a given architecture are highlighted. Here, we also include discussions of the functionality and maturity of the latest compilers available on the above platforms with respect to OpenACC or OpenMP implementations.

Authors:

^[1];

^[1]; Gayatri, Rahulkumar ^[2]; Daley, Christopher ^[2];

^[1];

^[1]

Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). National Center for Computational Sciences (NCCS)
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). National Energy Research Scientific Computing Center (NERSC)

Publication Date:: 2020-04-24

Research Org.:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States); Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). National Energy Research Scientific Computing Center (NERSC)

Sponsoring Org.:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)

OSTI Identifier:: 1649533

Alternate Identifier(s):: OSTI ID: 1615974

Grant/Contract Number:: AC05-00OR22725; AC02-05CH11231; AC02‐05CH11231; AC05‐00OR22725

Resource Type:: Accepted Manuscript

Journal Name:: Concurrency and Computation. Practice and Experience

Additional Journal Information:: Journal Volume: 32; Journal Issue: 20; Journal ID: ISSN 1532-0626

Publisher:: Wiley

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING; OpenMP; OpenACC; performance evaluation; mini-applications

Citation Formats


                    Vergara Larrea, Verónica G., Budiardja, Reuben D., Gayatri, Rahulkumar, Daley, Christopher, Hernandez, Oscar, and Joubert, Wayne. Experiences in porting mini-applications to OpenACC and OpenMP on heterogeneous systems.  United States: N. p., 2020. 
Web.  doi:10.1002/cpe.5780.

Copy to clipboard


                    Vergara Larrea, Verónica G., Budiardja, Reuben D., Gayatri, Rahulkumar, Daley, Christopher, Hernandez, Oscar, & Joubert, Wayne. Experiences in porting mini-applications to OpenACC and OpenMP on heterogeneous systems.  United States.  https://doi.org/10.1002/cpe.5780

Copy to clipboard


                    Vergara Larrea, Verónica G., Budiardja, Reuben D., Gayatri, Rahulkumar, Daley, Christopher, Hernandez, Oscar, and Joubert, Wayne. Fri .  
"Experiences in porting mini-applications to OpenACC and OpenMP on heterogeneous systems".  United States.  https://doi.org/10.1002/cpe.5780.  https://www.osti.gov/servlets/purl/1649533.

Copy to clipboard


                    
@article{osti_1649533,

  title        = {Experiences in porting mini-applications to OpenACC and OpenMP on heterogeneous systems},

  author       = {Vergara Larrea, Verónica G. and Budiardja, Reuben D. and Gayatri, Rahulkumar and Daley, Christopher and Hernandez, Oscar and Joubert, Wayne},

  abstractNote = {This article studies mini-applications—Minisweep, GenASiS, GPP, and FF—that use computational methods commonly encountered in HPC. We have ported these applications to develop OpenACC and OpenMP versions, and evaluated their performance on Titan (Cray XK7 with K20x GPUs), Cori (Cray XC40 with Intel KNL), Summit (IBM AC922 with Volta GPUs), and Cori-GPU (Cray CS-Storm 500NX with Intel Skylake and Volta GPUs). Our goals are for these new ports to be useful to both application and compiler developers, to document and describe the lessons learned and the methodology to create optimized OpenMP and OpenACC versions, and to provide a description of possible migration paths between the two specifications. Cases where specific directives or code patterns result in improved performance for a given architecture are highlighted. Here, we also include discussions of the functionality and maturity of the latest compilers available on the above platforms with respect to OpenACC or OpenMP implementations.},

  doi          = {10.1002/cpe.5780},

  journal      = {Concurrency and Computation. Practice and Experience},

  number       = 20,

  volume       = 32,

  place        = {United States},

  year         = {2020},

  month        = {4}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (Publisher)

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1002/cpe.5780

Other availability

Search WorldCat to find libraries that may hold this journal

Citation Metrics:

Cited by: 7 works

Citation information provided by
Web of Science

Save / Share:

Export Metadata

Save to My Library

Works referenced in this record:

GenASiS Basics: Object-oriented utilitarian functionality for large-scale physics simulations (Version 3)
journal, November 2019

Budiardja, Reuben D.; Cardall, Christian Y.
Computer Physics Communications, Vol. 244
DOI: 10.1016/j.cpc.2019.05.014

Genasis: General Astrophysical Simulation System. i. Refinable mesh and Nonrelativistic Hydrodynamics
journal, January 2014

Cardall, Christian Y.; Budiardja, Reuben D.; Endeve, Eirik
The Astrophysical Journal Supplement Series, Vol. 210, Issue 2
DOI: 10.1088/0067-0049/210/2/17

MiniApps derived from production HPC applications using multiple programing models
journal, September 2016

Messer, OE Bronson; D’Azevedo, Ed; Hill, Judy
The International Journal of High Performance Computing Applications, Vol. 32, Issue 4
DOI: 10.1177/1094342016668241

BerkeleyGW: A massively parallel computer package for the calculation of the quasiparticle and optical properties of materials and nanostructures
journal, June 2012

Deslippe, Jack; Samsonidze, Georgy; Strubbe, David A.
Computer Physics Communications, Vol. 183, Issue 6
DOI: 10.1016/j.cpc.2011.12.006

Targeting GPUs with OpenMP directives on Summit: A simple and effective Fortran experience
journal, October 2019

Budiardja, Reuben D.; Cardall, Christian Y.
Parallel Computing, Vol. 88
DOI: 10.1016/j.parco.2019.102544

High performance radiation transport simulations: Preparing for TITAN
conference, November 2012

Baker, C.; Davidson, G.; Evans, T. M.
2012 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis
DOI: 10.1109/SC.2012.64

MPI + OpenACC: Accelerating radiation transport mini-application, minisweep, on heterogeneous systems
journal, March 2019

Searles, Robert; Chandrasekaran, Sunita; Joubert, Wayne
Computer Physics Communications, Vol. 236
DOI: 10.1016/j.cpc.2018.10.007

GenASiS Basics: Object-oriented utilitarian functionality for large-scale physics simulations
journal, November 2015

Cardall, Christian Y.; Budiardja, Reuben D.
Computer Physics Communications, Vol. 196
DOI: 10.1016/j.cpc.2015.06.001

Electron self-energy calculation using a general multi-pole approximation
journal, April 2003

Soininen, J. A.; Rehr, J. J.; Shirley, Eric L.
Journal of Physics: Condensed Matter, Vol. 15, Issue 17
DOI: 10.1088/0953-8984/15/17/312

Hybridizing S3D into an Exascale application using OpenACC: An approach for moving to multi-petaflops and beyond
conference, November 2012

Levesque, John M.; Sankaran, Ramanan; Grout, Ray
2012 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis
DOI: 10.1109/SC.2012.69

Similar Records in DOE PAGES and OSTI.GOV collections:

MPI + OpenACC: Accelerating radiation transport mini-application, minisweep, on heterogeneous systems

Journal Article Searles, Robert ; Chandrasekaran, Sunita ; Joubert, Wayne ; ... - Computer Physics Communications

According to the article, architectures are rapidly evolving, and exascale machines are expected to offer billion-way concurrency. We need to rethink algorithms, languages and programming models among other components in order to migrate large scale applications and explore parallelism on these machines. Although directive-based programming models allow programmers to worry less about programming and more about science, expressing complex parallel patterns in these models can be a daunting task especially when the goal is to match the performance that the hardware platforms can offer. One such pattern is wavefront. This paper extensively studies a wavefront-based miniapplication for Denovo, a productionmore »« less
https://doi.org/10.1016/j.cpc.2018.10.007
Performance Analysis of a High-Level Abstractions-Based Hydrocode on Future Computing Systems

Journal Article

In this paper we present research on applying a domain specific high-level abstractions (HLA) development strategy with the aim to “future-proof” a key class of high performance computing (HPC) applications that simulate hydrodynamics computations at AWE plc. We build on an existing high-level abstraction framework, OPS, that is being developed for the solution of multi-block structured mesh-based applications at the University of Oxford. OPS uses an “active library” approach where a single application code written using the OPS API can be transformed into different highly optimized parallel implementations which can then be linked against the appropriate parallel library enabling executionmore »« less
https://doi.org/10.1007/978-3-319-17248-4_5
Experiences in extending parallware to support OpenACC. In: WACCPD '15 Proceedings of the Second Workshop on Accelerator Programming using Directives, Article No. 4

Conference Lobeiras, Jacobo ; Arenaz, Manuel ; Hernández, Oscar

Porting scientific codes to accelerator-based computers using OpenACC and OpenMP is an important topic for the HPC community. Programmability, performance portability and developer productivity are key issues for the widespread use of these systems. In the scope of general-purpose parallel computing, Parallware is a new commercial OpenMP-enabling source-to-source compiler that automatically adds OpenMP capabilities in scientific programs. Thus, extending Parallware with OpenACC or OpenMP 4.x support would contribute to improve programmability and developer productivity. In contrast, the performance portability of such approach needs to be demonstrated in practice. This paper presents a preliminary study to extend Parallware with OpenACC supportmore »« less
https://doi.org/10.1145/2832105.2832112
GPU acceleration of a petascale application for turbulent mixing at high Schmidt number using OpenMP 4.5

Journal Article Clay, M. P. ; Buaria, D. ; Yeung, P. K. ; ... - Computer Physics Communications

This paper reports on the successful implementation of a massively parallel GPU-accelerated algorithm for the direct numerical simulation of turbulent mixing at high Schmidt number. The work stems from a recent development (Comput. Phys. Commun., vol. 219, 2017, 313–328), in which a low-communication algorithm was shown to attain high degrees of scalability on the Cray XE6 architecture when overlapping communication and computation via dedicated communication threads. An even higher level of performance has now been achieved using OpenMP 4.5 on the Cray XK7 architecture, where on each node the 16 integer cores of an AMD Interlagos processor share a singlemore »« less
https://doi.org/10.1016/j.cpc.2018.02.020
Clacc: Translating OpenACC to OpenMP in Clang

Conference Denny, Joel ; Lee, Seyong ; Vetter, Jeffrey S.

OpenACC was launched in 2010 as a portable programming model for heterogeneous accelerators. Although various implementations already exist, no extensible, open-source, production-quality compiler support is available to the community. This deficiency poses a serious risk for HPC application developers targeting GPUs and other accelerators, and it limits experimentation and progress for the OpenACC specification. To address this deficiency, Clacc is a recent effort funded by the US Exascale Computing Project to develop production OpenACC compiler support for Clang and LLVM. A key feature of the Clacc design is to translate OpenACC to OpenMP to build on Clang's existing OpenMP compilermore »« less
https://doi.org/10.1109/LLVM-HPC.2018.8639349

Full Text Available

Similar Records

Title: Experiences in porting mini-applications to OpenACC and OpenMP on heterogeneous systems

Abstract

Citation Formats

GenASiS Basics: Object-oriented utilitarian functionality for large-scale physics simulations (Version 3) journal, November 2019

Genasis: General Astrophysical Simulation System. i. Refinable mesh and Nonrelativistic Hydrodynamics journal, January 2014

MiniApps derived from production HPC applications using multiple programing models journal, September 2016

BerkeleyGW: A massively parallel computer package for the calculation of the quasiparticle and optical properties of materials and nanostructures journal, June 2012

Targeting GPUs with OpenMP directives on Summit: A simple and effective Fortran experience journal, October 2019

High performance radiation transport simulations: Preparing for TITAN conference, November 2012

MPI + OpenACC: Accelerating radiation transport mini-application, minisweep, on heterogeneous systems journal, March 2019

GenASiS Basics: Object-oriented utilitarian functionality for large-scale physics simulations journal, November 2015

Electron self-energy calculation using a general multi-pole approximation journal, April 2003

Hybridizing S3D into an Exascale application using OpenACC: An approach for moving to multi-petaflops and beyond conference, November 2012

GenASiS Basics: Object-oriented utilitarian functionality for large-scale physics simulations (Version 3)
journal, November 2019

Genasis: General Astrophysical Simulation System. i. Refinable mesh and Nonrelativistic Hydrodynamics
journal, January 2014

MiniApps derived from production HPC applications using multiple programing models
journal, September 2016

BerkeleyGW: A massively parallel computer package for the calculation of the quasiparticle and optical properties of materials and nanostructures
journal, June 2012

Targeting GPUs with OpenMP directives on Summit: A simple and effective Fortran experience
journal, October 2019

High performance radiation transport simulations: Preparing for TITAN
conference, November 2012

MPI + OpenACC: Accelerating radiation transport mini-application, minisweep, on heterogeneous systems
journal, March 2019

GenASiS Basics: Object-oriented utilitarian functionality for large-scale physics simulations
journal, November 2015

Electron self-energy calculation using a general multi-pole approximation
journal, April 2003

Hybridizing S3D into an Exascale application using OpenACC: An approach for moving to multi-petaflops and beyond
conference, November 2012