skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Porting a 3D seismic modeling code (SW4) to CORAL machines

Abstract

Seismic waves fourth order (SW4) solves the seismic wave equations on Cartesian and curvilinear grids using large compute clusters with O (100,000) cores. In this article we discuss the porting of SW4 to run on the CORAL architecture using the RAJA performance portability abstraction layer. The performances of key kernels using RAJA and CUDA are compared to estimate the performance penalty of using the portability abstraction layer. Code changes required for efficiency on GPUs and minimizing time spent in Message Passing Interface (MPI) are discussed. This article describes a path for efficiently porting large code bases to GPU-based machines while avoiding the pitfalls of a new architecture in the early stages of its deployment. Current bottlenecks in the code are discussed along with possible architectural or software mitigations. SW4 runs 28× faster on one 4-GPU CORAL node than on a CTS-1 node (Dual Intel Xeon E5-2695 v4). SW4 is now in routine use on problems of unprecedented resolution (203 billion grid points) and scale on 1,200 nodes of Summit.

Authors:
 [1];  [1];  [1]
  1. Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
Publication Date:
Research Org.:
Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
Sponsoring Org.:
USDOE National Nuclear Security Administration (NNSA)
OSTI Identifier:
1762887
Report Number(s):
LLNL-JRNL-787242
Journal ID: ISSN 0018-8646; 984005
Grant/Contract Number:  
AC52-07NA27344
Resource Type:
Journal Article: Accepted Manuscript
Journal Name:
IBM Journal of Research and Development
Additional Journal Information:
Journal Volume: 64; Journal Issue: 3/4; Journal ID: ISSN 0018-8646
Publisher:
IEEE
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; graphics processing units; earthquakes; computational modeling; kernel; mathematical model; propagation; surface waves

Citation Formats

Pankajakshan, R., Lin, P. -H., and Sjogreen, B. Porting a 3D seismic modeling code (SW4) to CORAL machines. United States: N. p., 2019. Web. doi:10.1147/jrd.2019.2960218.
Pankajakshan, R., Lin, P. -H., & Sjogreen, B. Porting a 3D seismic modeling code (SW4) to CORAL machines. United States. https://doi.org/10.1147/jrd.2019.2960218
Pankajakshan, R., Lin, P. -H., and Sjogreen, B. Tue . "Porting a 3D seismic modeling code (SW4) to CORAL machines". United States. https://doi.org/10.1147/jrd.2019.2960218. https://www.osti.gov/servlets/purl/1762887.
@article{osti_1762887,
title = {Porting a 3D seismic modeling code (SW4) to CORAL machines},
author = {Pankajakshan, R. and Lin, P. -H. and Sjogreen, B.},
abstractNote = {Seismic waves fourth order (SW4) solves the seismic wave equations on Cartesian and curvilinear grids using large compute clusters with O (100,000) cores. In this article we discuss the porting of SW4 to run on the CORAL architecture using the RAJA performance portability abstraction layer. The performances of key kernels using RAJA and CUDA are compared to estimate the performance penalty of using the portability abstraction layer. Code changes required for efficiency on GPUs and minimizing time spent in Message Passing Interface (MPI) are discussed. This article describes a path for efficiently porting large code bases to GPU-based machines while avoiding the pitfalls of a new architecture in the early stages of its deployment. Current bottlenecks in the code are discussed along with possible architectural or software mitigations. SW4 runs 28× faster on one 4-GPU CORAL node than on a CTS-1 node (Dual Intel Xeon E5-2695 v4). SW4 is now in routine use on problems of unprecedented resolution (203 billion grid points) and scale on 1,200 nodes of Summit.},
doi = {10.1147/jrd.2019.2960218},
url = {https://www.osti.gov/biblio/1762887}, journal = {IBM Journal of Research and Development},
issn = {0018-8646},
number = 3/4,
volume = 64,
place = {United States},
year = {2019},
month = {12}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Save / Share: