skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Designing a graphics processing unit accelerated petaflop capable lattice Boltzmann solver: Read aligned data layouts and asynchronous communication

Abstract

The lattice Boltzmann method is a well-established numerical approach for complex fluid flow simulations. Recently, general-purpose graphics processing units (GPUs) have become available as high-performance computing resources at large scale. We report on designing and implementing a lattice Boltzmann solver for multi-GPU systems that achieves 1.79 PFLOPS performance on 16,384 GPUs. To achieve this performance, we introduce a GPU compatible version of the so-called bundle data layout and eliminate the halo sites in order to improve data access alignment. Furthermore, we make use of the possibility to overlap data transfer between the host central processing unit and the device GPU with computing on the GPU. As a benchmark case, we simulate flow in porous media and measure both strong and weak scaling performance with the emphasis being on large-scale simulations using realistic input data.

Authors:
 [1];  [1];  [2]
  1. Åbo Akademi University, Faculty of Science and Engineering, Åbo, Finland
  2. Department of Physics and Nanoscience Center, University of Jyväskylä, Jyväskylä, Finland; Department of Physics, Tampere University of Technology, Tampere, Finland
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF)
Sponsoring Org.:
USDOE Office of Science (SC)
OSTI Identifier:
1565624
DOE Contract Number:  
287703
Resource Type:
Journal Article
Journal Name:
International Journal of High Performance Computing Applications
Additional Journal Information:
Journal Volume: 31; Journal Issue: 3; Journal ID: ISSN 1094-3420
Publisher:
SAGE
Country of Publication:
United States
Language:
English
Subject:
Computer Science

Citation Formats

Robertsén, Fredrik, Westerholm, Jan, and Mattila, Keijo. Designing a graphics processing unit accelerated petaflop capable lattice Boltzmann solver: Read aligned data layouts and asynchronous communication. United States: N. p., 2016. Web. doi:10.1177/1094342016658109.
Robertsén, Fredrik, Westerholm, Jan, & Mattila, Keijo. Designing a graphics processing unit accelerated petaflop capable lattice Boltzmann solver: Read aligned data layouts and asynchronous communication. United States. doi:10.1177/1094342016658109.
Robertsén, Fredrik, Westerholm, Jan, and Mattila, Keijo. Wed . "Designing a graphics processing unit accelerated petaflop capable lattice Boltzmann solver: Read aligned data layouts and asynchronous communication". United States. doi:10.1177/1094342016658109.
@article{osti_1565624,
title = {Designing a graphics processing unit accelerated petaflop capable lattice Boltzmann solver: Read aligned data layouts and asynchronous communication},
author = {Robertsén, Fredrik and Westerholm, Jan and Mattila, Keijo},
abstractNote = {The lattice Boltzmann method is a well-established numerical approach for complex fluid flow simulations. Recently, general-purpose graphics processing units (GPUs) have become available as high-performance computing resources at large scale. We report on designing and implementing a lattice Boltzmann solver for multi-GPU systems that achieves 1.79 PFLOPS performance on 16,384 GPUs. To achieve this performance, we introduce a GPU compatible version of the so-called bundle data layout and eliminate the halo sites in order to improve data access alignment. Furthermore, we make use of the possibility to overlap data transfer between the host central processing unit and the device GPU with computing on the GPU. As a benchmark case, we simulate flow in porous media and measure both strong and weak scaling performance with the emphasis being on large-scale simulations using realistic input data.},
doi = {10.1177/1094342016658109},
journal = {International Journal of High Performance Computing Applications},
issn = {1094-3420},
number = 3,
volume = 31,
place = {United States},
year = {2016},
month = {8}
}

Works referenced in this record:

Lattice-Boltzmann Method for Complex Flows
journal, January 2010


The lattice Boltzmann equation: theory and applications
journal, December 1992


A Knudsen layer theory for lattice gases
journal, January 1991


Lattice BGK Models for Navier-Stokes Equation
journal, February 1992