skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Scalability of OpenFOAM Density-Based Solver with Runge–Kutta Temporal Discretization Scheme

Abstract

Compressible density-based solvers are widely used in OpenFOAM, and the parallel scalability of these solvers is crucial for large-scale simulations. In this paper, we report our experiences with the scalability of OpenFOAM’s native rhoCentralFoam solver, and by making a small number of modifications to it, we show the degree to which the scalability of the solver can be improved. The main modification made is to replace the first-order accurate Euler scheme in rhoCentralFoam with a third-order accurate, four-stage Runge-Kutta or RK4 scheme for the time integration. The scaling test we used is the transonic flow over the ONERA M6 wing. This is a common validation test for compressible flows solvers in aerospace and other engineering applications. Numerical experiments show that our modified solver, referred to as rhoCentralRK4Foam, for the same spatial discretization, achieves as much as a 123.2% improvement in scalability over the rhoCentralFoam solver. As expected, the better time resolution of the Runge–Kutta scheme makes it more suitable for unsteady problems such as the Taylor–Green vortex decay where the new solver showed a 50% decrease in the overall time-to-solution compared to rhoCentralFoam to get to the final solution with the same numerical accuracy. Finally, the improved scalability can bemore » traced to the improvement of the computation to communication ratio obtained by substituting the RK4 scheme in place of the Euler scheme. All numerical tests were conducted on a Cray XC40 parallel system, Theta, at Argonne National Laboratory.« less

Authors:
ORCiD logo [1]; ORCiD logo [2]; ORCiD logo [3]
  1. Department of Mechanical and Industrial Engineering, University of Illinois at Chicago, Chicago, IL, USA
  2. Department of Mechanical and Industrial Engineering, University of Illinois at Chicago, Chicago, IL, USA, Computational Science Division and Leadership Computing Facility, Argonne National Laboratory, Lemont, IL, USA
  3. Intel Corporation, Schaumburg, IL, USA
Publication Date:
Sponsoring Org.:
USDOE
OSTI Identifier:
1604122
Grant/Contract Number:  
[#ANL 4J-30361-0030A]
Resource Type:
Published Article
Journal Name:
Scientific Programming
Additional Journal Information:
[Journal Name: Scientific Programming Journal Volume: 2020]; Journal ID: ISSN 1058-9244
Publisher:
Hindawi Publishing Corporation
Country of Publication:
Egypt
Language:
English

Citation Formats

Li, Sibo, Paoli, Roberto, and D’Mello, Michael. Scalability of OpenFOAM Density-Based Solver with Runge–Kutta Temporal Discretization Scheme. Egypt: N. p., 2020. Web. doi:10.1155/2020/9083620.
Li, Sibo, Paoli, Roberto, & D’Mello, Michael. Scalability of OpenFOAM Density-Based Solver with Runge–Kutta Temporal Discretization Scheme. Egypt. doi:10.1155/2020/9083620.
Li, Sibo, Paoli, Roberto, and D’Mello, Michael. Wed . "Scalability of OpenFOAM Density-Based Solver with Runge–Kutta Temporal Discretization Scheme". Egypt. doi:10.1155/2020/9083620.
@article{osti_1604122,
title = {Scalability of OpenFOAM Density-Based Solver with Runge–Kutta Temporal Discretization Scheme},
author = {Li, Sibo and Paoli, Roberto and D’Mello, Michael},
abstractNote = {Compressible density-based solvers are widely used in OpenFOAM, and the parallel scalability of these solvers is crucial for large-scale simulations. In this paper, we report our experiences with the scalability of OpenFOAM’s native rhoCentralFoam solver, and by making a small number of modifications to it, we show the degree to which the scalability of the solver can be improved. The main modification made is to replace the first-order accurate Euler scheme in rhoCentralFoam with a third-order accurate, four-stage Runge-Kutta or RK4 scheme for the time integration. The scaling test we used is the transonic flow over the ONERA M6 wing. This is a common validation test for compressible flows solvers in aerospace and other engineering applications. Numerical experiments show that our modified solver, referred to as rhoCentralRK4Foam, for the same spatial discretization, achieves as much as a 123.2% improvement in scalability over the rhoCentralFoam solver. As expected, the better time resolution of the Runge–Kutta scheme makes it more suitable for unsteady problems such as the Taylor–Green vortex decay where the new solver showed a 50% decrease in the overall time-to-solution compared to rhoCentralFoam to get to the final solution with the same numerical accuracy. Finally, the improved scalability can be traced to the improvement of the computation to communication ratio obtained by substituting the RK4 scheme in place of the Euler scheme. All numerical tests were conducted on a Cray XC40 parallel system, Theta, at Argonne National Laboratory.},
doi = {10.1155/2020/9083620},
journal = {Scientific Programming},
number = ,
volume = [2020],
place = {Egypt},
year = {2020},
month = {3}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
DOI: 10.1155/2020/9083620

Save / Share:

Works referenced in this record:

Application of a fractional-step method to incompressible Navier-Stokes equations
journal, June 1985


The numerical simulation of two-dimensional fluid flow with strong shocks
journal, April 1984


Scalability of OpenFOAM for bio-medical flow simulations
journal, November 2014

  • Duran, Ahmet; Celebi, M. Serdar; Piskin, Senol
  • The Journal of Supercomputing, Vol. 71, Issue 3
  • DOI: 10.1007/s11227-014-1344-1

The Tau Parallel Performance System
journal, May 2006

  • Shende, Sameer S.; Malony, Allen D.
  • The International Journal of High Performance Computing Applications, Vol. 20, Issue 2
  • DOI: 10.1177/1094342006064482

A New Flux Splitting Scheme
journal, July 1993

  • Liou, Meng-Sing; Steffen, Christopher J.
  • Journal of Computational Physics, Vol. 107, Issue 1
  • DOI: 10.1006/jcph.1993.1122

On the implementation of low-dissipative Runge–Kutta projection methods for time dependent flows using OpenFOAM®
journal, April 2014


Numerical solution of unsteady Navier–Stokes equations on curvilinear meshes
journal, June 2012

  • Shah, Abdullah; Yuan, Li; Islam, Shamsul
  • Computers & Mathematics with Applications, Vol. 63, Issue 11
  • DOI: 10.1016/j.camwa.2012.03.047

Factorization methods for the numerical approximation of Navier–Stokes equations
journal, July 2000

  • Quarteroni, Alfio; Saleri, Fausto; Veneziani, Alessandro
  • Computer Methods in Applied Mechanics and Engineering, Vol. 188, Issue 1-3
  • DOI: 10.1016/s0045-7825(99)00192-9

Implementation of semi-discrete, non-staggered central schemes in a colocated, polyhedral, finite volume framework, for high-speed viscous flows
journal, January 2009

  • Greenshields, Christopher J.; Weller, Henry G.; Gasparini, Luca
  • International Journal for Numerical Methods in Fluids
  • DOI: 10.1002/fld.2069

Communication Optimization for Multiphase Flow Solver in the Library of OpenFOAM
journal, October 2018

  • Lin, Zhipeng; Yang, Wenjing; Zhou, Houcun
  • Water, Vol. 10, Issue 10
  • DOI: 10.3390/w10101461

Implementation of density-based solver for all speeds in the framework of OpenFOAM
journal, October 2014


New High-Resolution Central Schemes for Nonlinear Conservation Laws and Convection–Diffusion Equations
journal, May 2000

  • Kurganov, Alexander; Tadmor, Eitan
  • Journal of Computational Physics, Vol. 160, Issue 1
  • DOI: 10.1006/jcph.2000.6459

Modeling of Ice Accretion over Aircraft Wings Using a Compressible OpenFOAM Solver
journal, June 2019

  • Li, Sibo; Paoli, Roberto
  • International Journal of Aerospace Engineering, Vol. 2019
  • DOI: 10.1155/2019/4864927

Theta: Rapid installation and acceptance of an XC40 KNL system: Theta: Rapid installation and acceptance of an XC40 KNL system
journal, September 2017

  • Harms, Kevin; Leggett, Ti; Allen, Ben
  • Concurrency and Computation: Practice and Experience, Vol. 30, Issue 1
  • DOI: 10.1002/cpe.4336

A tensorial approach to computational continuum mechanics using object-oriented techniques
journal, January 1998

  • Weller, H. G.; Tabor, G.; Jasak, H.
  • Computers in Physics, Vol. 12, Issue 6
  • DOI: 10.1063/1.168744

reactingFoam-SCI: An open source CFD platform for reacting flow simulation
journal, August 2019


PT-Scotch: A tool for efficient parallel graph ordering
journal, July 2008