Development of a CPU/GPU portable software library for Lagrangian–Eulerian simulations of liquid sprays

Ge, Wenjun; Sankaran, Ramanan; Chen, Jacqueline H.

doi:10.1016/j.ijmultiphaseflow.2020.103293

Development of a CPU/GPU portable software library for Lagrangian–Eulerian simulations of liquid sprays

Journal Article · Thu Apr 09 00:00:00 EDT 2020 · International Journal of Multiphase Flow

DOI:https://doi.org/10.1016/j.ijmultiphaseflow.2020.103293· OSTI ID:1616824

^[1]; ^[1]; Chen, Jacqueline H. ^[2]

Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Computational Sciences and Engineering Division
Sandia National Lab. (SNL-CA), Livermore, CA (United States)

The Lagrangian–Eulerian method is widely used for simulations of fuel sprays in turbulent combustion because of the advantage of treating the spray droplets as discrete points. One challenge of the Lagrangian–Eulerian method is the intense computational requirement when tracking the large number of Lagrangian particles needed for high fidelity. We have developed a performance-portable library, Grit, to track the Lagrangian particles in parallel on central processing unit (CPU) and graphics processing unit (GPU) accelerated high performance computing (HPC) architectures. Grit is a C++ library which employs Message Passing Interface (MPI) for distributed memory parallelism and Kokkos programming model for on-node shared memory parallelism with performance portability across different architectures of GPUs and multi-core/manycore CPUs. The parallel algorithms, key parallel kernels, and their performances on the pre-exascale supercomputer, Summit, are presented. Grit is coupled with a direct numerical simulation (DNS) solver, S3D, for multiphase simulations. A conservative formulation has been developed and implemented in Grit for phase coupling with thermodynamic consistency. The formulation separates the conservation of mass, momentum and energy from the physical models to prevent accidental violation of conservation laws due to inconsistent models. The formulation also enforces consistent definitions of enthalpies of the fuel for both phases and the latent heat of evaporation. Finally, simulations of turbulent particle-laden flow and the evaporation of dilute turbulent spray jet are performed to verify the software implementation, and to demonstrate the scalability of Grit for large-scale multiphase simulations.

View Accepted Manuscript (DOE)

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Organization:: USDOE Office of Science (SC)

Grant/Contract Number:: AC05-00OR22725

OSTI ID:: 1616824

Alternate ID(s):: OSTI ID: 1703557

Journal Information:: International Journal of Multiphase Flow, Journal Name: International Journal of Multiphase Flow Journal Issue: C Vol. 128; ISSN 0301-9322

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

References (32)

Fast Parallel Algorithms for Short-Range Molecular Dynamics Plimpton, Steve Journal of Computational Physics, Vol. 117, Issue 1 https://doi.org/10.1006/jcph.1995.1039	journal	March 1995
Numerical Considerations in Simulating a Turbulent Suspension of Finite-Volume Particles Sundaram, Shivshankar; Collins, Lance R. Journal of Computational Physics, Vol. 124, Issue 2 https://doi.org/10.1006/jcph.1996.0064	journal	March 1996
Recent progress and challenges in exploiting graphics processors in computational fluid dynamics Niemeyer, Kyle E.; Sung, Chih-Jen The Journal of Supercomputing, Vol. 67, Issue 2 https://doi.org/10.1007/s11227-013-1015-7	journal	September 2013
Droplet vaporization model for spray combustion calculations Abramzon, B.; Sirignano, W. A. International Journal of Heat and Mass Transfer, Vol. 32, Issue 9 https://doi.org/10.1016/0017-9310(89)90043-4	journal	September 1989
An algorithm for tracking fluid particles in numerical simulations of homogeneous turbulence Yeung, P. K.; Pope, S. B. Journal of Computational Physics, Vol. 79, Issue 2 https://doi.org/10.1016/0021-9991(88)90022-8	journal	December 1988
An equation set for non-equilibrium two phase flow, and an analysis of some aspects of choking, acoustic propagation, and losses in low pressure wet steam Jackson, R.; Davidson, B. J. International Journal of Multiphase Flow, Vol. 9, Issue 5 https://doi.org/10.1016/0301-9322(83)90014-9	journal	October 1983
The fundamental equations of gas-droplet multiphase flow Young, J. B. International Journal of Multiphase Flow, Vol. 21, Issue 2 https://doi.org/10.1016/0301-9322(94)00078-X	journal	April 1995
Parallelization of a multi-blocked CFD code via three strategies for fluid flow and heat transfer analysis Jia, Rongguang; Sundén, Bengt Computers & Fluids, Vol. 33, Issue 1 https://doi.org/10.1016/S0045-7930(03)00029-X	journal	January 2004
The combustion of liquid fuels Spalding, D. B. Symposium (International) on Combustion, Vol. 4, Issue 1 https://doi.org/10.1016/S0082-0784(53)80110-4	journal	January 1953
Evaluation of equilibrium and non-equilibrium evaporation models for many-droplet gas-liquid flow simulations Miller, R. S.; Harstad, K.; Bellan, J. International Journal of Multiphase Flow, Vol. 24, Issue 6 https://doi.org/10.1016/S0301-9322(98)00028-7	journal	September 1998
Direct numerical simulation of ignition in turbulent n-heptane liquid-fuel spray jets Wang, Y.; Rutland, C. Combustion and Flame, Vol. 149, Issue 4 https://doi.org/10.1016/j.combustflame.2007.03.005	journal	June 2007
Complex chemistry DNS of n-heptane spray autoignition at high pressure and intermediate temperature conditions Borghesi, Giulio; Mastorakos, Epaminondas; Cant, R. Stewart Combustion and Flame, Vol. 160, Issue 7 https://doi.org/10.1016/j.combustflame.2013.02.009	journal	July 2013
GPU-acceleration for Moving Particle Semi-Implicit method Hori, Chiemi; Gotoh, Hitoshi; Ikari, Hiroyuki Computers & Fluids, Vol. 51, Issue 1 https://doi.org/10.1016/j.compfluid.2011.08.004	journal	December 2011
Directive-based GPU programming for computational fluid dynamics Pickering, Brent P.; Jackson, Charles W.; Scogland, Thomas R. W. Computers & Fluids, Vol. 114 https://doi.org/10.1016/j.compfluid.2015.03.008	journal	July 2015
GPU accelerated VOF based multiphase flow solver and its application to sprays Reddy, Rajesh; Banerjee, R. Computers & Fluids, Vol. 117 https://doi.org/10.1016/j.compfluid.2015.05.013	journal	August 2015
Direct numerical simulations of reacting flows with detailed chemistry using many-core/GPU acceleration Hernández Pérez, Francisco E.; Mukhadiyev, Nurzhan; Xu, Xiao Computers & Fluids, Vol. 173 https://doi.org/10.1016/j.compfluid.2018.03.074	journal	September 2018
Accelerating finite-rate chemical kinetics with coprocessors: Comparing vectorization methods on GPUs, MICs, and CPUs Stone, Christopher P.; Alferman, Andrew T.; Niemeyer, Kyle E. Computer Physics Communications, Vol. 226 https://doi.org/10.1016/j.cpc.2018.01.015	journal	May 2018
A detailed experimental investigation of well-defined, turbulent evaporating spray jets of acetone Chen, Yung-Cheng; Stårner, Sten H.; Masri, Assaad R. International Journal of Multiphase Flow, Vol. 32, Issue 4 https://doi.org/10.1016/j.ijmultiphaseflow.2005.09.002	journal	April 2006
A numerically convergent Lagrangian–Eulerian simulation method for dispersed two-phase flows Garg, R.; Narayanan, C.; Subramaniam, S. International Journal of Multiphase Flow, Vol. 35, Issue 4 https://doi.org/10.1016/j.ijmultiphaseflow.2008.12.004	journal	April 2009
GPU acceleration of Eulerian–Lagrangian particle-laden turbulent flow simulations Sweet, James; Richter, David H.; Thain, Douglas International Journal of Multiphase Flow, Vol. 99 https://doi.org/10.1016/j.ijmultiphaseflow.2017.11.010	journal	February 2018
An analysis of the convergence of stochastic Lagrangian/Eulerian spray simulations Schmidt, David P.; Bedford, Frederick International Journal of Multiphase Flow, Vol. 102 https://doi.org/10.1016/j.ijmultiphaseflow.2018.01.024	journal	May 2018
Detailed numerical simulation of liquid jet atomization in crossflow of increasing density Li, Xiaoyi; Soteriou, Marios C. International Journal of Multiphase Flow, Vol. 104 https://doi.org/10.1016/j.ijmultiphaseflow.2018.02.016	journal	July 2018
Physical modelling and advanced simulations of gas–liquid two-phase jet flows in atomization and sprays Jiang, X.; Siamas, G. A.; Jagus, K. Progress in Energy and Combustion Science, Vol. 36, Issue 2 https://doi.org/10.1016/j.pecs.2009.09.002	journal	April 2010
Modeling of turbulent dilute spray combustion Jenny, Patrick; Roekaerts, Dirk; Beishuizen, Nijso Progress in Energy and Combustion Science, Vol. 38, Issue 6 https://doi.org/10.1016/j.pecs.2012.07.001	journal	December 2012
Direct numerical simulation of a confined three-dimensional gas mixing layer with one evaporating hydrocarbon-droplet-laden stream Miller, R. S.; Bellan, J. Journal of Fluid Mechanics, Vol. 384 https://doi.org/10.1017/S0022112098004042	journal	April 1999
Thermodynamics of the Liquid State Watson, K. M. Industrial & Engineering Chemistry, Vol. 35, Issue 4 https://doi.org/10.1021/ie50400a004	journal	April 1943
On Drag of Evaporating Liquid Droplets Yuen, M. C.; Chen, L. W. Combustion Science and Technology, Vol. 14, Issue 4-6 https://doi.org/10.1080/00102207608547524	journal	October 1976
Characteristic boundary conditions for simulations of compressible reacting flows with multi-dimensional, viscous and reaction effects Yoo, C. S.; Im, H. G. Combustion Theory and Modelling, Vol. 11, Issue 2 https://doi.org/10.1080/13647830600898995	journal	April 2007
Terascale direct numerical simulations of turbulent combustion using S3D Chen, J. H.; Choudhary, A.; de Supinski, B. Computational Science & Discovery, Vol. 2, Issue 1 https://doi.org/10.1088/1749-4699/2/1/015001	journal	January 2009
OpenCL: A Parallel Programming Standard for Heterogeneous Computing Systems Stone, John E.; Gohara, David; Shi, Guochun Computing in Science & Engineering, Vol. 12, Issue 3, p. 66-73 https://doi.org/10.1109/MCSE.2010.69	journal	May 2010
Toward performance portability of the Albany finite element analysis code using the Kokkos library Demeshko, Irina; Watkins, Jerry; Tezaur, Irina K. The International Journal of High Performance Computing Applications, Vol. 33, Issue 2 https://doi.org/10.1177/1094342017749957	journal	October 2017
Simulation of the Taylor–Green Vortex Using High-Order Flux Reconstruction Schemes Bull, J. R.; Jameson, A. AIAA Journal, Vol. 53, Issue 9 https://doi.org/10.2514/1.J053766	journal	September 2015

Similar Records

ORNL/grit

Software · Wed Jun 21 20:00:00 EDT 2023 · OSTI ID:code-108669

An Adaptive Particle Tracking Algorithm for Lagrangian-Eulerian Simulations of Dispersed Multiphase Flows

Conference · Mon Dec 31 23:00:00 EST 2018 · OSTI ID:1491318

Kokkos Version 2.0

Software · Sun Oct 18 20:00:00 EDT 2015 · OSTI ID:code-3666

Related Subjects

97 MATHEMATICS AND COMPUTING
GPU
direct numerical simulations
high performance computing
multiphase flow

Development of a CPU/GPU portable software library for Lagrangian–Eulerian simulations of liquid sprays

Citation Formats

References (32)

Similar Records

Related Subjects