A scalable Euler–Lagrange approach for multiphase flow simulation on spectral elements
Abstract
Multiphase flow can be difficult to simulate with high accuracy due to the wide range of scales associated with various multiphase phenomena. These scales may range from the size of individual particles to the entire domain of interest. Traditionally, large scale systems can only be simulated using averaging approaches that filter out the locations of individual particles. In this work, the Euler–Lagrange method is used to simulate large-scale dense particle systems in which each individual particle is tracked. In order to accomplish this, the highly scalable spectral element code nek5000 has been extended to handle the multiple levels of multiphase coupling in these systems. These levels include what has been called one-, two-, and four-way coupling. Here, each level has been separated to detail the computational impact of each stage. A binned ghost particle algorithm has also been developed to efficiently handle the challenges of two- and four-way coupling in a parallel processing context. The algorithms and their implementations are then shown to scale to 65,536 Message Passing Interface (MPI) ranks in both the strong and weak limits. After this, validation is performed through simulation of a small-scale fluidized bed. Lastly, a large-scale fluidized bed is simulated with 65,536 MPImore »
- Authors:
-
- Department of Mechanical and Aerospace Engineering, University of Florida, Gainesville, FL, USA
- Publication Date:
- Sponsoring Org.:
- USDOE
- OSTI Identifier:
- 1557001
- Resource Type:
- Published Article
- Journal Name:
- International Journal of High Performance Computing Applications
- Additional Journal Information:
- Journal Name: International Journal of High Performance Computing Applications Journal Volume: 34 Journal Issue: 3; Journal ID: ISSN 1094-3420
- Publisher:
- SAGE Publications
- Country of Publication:
- United States
- Language:
- English
Citation Formats
Zwick, David, and Balachandar, S. A scalable Euler–Lagrange approach for multiphase flow simulation on spectral elements. United States: N. p., 2019.
Web. doi:10.1177/1094342019867756.
Zwick, David, & Balachandar, S. A scalable Euler–Lagrange approach for multiphase flow simulation on spectral elements. United States. https://doi.org/10.1177/1094342019867756
Zwick, David, and Balachandar, S. Mon .
"A scalable Euler–Lagrange approach for multiphase flow simulation on spectral elements". United States. https://doi.org/10.1177/1094342019867756.
@article{osti_1557001,
title = {A scalable Euler–Lagrange approach for multiphase flow simulation on spectral elements},
author = {Zwick, David and Balachandar, S.},
abstractNote = {Multiphase flow can be difficult to simulate with high accuracy due to the wide range of scales associated with various multiphase phenomena. These scales may range from the size of individual particles to the entire domain of interest. Traditionally, large scale systems can only be simulated using averaging approaches that filter out the locations of individual particles. In this work, the Euler–Lagrange method is used to simulate large-scale dense particle systems in which each individual particle is tracked. In order to accomplish this, the highly scalable spectral element code nek5000 has been extended to handle the multiple levels of multiphase coupling in these systems. These levels include what has been called one-, two-, and four-way coupling. Here, each level has been separated to detail the computational impact of each stage. A binned ghost particle algorithm has also been developed to efficiently handle the challenges of two- and four-way coupling in a parallel processing context. The algorithms and their implementations are then shown to scale to 65,536 Message Passing Interface (MPI) ranks in both the strong and weak limits. After this, validation is performed through simulation of a small-scale fluidized bed. Lastly, a large-scale fluidized bed is simulated with 65,536 MPI ranks and is able to capture the unique physics of the onset of fluidization.},
doi = {10.1177/1094342019867756},
journal = {International Journal of High Performance Computing Applications},
number = 3,
volume = 34,
place = {United States},
year = {Mon Aug 12 00:00:00 EDT 2019},
month = {Mon Aug 12 00:00:00 EDT 2019}
}
https://doi.org/10.1177/1094342019867756
Web of Science
Works referenced in this record:
A Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs
journal, January 1998
- Karypis, George; Kumar, Vipin
- SIAM Journal on Scientific Computing, Vol. 20, Issue 1
The numerical stability of barycentric Lagrange interpolation
journal, October 2004
- Higham, N. J.
- IMA Journal of Numerical Analysis, Vol. 24, Issue 4
Scaling Limits for PDE-Based Simulation (Invited)
conference, June 2015
- Fischer, Paul F.
- 22nd AIAA Computational Fluid Dynamics Conference
Methods for evaluating fluid velocities in spectral simulations of turbulence
journal, July 1989
- Balachandar, S.; Maxey, M. R.
- Journal of Computational Physics, Vol. 83, Issue 1
ppiclF: A Parallel Particle-In-Cell Library in Fortran
journal, May 2019
- Zwick, David
- Journal of Open Source Software, Vol. 4, Issue 37
Modeling and numerical simulation of particulate flows by the Eulerian–Lagrangian approach
journal, October 2001
- Patankar, N. A.; Joseph, D. D.
- International Journal of Multiphase Flow, Vol. 27, Issue 10
Dynamic alignment and distribution of irregularly coupled data arrays for scalable parallelization of particle-in-cell problems
conference, January 1996
- Wei-Keng Liao, ; Ranka, S.
- Proceedings of International Conference on Parallel Processing
Partitioning of unstructured problems for parallel processing
journal, January 1991
- Simon, H. D.
- Computing Systems in Engineering, Vol. 2, Issue 2-3
Visualization of the Bulle-Effect at River Bifurcations
conference, January 2018
- Dutta, Som; Moer, Mark W. Van; Fischer, Paul
- Proceedings of the Practice and Experience on Advanced Research Computing - PEARC '18
Discrete particle simulation of two-dimensional fluidized bed
journal, October 1993
- Tsuji, Y.; Kawaguchi, T.; Tanaka, T.
- Powder Technology, Vol. 77, Issue 1
On predicting particle-laden turbulent flows
journal, June 1994
- Elghobashi, S.
- Applied Scientific Research, Vol. 52, Issue 4
Determination of the normal spring stiffness coefficient in the linear spring–dashpot contact model of discrete element method
journal, September 2013
- Navarro, Helio A.; de Souza Braun, Meire P.
- Powder Technology, Vol. 246
Particle transport in turbulent curved pipe flow
journal, March 2016
- Noorani, Azad; Sardina, Gaetano; Brandt, Luca
- Journal of Fluid Mechanics, Vol. 793
A proof of the Kepler conjecture
journal, January 2005
- Hales, Thomas
- Annals of Mathematics, Vol. 162, Issue 3
Accurate signal reconstruction for higher order Lagrangian–Eulerian back-coupling in multiphase turbulence
journal, September 2017
- Zwick, D.; Sakhaee, E.; Balachandar, S.
- Fluid Dynamics Research, Vol. 49, Issue 5
An Euler–Lagrange strategy for simulating particle-laden flows
journal, April 2013
- Capecelatro, Jesse; Desjardins, Olivier
- Journal of Computational Physics, Vol. 238
Accurate calculation of Stokes drag for point–particle tracking in two-way coupled flows
journal, August 2016
- Horwitz, J. A. K.; Mani, A.
- Journal of Computational Physics, Vol. 318
Dynamic Load Balancing for Compressible Multiphase Turbulence
conference, January 2018
- Zhai, Keke; Banerjee, Tania; Zwick, David
- Proceedings of the 2018 International Conference on Supercomputing - ICS '18
OpenACC acceleration of the Nek5000 spectral element code
journal, March 2015
- Markidis, Stefano; Gong, Jing; Schliephake, Michael
- The International Journal of High Performance Computing Applications, Vol. 29, Issue 3
Turbulent Dispersed Multiphase Flow
journal, January 2010
- Balachandar, S.; Eaton, John K.
- Annual Review of Fluid Mechanics, Vol. 42, Issue 1
Fluid Mechanical Description of Fluidized Beds. Equations of Motion
journal, November 1967
- Anderson, T. B.; Jackson, Roy
- Industrial & Engineering Chemistry Fundamentals, Vol. 6, Issue 4
Exact regularized point particle method for multiphase flows in the two-way coupling regime
journal, May 2015
- Gualtieri, P.; Picano, F.; Sardina, G.
- Journal of Fluid Mechanics, Vol. 773
Inter-phase heat transfer and energy coupling in turbulent dispersed multiphase flows
journal, March 2016
- Ling, Y.; Balachandar, S.; Parmar, M.
- Physics of Fluids, Vol. 28, Issue 3
Barycentric Lagrange Interpolation
journal, January 2004
- Berrut, Jean-Paul; Trefethen, Lloyd N.
- SIAM Review, Vol. 46, Issue 3
Numerical Simulation of Dense Gas-Solid Fluidized Beds: A Multiscale Modeling Strategy
journal, January 2008
- van der Hoef, M. A.; van Sint Annaland, M.; Deen, N. G.
- Annual Review of Fluid Mechanics, Vol. 40, Issue 1
Total variation diminishing Runge-Kutta schemes
journal, January 1998
- Gottlieb, Sigal; Shu, Chi-Wang
- Mathematics of Computation of the American Mathematical Society, Vol. 67, Issue 221
Multiphase flow above explosion sites in debris-filled volcanic vents: Insights from analogue experiments
journal, November 2008
- Ross, Pierre-Simon; White, James D. L.; Zimanowski, Bernd
- Journal of Volcanology and Geothermal Research, Vol. 178, Issue 1
Fast Parallel Direct Solvers for Coarse Grid Problems
journal, February 2001
- Tufo, H. M.; Fischer, P. F.
- Journal of Parallel and Distributed Computing, Vol. 61, Issue 2
Particle-laden turbulent flows: direct simulation and closure models
journal, October 1991
- Elghobashi, S.
- Applied Scientific Research, Vol. 48, Issue 3-4
A spectral element method for fluid dynamics: Laminar flow in a channel expansion
journal, June 1984
- Patera, Anthony T.
- Journal of Computational Physics, Vol. 54, Issue 3
Turbulence modulation and drag reduction by spherical particles
journal, August 2010
- Zhao, L. H.; Andersson, H. I.; Gillissen, J. J. J.
- Physics of Fluids, Vol. 22, Issue 8
A scaling analysis of added-mass and history forces and their coupling in dispersed multiphase flows
journal, December 2013
- Ling, Y.; Parmar, M.; Balachandar, S.
- International Journal of Multiphase Flow, Vol. 57
Revisiting the Wen and Yu Equations for Minimum Fluidization Velocity Prediction
journal, May 2004
- Delebarre, A.
- Chemical Engineering Research and Design, Vol. 82, Issue 5
Validation of a discrete element model using magnetic resonance measurements
journal, August 2009
- Müller, Christoph R.; Scott, Stuart A.; Holland, Daniel J.
- Particuology, Vol. 7, Issue 4
A generalized method for predicting the minimum fluidization velocity
journal, May 1966
- Wen, C. Y.; Yu, Y. H.
- AIChE Journal, Vol. 12, Issue 3
Improving particle drag predictions in Euler–Lagrange simulations with two-way coupling
journal, June 2017
- Ireland, Peter J.; Desjardins, Olivier
- Journal of Computational Physics, Vol. 338
Interaction of a planar shock wave with a dense particle curtain: Modeling and experiments
journal, November 2012
- Ling, Y.; Wagner, J. L.; Beresh, S. J.
- Physics of Fluids, Vol. 24, Issue 11
A scaling analysis for point–particle approaches to turbulent multiphase flows
journal, September 2009
- Balachandar, S.
- International Journal of Multiphase Flow, Vol. 35, Issue 9
A discrete numerical model for granular assemblies
journal, March 1979
- Cundall, P. A.; Strack, O. D. L.
- Géotechnique, Vol. 29, Issue 1