A scalable Euler–Lagrange approach for multiphase flow simulation on spectral elements

Zwick, David; Balachandar, S.

doi:10.1177/1094342019867756

Title: A scalable Euler–Lagrange approach for multiphase flow simulation on spectral elements

Journal Article · Mon Aug 12 00:00:00 EDT 2019 · International Journal of High Performance Computing Applications

DOI:https://doi.org/10.1177/1094342019867756· OSTI ID:1557001

;

^[1]

Department of Mechanical and Aerospace Engineering, University of Florida, Gainesville, FL, USA

Multiphase flow can be difficult to simulate with high accuracy due to the wide range of scales associated with various multiphase phenomena. These scales may range from the size of individual particles to the entire domain of interest. Traditionally, large scale systems can only be simulated using averaging approaches that filter out the locations of individual particles. In this work, the Euler–Lagrange method is used to simulate large-scale dense particle systems in which each individual particle is tracked. In order to accomplish this, the highly scalable spectral element code nek5000 has been extended to handle the multiple levels of multiphase coupling in these systems. These levels include what has been called one-, two-, and four-way coupling. Here, each level has been separated to detail the computational impact of each stage. A binned ghost particle algorithm has also been developed to efficiently handle the challenges of two- and four-way coupling in a parallel processing context. The algorithms and their implementations are then shown to scale to 65,536 Message Passing Interface (MPI) ranks in both the strong and weak limits. After this, validation is performed through simulation of a small-scale fluidized bed. Lastly, a large-scale fluidized bed is simulated with 65,536 MPI ranks and is able to capture the unique physics of the onset of fluidization.

View Journal Article

Cite

Export

Save

Sponsoring Organization:: USDOE

OSTI ID:: 1557001

Journal Information:: International Journal of High Performance Computing Applications, Journal Name: International Journal of High Performance Computing Applications Vol. 34 Journal Issue: 3; ISSN 1094-3420

Publisher:: SAGE PublicationsCopyright Statement

Country of Publication:: United States

Language:: English

Citation Metrics:

Cited by: 37 works

Citation information provided by
Web of Science

References (39)

A Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs Karypis, George; Kumar, Vipin SIAM Journal on Scientific Computing, Vol. 20, Issue 1 https://doi.org/10.1137/S1064827595287997	journal	January 1998
The numerical stability of barycentric Lagrange interpolation Higham, N. J. IMA Journal of Numerical Analysis, Vol. 24, Issue 4 https://doi.org/10.1093/imanum/24.4.547	journal	October 2004
Scaling Limits for PDE-Based Simulation (Invited) Fischer, Paul F. 22nd AIAA Computational Fluid Dynamics Conference https://doi.org/10.2514/6.2015-3049	conference	June 2015
Methods for evaluating fluid velocities in spectral simulations of turbulence Balachandar, S.; Maxey, M. R. Journal of Computational Physics, Vol. 83, Issue 1 https://doi.org/10.1016/0021-9991(89)90224-6	journal	July 1989
ppiclF: A Parallel Particle-In-Cell Library in Fortran Zwick, David Journal of Open Source Software, Vol. 4, Issue 37 https://doi.org/10.21105/joss.01400	journal	May 2019
Modeling and numerical simulation of particulate flows by the Eulerian–Lagrangian approach Patankar, N. A.; Joseph, D. D. International Journal of Multiphase Flow, Vol. 27, Issue 10 https://doi.org/10.1016/S0301-9322(01)00021-0	journal	October 2001
Dynamic alignment and distribution of irregularly coupled data arrays for scalable parallelization of particle-in-cell problems Proceedings of International Conference on Parallel Processing https://doi.org/10.1109/IPPS.1996.508039	conference	January 1996
Partitioning of unstructured problems for parallel processing Simon, H. D. Computing Systems in Engineering, Vol. 2, Issue 2-3 https://doi.org/10.1016/0956-0521(91)90014-V	journal	January 1991
Visualization of the Bulle-Effect at River Bifurcations Dutta, Som; Moer, Mark W. Van; Fischer, Paul Proceedings of the Practice and Experience on Advanced Research Computing - PEARC '18 https://doi.org/10.1145/3219104.3229433	conference	January 2018
Discrete particle simulation of two-dimensional fluidized bed Tsuji, Y.; Kawaguchi, T.; Tanaka, T. Powder Technology, Vol. 77, Issue 1 https://doi.org/10.1016/0032-5910(93)85010-7	journal	October 1993
On predicting particle-laden turbulent flows Elghobashi, S. Applied Scientific Research, Vol. 52, Issue 4 https://doi.org/10.1007/BF00936835	journal	June 1994
Determination of the normal spring stiffness coefficient in the linear spring–dashpot contact model of discrete element method Navarro, Helio A.; de Souza Braun, Meire P. Powder Technology, Vol. 246 https://doi.org/10.1016/j.powtec.2013.05.049	journal	September 2013
Particle transport in turbulent curved pipe flow Noorani, Azad; Sardina, Gaetano; Brandt, Luca Journal of Fluid Mechanics, Vol. 793 https://doi.org/10.1017/jfm.2016.136	journal	March 2016
A proof of the Kepler conjecture Hales, Thomas Annals of Mathematics, Vol. 162, Issue 3 https://doi.org/10.4007/annals.2005.162.1065	journal	January 2005
Accurate signal reconstruction for higher order Lagrangian–Eulerian back-coupling in multiphase turbulence Zwick, D.; Sakhaee, E.; Balachandar, S. Fluid Dynamics Research, Vol. 49, Issue 5 https://doi.org/10.1088/1873-7005/aa871f	journal	September 2017
An Euler–Lagrange strategy for simulating particle-laden flows Capecelatro, Jesse; Desjardins, Olivier Journal of Computational Physics, Vol. 238 https://doi.org/10.1016/j.jcp.2012.12.015	journal	April 2013
Accurate calculation of Stokes drag for point–particle tracking in two-way coupled flows Horwitz, J. A. K.; Mani, A. Journal of Computational Physics, Vol. 318 https://doi.org/10.1016/j.jcp.2016.04.034	journal	August 2016
Dynamic Load Balancing for Compressible Multiphase Turbulence Zhai, Keke; Banerjee, Tania; Zwick, David Proceedings of the 2018 International Conference on Supercomputing - ICS '18 https://doi.org/10.1145/3205289.3205304	conference	January 2018
OpenACC acceleration of the Nek5000 spectral element code Markidis, Stefano; Gong, Jing; Schliephake, Michael The International Journal of High Performance Computing Applications, Vol. 29, Issue 3 https://doi.org/10.1177/1094342015576846	journal	March 2015
Turbulent Dispersed Multiphase Flow Balachandar, S.; Eaton, John K. Annual Review of Fluid Mechanics, Vol. 42, Issue 1 https://doi.org/10.1146/annurev.fluid.010908.165243	journal	January 2010
Fluid Mechanical Description of Fluidized Beds. Equations of Motion Anderson, T. B.; Jackson, Roy Industrial & Engineering Chemistry Fundamentals, Vol. 6, Issue 4 https://doi.org/10.1021/i160024a007	journal	November 1967
Exact regularized point particle method for multiphase flows in the two-way coupling regime Gualtieri, P.; Picano, F.; Sardina, G. Journal of Fluid Mechanics, Vol. 773 https://doi.org/10.1017/jfm.2015.258	journal	May 2015
Inter-phase heat transfer and energy coupling in turbulent dispersed multiphase flows Ling, Y.; Balachandar, S.; Parmar, M. Physics of Fluids, Vol. 28, Issue 3 https://doi.org/10.1063/1.4942184	journal	March 2016
Barycentric Lagrange Interpolation Berrut, Jean-Paul; Trefethen, Lloyd N. SIAM Review, Vol. 46, Issue 3 https://doi.org/10.1137/S0036144502417715	journal	January 2004
Numerical Simulation of Dense Gas-Solid Fluidized Beds: A Multiscale Modeling Strategy van der Hoef, M. A.; van Sint Annaland, M.; Deen, N. G. Annual Review of Fluid Mechanics, Vol. 40, Issue 1 https://doi.org/10.1146/annurev.fluid.40.111406.102130	journal	January 2008
Total variation diminishing Runge-Kutta schemes Gottlieb, Sigal; Shu, Chi-Wang Mathematics of Computation of the American Mathematical Society, Vol. 67, Issue 221 https://doi.org/10.1090/S0025-5718-98-00913-2	journal	January 1998
Multiphase flow above explosion sites in debris-filled volcanic vents: Insights from analogue experiments Ross, Pierre-Simon; White, James D. L.; Zimanowski, Bernd Journal of Volcanology and Geothermal Research, Vol. 178, Issue 1 https://doi.org/10.1016/j.jvolgeores.2008.01.013	journal	November 2008
Fast Parallel Direct Solvers for Coarse Grid Problems Tufo, H. M.; Fischer, P. F. Journal of Parallel and Distributed Computing, Vol. 61, Issue 2 https://doi.org/10.1006/jpdc.2000.1676	journal	February 2001
Particle-laden turbulent flows: direct simulation and closure models Elghobashi, S. Applied Scientific Research, Vol. 48, Issue 3-4 https://doi.org/10.1007/BF02008202	journal	October 1991
A spectral element method for fluid dynamics: Laminar flow in a channel expansion Patera, Anthony T. Journal of Computational Physics, Vol. 54, Issue 3 https://doi.org/10.1016/0021-9991(84)90128-1	journal	June 1984
Turbulence modulation and drag reduction by spherical particles Zhao, L. H.; Andersson, H. I.; Gillissen, J. J. J. Physics of Fluids, Vol. 22, Issue 8 https://doi.org/10.1063/1.3478308	journal	August 2010
A scaling analysis of added-mass and history forces and their coupling in dispersed multiphase flows Ling, Y.; Parmar, M.; Balachandar, S. International Journal of Multiphase Flow, Vol. 57 https://doi.org/10.1016/j.ijmultiphaseflow.2013.07.005	journal	December 2013
Revisiting the Wen and Yu Equations for Minimum Fluidization Velocity Prediction Delebarre, A. Chemical Engineering Research and Design, Vol. 82, Issue 5 https://doi.org/10.1205/026387604323142621	journal	May 2004
Validation of a discrete element model using magnetic resonance measurements Müller, Christoph R.; Scott, Stuart A.; Holland, Daniel J. Particuology, Vol. 7, Issue 4 https://doi.org/10.1016/j.partic.2009.04.002	journal	August 2009
A generalized method for predicting the minimum fluidization velocity Wen, C. Y.; Yu, Y. H. AIChE Journal, Vol. 12, Issue 3 https://doi.org/10.1002/aic.690120343	journal	May 1966
Improving particle drag predictions in Euler–Lagrange simulations with two-way coupling Ireland, Peter J.; Desjardins, Olivier Journal of Computational Physics, Vol. 338 https://doi.org/10.1016/j.jcp.2017.02.070	journal	June 2017
Interaction of a planar shock wave with a dense particle curtain: Modeling and experiments Ling, Y.; Wagner, J. L.; Beresh, S. J. Physics of Fluids, Vol. 24, Issue 11 https://doi.org/10.1063/1.4768815	journal	November 2012
A scaling analysis for point–particle approaches to turbulent multiphase flows Balachandar, S. International Journal of Multiphase Flow, Vol. 35, Issue 9 https://doi.org/10.1016/j.ijmultiphaseflow.2009.02.013	journal	September 2009
A discrete numerical model for granular assemblies Cundall, P. A.; Strack, O. D. L. Géotechnique, Vol. 29, Issue 1 https://doi.org/10.1680/geot.1979.29.1.47	journal	March 1979

Similar Records

A scalable Euler–Lagrange approach for multiphase flow simulation on spectral elements

Journal Article · Mon Aug 12 00:00:00 EDT 2019 · International Journal of High Performance Computing Applications · OSTI ID:1557001

Zwick, David; Balachandar, S.

Numerical Analysis of the Dynamics of Two- and Three-Dimensional Fluidized Bed Reactors using an Euler-Lagrange Approach

Journal Article · Sun Apr 01 00:00:00 EDT 2012 · Powder Technology · OSTI ID:1557001

Pepiot, Perrine; Desjardins, Oliver

Mesoscale Simulations of Particulate Flows with Parallel Distributed Lagrange Multiplier Technique

Conference · Wed Mar 24 00:00:00 EDT 2010 · OSTI ID:1557001

Kanarska, Y

Title: A scalable Euler–Lagrange approach for multiphase flow simulation on spectral elements

Citation Formats

References (39)

Similar Records

Related Subjects