DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: A scalable Euler–Lagrange approach for multiphase flow simulation on spectral elements

Abstract

Multiphase flow can be difficult to simulate with high accuracy due to the wide range of scales associated with various multiphase phenomena. These scales may range from the size of individual particles to the entire domain of interest. Traditionally, large scale systems can only be simulated using averaging approaches that filter out the locations of individual particles. In this work, the Euler–Lagrange method is used to simulate large-scale dense particle systems in which each individual particle is tracked. In order to accomplish this, the highly scalable spectral element code nek5000 has been extended to handle the multiple levels of multiphase coupling in these systems. These levels include what has been called one-, two-, and four-way coupling. Here, each level has been separated to detail the computational impact of each stage. A binned ghost particle algorithm has also been developed to efficiently handle the challenges of two- and four-way coupling in a parallel processing context. The algorithms and their implementations are then shown to scale to 65,536 Message Passing Interface (MPI) ranks in both the strong and weak limits. After this, validation is performed through simulation of a small-scale fluidized bed. Lastly, a large-scale fluidized bed is simulated with 65,536 MPImore » ranks and is able to capture the unique physics of the onset of fluidization.« less

Authors:
ORCiD logo; ORCiD logo [1]
  1. Department of Mechanical and Aerospace Engineering, University of Florida, Gainesville, FL, USA
Publication Date:
Sponsoring Org.:
USDOE
OSTI Identifier:
1557001
Resource Type:
Published Article
Journal Name:
International Journal of High Performance Computing Applications
Additional Journal Information:
Journal Name: International Journal of High Performance Computing Applications Journal Volume: 34 Journal Issue: 3; Journal ID: ISSN 1094-3420
Publisher:
SAGE Publications
Country of Publication:
United States
Language:
English

Citation Formats

Zwick, David, and Balachandar, S. A scalable Euler–Lagrange approach for multiphase flow simulation on spectral elements. United States: N. p., 2019. Web. doi:10.1177/1094342019867756.
Zwick, David, & Balachandar, S. A scalable Euler–Lagrange approach for multiphase flow simulation on spectral elements. United States. https://doi.org/10.1177/1094342019867756
Zwick, David, and Balachandar, S. Mon . "A scalable Euler–Lagrange approach for multiphase flow simulation on spectral elements". United States. https://doi.org/10.1177/1094342019867756.
@article{osti_1557001,
title = {A scalable Euler–Lagrange approach for multiphase flow simulation on spectral elements},
author = {Zwick, David and Balachandar, S.},
abstractNote = {Multiphase flow can be difficult to simulate with high accuracy due to the wide range of scales associated with various multiphase phenomena. These scales may range from the size of individual particles to the entire domain of interest. Traditionally, large scale systems can only be simulated using averaging approaches that filter out the locations of individual particles. In this work, the Euler–Lagrange method is used to simulate large-scale dense particle systems in which each individual particle is tracked. In order to accomplish this, the highly scalable spectral element code nek5000 has been extended to handle the multiple levels of multiphase coupling in these systems. These levels include what has been called one-, two-, and four-way coupling. Here, each level has been separated to detail the computational impact of each stage. A binned ghost particle algorithm has also been developed to efficiently handle the challenges of two- and four-way coupling in a parallel processing context. The algorithms and their implementations are then shown to scale to 65,536 Message Passing Interface (MPI) ranks in both the strong and weak limits. After this, validation is performed through simulation of a small-scale fluidized bed. Lastly, a large-scale fluidized bed is simulated with 65,536 MPI ranks and is able to capture the unique physics of the onset of fluidization.},
doi = {10.1177/1094342019867756},
journal = {International Journal of High Performance Computing Applications},
number = 3,
volume = 34,
place = {United States},
year = {Mon Aug 12 00:00:00 EDT 2019},
month = {Mon Aug 12 00:00:00 EDT 2019}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
https://doi.org/10.1177/1094342019867756

Citation Metrics:
Cited by: 37 works
Citation information provided by
Web of Science

Save / Share:

Works referenced in this record:

A Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs
journal, January 1998


The numerical stability of barycentric Lagrange interpolation
journal, October 2004


Scaling Limits for PDE-Based Simulation (Invited)
conference, June 2015


Methods for evaluating fluid velocities in spectral simulations of turbulence
journal, July 1989


ppiclF: A Parallel Particle-In-Cell Library in Fortran
journal, May 2019


Modeling and numerical simulation of particulate flows by the Eulerian–Lagrangian approach
journal, October 2001


Dynamic alignment and distribution of irregularly coupled data arrays for scalable parallelization of particle-in-cell problems
conference, January 1996

  • Wei-Keng Liao, ; Ranka, S.
  • Proceedings of International Conference on Parallel Processing
  • DOI: 10.1109/IPPS.1996.508039

Partitioning of unstructured problems for parallel processing
journal, January 1991


Visualization of the Bulle-Effect at River Bifurcations
conference, January 2018

  • Dutta, Som; Moer, Mark W. Van; Fischer, Paul
  • Proceedings of the Practice and Experience on Advanced Research Computing - PEARC '18
  • DOI: 10.1145/3219104.3229433

Discrete particle simulation of two-dimensional fluidized bed
journal, October 1993


On predicting particle-laden turbulent flows
journal, June 1994


Determination of the normal spring stiffness coefficient in the linear spring–dashpot contact model of discrete element method
journal, September 2013


Particle transport in turbulent curved pipe flow
journal, March 2016

  • Noorani, Azad; Sardina, Gaetano; Brandt, Luca
  • Journal of Fluid Mechanics, Vol. 793
  • DOI: 10.1017/jfm.2016.136

A proof of the Kepler conjecture
journal, January 2005


Accurate signal reconstruction for higher order Lagrangian–Eulerian back-coupling in multiphase turbulence
journal, September 2017


An Euler–Lagrange strategy for simulating particle-laden flows
journal, April 2013


Accurate calculation of Stokes drag for point–particle tracking in two-way coupled flows
journal, August 2016


Dynamic Load Balancing for Compressible Multiphase Turbulence
conference, January 2018

  • Zhai, Keke; Banerjee, Tania; Zwick, David
  • Proceedings of the 2018 International Conference on Supercomputing - ICS '18
  • DOI: 10.1145/3205289.3205304

OpenACC acceleration of the Nek5000 spectral element code
journal, March 2015

  • Markidis, Stefano; Gong, Jing; Schliephake, Michael
  • The International Journal of High Performance Computing Applications, Vol. 29, Issue 3
  • DOI: 10.1177/1094342015576846

Turbulent Dispersed Multiphase Flow
journal, January 2010


Fluid Mechanical Description of Fluidized Beds. Equations of Motion
journal, November 1967

  • Anderson, T. B.; Jackson, Roy
  • Industrial & Engineering Chemistry Fundamentals, Vol. 6, Issue 4
  • DOI: 10.1021/i160024a007

Exact regularized point particle method for multiphase flows in the two-way coupling regime
journal, May 2015

  • Gualtieri, P.; Picano, F.; Sardina, G.
  • Journal of Fluid Mechanics, Vol. 773
  • DOI: 10.1017/jfm.2015.258

Inter-phase heat transfer and energy coupling in turbulent dispersed multiphase flows
journal, March 2016

  • Ling, Y.; Balachandar, S.; Parmar, M.
  • Physics of Fluids, Vol. 28, Issue 3
  • DOI: 10.1063/1.4942184

Barycentric Lagrange Interpolation
journal, January 2004


Numerical Simulation of Dense Gas-Solid Fluidized Beds: A Multiscale Modeling Strategy
journal, January 2008


Total variation diminishing Runge-Kutta schemes
journal, January 1998

  • Gottlieb, Sigal; Shu, Chi-Wang
  • Mathematics of Computation of the American Mathematical Society, Vol. 67, Issue 221
  • DOI: 10.1090/S0025-5718-98-00913-2

Multiphase flow above explosion sites in debris-filled volcanic vents: Insights from analogue experiments
journal, November 2008

  • Ross, Pierre-Simon; White, James D. L.; Zimanowski, Bernd
  • Journal of Volcanology and Geothermal Research, Vol. 178, Issue 1
  • DOI: 10.1016/j.jvolgeores.2008.01.013

Fast Parallel Direct Solvers for Coarse Grid Problems
journal, February 2001

  • Tufo, H. M.; Fischer, P. F.
  • Journal of Parallel and Distributed Computing, Vol. 61, Issue 2
  • DOI: 10.1006/jpdc.2000.1676

Particle-laden turbulent flows: direct simulation and closure models
journal, October 1991


A spectral element method for fluid dynamics: Laminar flow in a channel expansion
journal, June 1984


Turbulence modulation and drag reduction by spherical particles
journal, August 2010

  • Zhao, L. H.; Andersson, H. I.; Gillissen, J. J. J.
  • Physics of Fluids, Vol. 22, Issue 8
  • DOI: 10.1063/1.3478308

A scaling analysis of added-mass and history forces and their coupling in dispersed multiphase flows
journal, December 2013


Revisiting the Wen and Yu Equations for Minimum Fluidization Velocity Prediction
journal, May 2004


Validation of a discrete element model using magnetic resonance measurements
journal, August 2009


A generalized method for predicting the minimum fluidization velocity
journal, May 1966


Improving particle drag predictions in Euler–Lagrange simulations with two-way coupling
journal, June 2017


Interaction of a planar shock wave with a dense particle curtain: Modeling and experiments
journal, November 2012

  • Ling, Y.; Wagner, J. L.; Beresh, S. J.
  • Physics of Fluids, Vol. 24, Issue 11
  • DOI: 10.1063/1.4768815

A scaling analysis for point–particle approaches to turbulent multiphase flows
journal, September 2009


A discrete numerical model for granular assemblies
journal, March 1979