Direct simulation Monte Carlo on petaflop supercomputers and beyond
Abstract
The gold-standard definition of the Direct Simulation Monte Carlo (DSMC) method is given in the 1994 book by Bird [Molecular Gas Dynamics and the Direct Simulation of Gas Flows (Clarendon Press, Oxford, UK, 1994)], which refined his pioneering earlier papers in which he first formulated the method. In the intervening 25 years, DSMC has become the method of choice for modeling rarefied gas dynamics in a variety of scenarios. The chief concern to applying DSMC to more dense or even continuum flows is its computational expense compared to continuum computational fluid dynamics methods. The dramatic (nearly billion-fold) increase in speed of the largest supercomputers over the last 30 years has thus been a key enabling factor in using DSMC to model a richer variety of flows, due to the method’s inherent parallelism. We have developed the open-source SPARTA DSMC code with the goal of running DSMC efficiently on the largest machines, both current and future. It is largely an implementation of Bird’s 1994 formulation. In this work, we describe algorithms used in SPARTA to enable DSMC to operate in parallel at the scale of many billions of particles or grid cells, or with billions of surface elements. We highlight amore »
- Authors:
-
- Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
- Science and Technology Corporation at NASA Ames Research Center, Moffett Field, California 94035, USA
- Publication Date:
- Research Org.:
- Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
- Sponsoring Org.:
- USDOE National Nuclear Security Administration (NNSA)
- OSTI Identifier:
- 1559524
- Report Number(s):
- SAND-2019-9819J
Journal ID: ISSN 1070-6631; 678677; TRN: US2000361
- Grant/Contract Number:
- AC04-94AL85000; NA0003525
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Physics of Fluids
- Additional Journal Information:
- Journal Volume: 31; Journal Issue: 8; Journal ID: ISSN 1070-6631
- Publisher:
- American Institute of Physics (AIP)
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 97 MATHEMATICS AND COMPUTING; Direct Simulation Monte Carlo; DSMC; parallel algorithms; SPARTA
Citation Formats
Plimpton, S. J., Moore, S. G., Borner, A., Stagg, A. K., Koehler, T. P., Torczynski, J. R., and Gallis, M. A. Direct simulation Monte Carlo on petaflop supercomputers and beyond. United States: N. p., 2019.
Web. doi:10.1063/1.5108534.
Plimpton, S. J., Moore, S. G., Borner, A., Stagg, A. K., Koehler, T. P., Torczynski, J. R., & Gallis, M. A. Direct simulation Monte Carlo on petaflop supercomputers and beyond. United States. https://doi.org/10.1063/1.5108534
Plimpton, S. J., Moore, S. G., Borner, A., Stagg, A. K., Koehler, T. P., Torczynski, J. R., and Gallis, M. A. Thu .
"Direct simulation Monte Carlo on petaflop supercomputers and beyond". United States. https://doi.org/10.1063/1.5108534. https://www.osti.gov/servlets/purl/1559524.
@article{osti_1559524,
title = {Direct simulation Monte Carlo on petaflop supercomputers and beyond},
author = {Plimpton, S. J. and Moore, S. G. and Borner, A. and Stagg, A. K. and Koehler, T. P. and Torczynski, J. R. and Gallis, M. A.},
abstractNote = {The gold-standard definition of the Direct Simulation Monte Carlo (DSMC) method is given in the 1994 book by Bird [Molecular Gas Dynamics and the Direct Simulation of Gas Flows (Clarendon Press, Oxford, UK, 1994)], which refined his pioneering earlier papers in which he first formulated the method. In the intervening 25 years, DSMC has become the method of choice for modeling rarefied gas dynamics in a variety of scenarios. The chief concern to applying DSMC to more dense or even continuum flows is its computational expense compared to continuum computational fluid dynamics methods. The dramatic (nearly billion-fold) increase in speed of the largest supercomputers over the last 30 years has thus been a key enabling factor in using DSMC to model a richer variety of flows, due to the method’s inherent parallelism. We have developed the open-source SPARTA DSMC code with the goal of running DSMC efficiently on the largest machines, both current and future. It is largely an implementation of Bird’s 1994 formulation. In this work, we describe algorithms used in SPARTA to enable DSMC to operate in parallel at the scale of many billions of particles or grid cells, or with billions of surface elements. We highlight a few examples of the kinds of fundamental physics questions and engineering applications that DSMC can address at these scales.},
doi = {10.1063/1.5108534},
journal = {Physics of Fluids},
number = 8,
volume = 31,
place = {United States},
year = {Thu Aug 01 00:00:00 EDT 2019},
month = {Thu Aug 01 00:00:00 EDT 2019}
}
Web of Science
Figures / Tables:
Works referenced in this record:
Application highlights of the DSMC Analysis Code (DAC) software for simulating rarefied flows
journal, December 2001
- LeBeau, G. J.; Lumpkin III, F. E.
- Computer Methods in Applied Mechanics and Engineering, Vol. 191, Issue 6-7
Robust cut-cell algorithms for DSMC implementations employing multi-level Cartesian grids
journal, October 2012
- Zhang, Chonglin; Schwartzentruber, Thomas E.
- Computers & Fluids, Vol. 69
An overview of Rayleigh-Taylor instability
journal, July 1984
- Sharp, D. H.
- Physica D: Nonlinear Phenomena, Vol. 12, Issue 1-3
Kokkos: Enabling manycore performance portability through polymorphic memory access patterns
journal, December 2014
- Carter Edwards, H.; Trott, Christian R.; Sunderland, Daniel
- Journal of Parallel and Distributed Computing, Vol. 74, Issue 12
Efficient Implementation of Marching Cubes' Cases with Topological Guarantees
journal, January 2003
- Lewiner, Thomas; Lopes, Hélio; Vieira, Antônio Wilson
- Journal of Graphics Tools, Vol. 8, Issue 2
Rayleigh–Taylor and Richtmyer–Meshkov instability induced flow, turbulence, and mixing. I
journal, December 2017
- Zhou, Ye
- Physics Reports, Vol. 720-722
Rayleigh–Taylor and Richtmyer–Meshkov instability induced flow, turbulence, and mixing. II
journal, December 2017
- Zhou, Ye
- Physics Reports, Vol. 723-725
PuMA: the Porous Microstructure Analysis software
journal, January 2018
- Ferguson, Joseph C.; Panerai, Francesco; Borner, Arnaud
- SoftwareX, Vol. 7
A new algorithm for Monte Carlo simulation of Ising spin systems
journal, January 1975
- Bortz, A. B.; Kalos, M. H.; Lebowitz, J. L.
- Journal of Computational Physics, Vol. 17, Issue 1
High temperature permeability of fibrous materials using direct simulation Monte Carlo
journal, March 2017
- Borner, Arnaud; Panerai, Francesco; Mansour, Nagi N.
- International Journal of Heat and Mass Transfer, Vol. 106
Nonequilibrium flow through porous thermal protection materials, Part I: Numerical methods
journal, March 2019
- Stern, Eric C.; Poovathingal, Savio; Nompelis, Ioannis
- Journal of Computational Physics, Vol. 380
The instability of liquid surfaces when accelerated in a direction perpendicular to their planes. II
journal, June 1950
- Lewis, D. J.
- Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences, Vol. 202, Issue 1068, p. 81-96
Hidden surface removal using polygon area sorting
journal, August 1977
- Weiler, Kevin; Atherton, Peter
- ACM SIGGRAPH Computer Graphics, Vol. 11, Issue 2
Effects of Diffusion on Interface Instability between Gases
journal, January 1962
- Duff, R. E.; Harlow, F. H.; Hirt, C. W.
- Physics of Fluids, Vol. 5, Issue 4
Statistical simulation of the transition between regular and mach reflection in steady flows
journal, January 1998
- Ivanov, M. S.; Gimelshein, S. F.; Markelov, G. N.
- Computers & Mathematics with Applications, Vol. 35, Issue 1-2
An open source, parallel DSMC code for rarefied gas flows in arbitrary geometries
journal, December 2010
- Scanlon, T. J.; Roohi, E.; White, C.
- Computers & Fluids, Vol. 39, Issue 10
A new Strouhal–Reynolds-number relationship for the circular cylinder in the range 47<Re<2×105
journal, July 1998
- Fey, Uwe; König, Michael; Eckelmann, Helmut
- Physics of Fluids, Vol. 10, Issue 7
Practical considerations on Marching Cubes 33 topological correctness
journal, November 2013
- Custodio, Lis; Etiene, Tiago; Pesco, Sinesio
- Computers & Graphics, Vol. 37, Issue 7
Direct simulation Monte Carlo investigation of the Richtmyer-Meshkov instability
journal, August 2015
- Gallis, M. A.; Koehler, T. P.; Torczynski, J. R.
- Physics of Fluids, Vol. 27, Issue 8
Direct Simulation Monte Carlo Simulations of Hypersonic Flows With Shock Interactions
journal, December 2005
- Moss, James N.; Bird, Graeme A.
- AIAA Journal, Vol. 43, Issue 12
Exact stochastic simulation of coupled chemical reactions
journal, December 1977
- Gillespie, Daniel T.
- The Journal of Physical Chemistry, Vol. 81, Issue 25
Modeling the oxidation of low-density carbon fiber material based on micro-tomography
journal, January 2016
- Ferguson, Joseph C.; Panerai, Francesco; Lachaud, Jean
- Carbon, Vol. 96
Gas-kinetic simulation of sustained turbulence in minimal Couette flow
journal, July 2018
- Gallis, M. A.; Torczynski, J. R.; Bitter, N. P.
- Physical Review Fluids, Vol. 3, Issue 7
An experimental assembly for precise measurement of thermal accommodation coefficients
journal, March 2011
- Trott, Wayne M.; Castañeda, Jaime N.; Torczynski, John R.
- Review of Scientific Instruments, Vol. 82, Issue 3
Nanohydrodynamics simulations: An atomistic view of the Rayleigh-Taylor instability
journal, April 2004
- Kadau, K.; Germann, T. C.; Hadjiconstantinou, N. G.
- Proceedings of the National Academy of Sciences, Vol. 101, Issue 16
Marching cubes: A high resolution 3D surface construction algorithm
journal, August 1987
- Lorensen, William E.; Cline, Harvey E.
- ACM SIGGRAPH Computer Graphics, Vol. 21, Issue 4
Approach to Translational Equilibrium in a Rigid Sphere Gas
journal, January 1963
- Bird, G. A.
- Physics of Fluids, Vol. 6, Issue 10
The instability of liquid surfaces when accelerated in a direction perpendicular to their planes. I
journal, March 1950
- Taylor, Geoffrey Ingram
- Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences, Vol. 201, Issue 1065, p. 192-196
A general method for numerically simulating the stochastic time evolution of coupled chemical reactions
journal, December 1976
- Gillespie, Daniel T.
- Journal of Computational Physics, Vol. 22, Issue 4
Atomistic methods in fluid simulation
journal, April 2010
- Kadau, Kai; Barber, John L.; Germann, Timothy C.
- Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 368, Issue 1916
dsmcFoam+: An OpenFOAM based direct simulation Monte Carlo solver
journal, March 2018
- White, C.; Borg, M. K.; Scanlon, T. J.
- Computer Physics Communications, Vol. 224
Investigation of the ellipsoidal-statistical Bhatnagar–Gross–Krook kinetic model applied to gas-phase transport of heat and tangential momentum between parallel walls
journal, March 2011
- Gallis, M. A.; Torczynski, J. R.
- Physics of Fluids, Vol. 23, Issue 3
3D DSMC computations on a heterogeneous CPU-GPU cluster with a large number of GPUs
conference, January 2014
- Kashkovsky, Alexander
- PROCEEDINGS OF THE 29TH INTERNATIONAL SYMPOSIUM ON RAREFIED GAS DYNAMICS, AIP Conference Proceedings
On the Instability of Superposed Fluids in a Gravitational Field.
journal, July 1955
- Layzer, David
- The Astrophysical Journal, Vol. 122
A parallel rendezvous algorithm for interpolation between multiple grids
journal, February 2004
- Plimpton, Steven J.; Hendrickson, Bruce; Stewart, James R.
- Journal of Parallel and Distributed Computing, Vol. 64, Issue 2
Review of theoretical modelling approaches of Rayleigh–Taylor instabilities and turbulent mixing
journal, April 2010
- Abarzhi, Snezhana I.
- Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 368, Issue 1916
Atomistic simulation of the Rayleigh-Taylor instability
journal, September 2006
- Barber, J. L.; Kadau, K.; Germann, T. C.
- Journal of Physics: Conference Series, Vol. 46
A Parallel Rendezvous Algorithm for Interpolation Between Multiple Grids
conference, January 1998
- Plimpton, S.; Hendrickson, B.; Stewart, J.
- SC98 - High Performance Networking and Computing Conference, Proceedings of the IEEE/ACM SC98 Conference
Marching cubes: A high resolution 3D surface construction algorithm
conference, January 1987
- Lorensen, William E.; Cline, Harvey E.
- Proceedings of the 14th annual conference on Computer graphics and interactive techniques - SIGGRAPH '87
Robust Cut-cell Algorithms for DSMC Implementations Employing Multi-level Cartesian Grids
conference, June 2011
- Zhang, Chonglin; Schwartzentruber, Thomas
- 42nd AIAA Thermophysics Conference
Hidden surface removal using polygon area sorting
conference, January 1977
- Weiler, Kevin; Atherton, Peter
- Proceedings of the 4th annual conference on Computer graphics and interactive techniques - SIGGRAPH '77
Works referencing / citing this record: