EMMA: an adaptive mesh refinement cosmological simulation code with radiative transfer

Aubert, Dominique; Deparis, Nicolas; Ocvirk, Pierre

doi:10.1093/mnras/stv1896

EMMA: an adaptive mesh refinement cosmological simulation code with radiative transfer

Journal Article · Wed Sep 30 00:00:00 EDT 2015 · Monthly Notices of the Royal Astronomical Society

DOI:https://doi.org/10.1093/mnras/stv1896· OSTI ID:1565490

Aubert, Dominique ^[1]; Deparis, Nicolas ^[1]; Ocvirk, Pierre ^[1]

Univ. de Strasbourg, Strasbourg (France)

EMMA is a cosmological simulation code aimed at investigating the reionization epoch. It handles simultaneously collisionless and gas dynamics, as well as radiative transfer physics using a moment-based description with the M1 approximation. Field quantities are stored and computed on an adaptive three-dimensional mesh and the spatial resolution can be dynamically modified based on physically motivated criteria. Physical processes can be coupled at all spatial and temporal scales. We also introduce a new and optional approximation to handle radiation: the light is transported at the resolution of the non-refined grid and only once the dynamics has been fully updated, whereas thermo-chemical processes are still tracked on the refined elements. Such an approximation reduces the overheads induced by the treatment of radiation physics. A suite of standard tests are presented and passed by EMMA, providing a validation for its future use in studies of the reionization epoch. The code is parallel and is able to use graphics processing units (GPUs) to accelerate hydrodynamics and radiative transfer calculations. Depending on the optimizations and the compilers used to generate the CPU reference, global GPU acceleration factors between ×3.9 and ×16.9 can be obtained. As a result, vectorization and transfer operations currently prevent better GPU performance and we expect that future optimizations and hardware evolution will lead to greater accelerations.

View Accepted Manuscript (DOE)

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF)

Sponsoring Organization:: USDOE Office of Science (SC)

Grant/Contract Number:: AC05-00OR22725

OSTI ID:: 1565490

Journal Information:: Monthly Notices of the Royal Astronomical Society, Journal Name: Monthly Notices of the Royal Astronomical Society Journal Issue: 1 Vol. 454; ISSN 0035-8711

Publisher:: Royal Astronomical SocietyCopyright Statement

Country of Publication:: United States

Language:: English

References (54)

Fully Threaded Tree Algorithms for Adaptive Refinement Fluid Dynamics Simulations Khokhlov, A. M. Journal of Computational Physics, Vol. 143, Issue 2 https://doi.org/10.1006/jcph.1998.9998	journal	July 1998
Restoration of the contact surface in the HLL-Riemann solver Toro, E. F.; Spruce, M.; Speares, W. Shock Waves, Vol. 4, Issue 1 https://doi.org/10.1007/BF01414629	journal	July 1994
Relating Eddington factors to flux limiters Levermore, C. D. Journal of Quantitative Spectroscopy and Radiative Transfer, Vol. 31, Issue 2 https://doi.org/10.1016/0022-4073(84)90112-2	journal	February 1984
In the beginning: the first sources of light and the reionization of the universe Barkana, Rennan; Loeb, Abraham Physics Reports, Vol. 349, Issue 2 https://doi.org/10.1016/S0370-1573(01)00019-9	journal	July 2001
Multi-dimensional cosmological radiative transfer with a Variable Eddington Tensor formalism Gnedin, Nickolay Y.; Abel, Tom New Astronomy, Vol. 6, Issue 7 https://doi.org/10.1016/S1384-1076(01)00068-9	journal	October 2001
A parallel tree code Dubinski, John New Astronomy, Vol. 1, Issue 2 https://doi.org/10.1016/S1384-1076(96)00009-7	journal	October 1996
A convenient set of comoving cosmological variables and their application Martel, H.; Shapiro, P. R. Monthly Notices of the Royal Astronomical Society, Vol. 297, Issue 2 https://doi.org/10.1046/j.1365-8711.1998.01497.x	journal	June 1998
P3M-SPH simulations of the Lyα forest Theuns, Tom; Leonard, Anthony; Efstathiou, George Monthly Notices of the Royal Astronomical Society, Vol. 301, Issue 2 https://doi.org/10.1046/j.1365-8711.1998.02040.x	journal	December 1998
Ellipsoidal collapse and an improved model for the number and spatial distribution of dark matter haloes Sheth, R. K.; Mo, H. J.; Tormen, G. Monthly Notices of the Royal Astronomical Society, Vol. 323, Issue 1 https://doi.org/10.1046/j.1365-8711.2001.04006.x	journal	May 2001
Including star formation and supernova feedback within cosmological simulations of galaxy formation Kay, Scott T.; Pearce, Frazer R.; Frenk, Carlos S. Monthly Notices of the Royal Astronomical Society, Vol. 330, Issue 1 https://doi.org/10.1046/j.1365-8711.2002.05070.x	journal	February 2002
Simulating intergalactic medium reionization Ciardi, B.; Stoehr, F.; White, S. D. M. Monthly Notices of the Royal Astronomical Society, Vol. 343, Issue 4 https://doi.org/10.1046/j.1365-8711.2003.06797.x	journal	August 2003
Reionization by UV or X-ray sources Baek, S.; Semelin, B.; Di Matteo, P. Astronomy & Astrophysics, Vol. 523 https://doi.org/10.1051/0004-6361/201014347	journal	November 2010
A study of simulated reionization histories with merger trees of HII regions Chardin, J.; Aubert, D.; Ocvirk, P. Astronomy & Astrophysics, Vol. 548 https://doi.org/10.1051/0004-6361/201219992	journal	November 2012
Planck 2013 results. XVI. Cosmological parameters Ade, P. A. R.; Aghanim, N.; Armitage-Caplan, C. Astronomy & Astrophysics, Vol. 571 https://doi.org/10.1051/0004-6361/201321591	journal	October 2014
Cosmological hydrodynamics with adaptive mesh refinement: A new high resolution code called RAMSES Teyssier, R. Astronomy & Astrophysics, Vol. 385, Issue 1 https://doi.org/10.1051/0004-6361:20011817	journal	April 2002
The history of the baryon budget: Cosmic logistics in a hierarchical universe Rasera, Y.; Teyssier, R. Astronomy & Astrophysics, Vol. 445, Issue 1 https://doi.org/10.1051/0004-6361:20053116	journal	December 2005
HERACLES: a three-dimensional radiation hydrodynamics code González, M.; Audit, E.; Huynh, P. Astronomy & Astrophysics, Vol. 464, Issue 2 https://doi.org/10.1051/0004-6361:20065486	journal	January 2007
On the onset of galactic winds in quiescent star forming galaxies Dubois, Y.; Teyssier, R. Astronomy & Astrophysics, Vol. 477, Issue 1 https://doi.org/10.1051/0004-6361:20078326	journal	October 2007
The simulated 21 cm signal during the epoch of reionization: full modeling of the Ly- α pumping Baek, S.; Di Matteo, P.; Semelin, B. Astronomy & Astrophysics, Vol. 495, Issue 2 https://doi.org/10.1051/0004-6361:200810757	journal	December 2008
Self-similar secondary infall and accretion in an Einstein-de Sitter universe Bertschinger, E. The Astrophysical Journal Supplement Series, Vol. 58 https://doi.org/10.1086/191028	journal	May 1985
Cosmological Simulations with TreeSPH Katz, Neal; Weinberg, David H.; Hernquist, Lars The Astrophysical Journal Supplement Series, Vol. 105 https://doi.org/10.1086/192305	journal	July 1996
Baryonic Features in the Matter Transfer Function Eisenstein, Daniel J.; Hu, Wayne The Astrophysical Journal, Vol. 496, Issue 2 https://doi.org/10.1086/305424	journal	April 1998
HOP: A New Group‐finding Algorithm for N ‐Body Simulations Eisenstein, Daniel J.; Hut, Piet The Astrophysical Journal, Vol. 498, Issue 1 https://doi.org/10.1086/305535	journal	May 1998
Adaptive Refinement Tree: A New High‐Resolution N ‐Body Code for Cosmological Simulations Kravtsov, Andrey V.; Klypin, Anatoly A.; Khokhlov, Alexei M. The Astrophysical Journal Supplement Series, Vol. 111, Issue 1 https://doi.org/10.1086/313015	journal	July 1997
Effect of Reionization on Structure Formation in the Universe Gnedin, Nickolay Y. The Astrophysical Journal, Vol. 542, Issue 2 https://doi.org/10.1086/317042	journal	October 2000
Constraining the Evolution of the Ionizing Background and the Epoch of Reionization with z ~ 6 Quasars. II. A Sample of 19 Quasars Fan, Xiaohui; Strauss, Michael A.; Becker, Robert H. The Astronomical Journal, Vol. 132, Issue 1 https://doi.org/10.1086/504836	journal	June 2006
Simulations and Analytic Calculations of Bubble Growth during Hydrogen Reionization Zahn, Oliver; Lidz, Adam; McQuinn, Matthew The Astrophysical Journal, Vol. 654, Issue 1 https://doi.org/10.1086/509597	journal	January 2007
Radiative Transfer Simulations of Cosmic Reionization. I. Methodology and Initial Results Trac, Hy; Cen, Renyue The Astrophysical Journal, Vol. 671, Issue 1 https://doi.org/10.1086/522566	journal	December 2007
Initial Conditions For Large Cosmological Simulations Prunet, S.; Pichon, C.; Aubert, D. The Astrophysical Journal Supplement Series, Vol. 178, Issue 2 https://doi.org/10.1086/590370	journal	October 2008
Reionization Simulations Powered by Graphics Processing Units. i. on the Structure of the Ultraviolet Radiation Field Aubert, Dominique; Teyssier, Romain The Astrophysical Journal, Vol. 724, Issue 1 https://doi.org/10.1088/0004-637X/724/1/244	journal	November 2010
Galactic Outflows and Photoionization Heating in the Reionization Epoch Finlator, K.; Davé, R.; Özel, F. The Astrophysical Journal, Vol. 743, Issue 2 https://doi.org/10.1088/0004-637X/743/2/169	journal	December 2011
High-Resolution Simulations of the Reionization of an Isolated Milky Way-M31 Galaxy pair Ocvirk, P.; Aubert, D.; Chardin, J. The Astrophysical Journal, Vol. 777, Issue 1 https://doi.org/10.1088/0004-637X/777/1/51	journal	October 2013
Cosmic Reionization on Computers. i. Design and Calibration of Simulations Gnedin, Nickolay Y. The Astrophysical Journal, Vol. 793, Issue 1 https://doi.org/10.1088/0004-637X/793/1/29	journal	September 2014
The Reionization of Galactic Satellite Populations Ocvirk, P.; Gillet, N.; Aubert, D. The Astrophysical Journal, Vol. 794, Issue 1 https://doi.org/10.1088/0004-637X/794/1/20	journal	September 2014
UV LUMINOSITY FUNCTIONS AT REDSHIFTS z ∼ 4 TO z ∼ 10: 10,000 GALAXIES FROM HST LEGACY FIELDS Bouwens, R. J.; Illingworth, G. D.; Oesch, P. A. The Astrophysical Journal, Vol. 803, Issue 1 https://doi.org/10.1088/0004-637X/803/1/34	journal	April 2015
21 cm cosmology in the 21st century Pritchard, Jonathan R.; Loeb, Abraham Reports on Progress in Physics, Vol. 75, Issue 8 https://doi.org/10.1088/0034-4885/75/8/086901	journal	July 2012
Fully Coupled Simulation of Cosmic Reionization. i. Numerical Methods and Tests Norman, Michael L.; Reynolds, Daniel R.; So, Geoffrey C. The Astrophysical Journal Supplement Series, Vol. 216, Issue 1 https://doi.org/10.1088/0067-0049/216/1/16	journal	January 2015
Equation of state of the photoionized intergalactic medium Hui, L.; Gnedin, N. Y. Monthly Notices of the Royal Astronomical Society, Vol. 292, Issue 1 https://doi.org/10.1093/mnras/292.1.27	journal	November 1997
ramses-rt: radiation hydrodynamics in the cosmological context Rosdahl, J.; Blaizot, J.; Aubert, D. Monthly Notices of the Royal Astronomical Society, Vol. 436, Issue 3 https://doi.org/10.1093/mnras/stt1722	journal	October 2013
Light-cone anisotropy in the 21 cm signal from the epoch of reionization Zawada, Karolina; Semelin, Benoît; Vonlanthen, Patrick Monthly Notices of the Royal Astronomical Society, Vol. 439, Issue 2 https://doi.org/10.1093/mnras/stu035	journal	February 2014
Spatially adaptive radiation-hydrodynamical simulations of galaxy formation during cosmological reionization Pawlik, Andreas H.; Schaye, Joop; Vecchia, Claudio Dalla Monthly Notices of the Royal Astronomical Society, Vol. 451, Issue 2 https://doi.org/10.1093/mnras/stv976	journal	June 2015
The First Billion Years project: proto-galaxies reionizing the Universe Paardekooper, Jan-Pieter; Khochfar, Sadegh; Dalla, Claudio Vecchia Monthly Notices of the Royal Astronomical Society: Letters, Vol. 429, Issue 1 https://doi.org/10.1093/mnrasl/sls032	journal	December 2012
Dwarf galaxies in voids: suppressing star formation with photoheating: Dwarf galaxies in voids Hoeft, Matthias; Yepes, Gustavo; Gottlöber, Stefan Monthly Notices of the Royal Astronomical Society, Vol. 371, Issue 1 https://doi.org/10.1111/j.1365-2966.2006.10678.x	journal	August 2006
Cosmological radiative transfer codes comparison project �� I. The static density field tests Iliev, Ilian T.; Ciardi, Benedetta; Alvarez, Marcelo A. Monthly Notices of the Royal Astronomical Society, Vol. 371, Issue 3 https://doi.org/10.1111/j.1365-2966.2006.10775.x	journal	September 2006
Simulating cosmic reionization at large scales - II. The 21-cm emission features and statistical signals Mellema, Garrelt; Iliev, Ilian T.; Pen, Ue-Li Monthly Notices of the Royal Astronomical Society, Vol. 372, Issue 2 https://doi.org/10.1111/j.1365-2966.2006.10919.x	journal	October 2006
The morphology of H II regions during reionization McQuinn, M.; Lidz, A.; Zahn, O. Monthly Notices of the Royal Astronomical Society, Vol. 377, Issue 3 https://doi.org/10.1111/j.1365-2966.2007.11489.x	journal	May 2007
A radiative transfer scheme for cosmological reionization based on a local Eddington tensor Aubert, Dominique; Teyssier, Romain Monthly Notices of the Royal Astronomical Society, Vol. 387, Issue 1 https://doi.org/10.1111/j.1365-2966.2008.13223.x	journal	June 2008
Mass loss of galaxies due to an ultraviolet background Okamoto, Takashi; Gao, Liang; Theuns, Tom Monthly Notices of the Royal Astronomical Society, Vol. 390, Issue 3 https://doi.org/10.1111/j.1365-2966.2008.13830.x	journal	November 2008
Cosmological radiative transfer comparison project â€“ II. The radiation-hydrodynamic tests Iliev, Ilian T.; Whalen, Daniel; Mellema, Garrelt Monthly Notices of the Royal Astronomical Society, Vol. 400, Issue 3 https://doi.org/10.1111/j.1365-2966.2009.15558.x	journal	December 2009
Measurements of the ultraviolet background at 4.6 < z < 6.4 using the quasar proximity effect★: Proximity-effect measurements of the UVB at high redshift Calverley, Alexander P.; Becker, George D.; Haehnelt, Martin G. Monthly Notices of the Royal Astronomical Society, Vol. 412, Issue 4 https://doi.org/10.1111/j.1365-2966.2010.18072.x	journal	March 2011
How supernova feedback turns dark matter cusps into cores: Supernova feedback and dark matter cores Pontzen, Andrew; Governato, Fabio Monthly Notices of the Royal Astronomical Society, Vol. 421, Issue 4 https://doi.org/10.1111/j.1365-2966.2012.20571.x	journal	March 2012
The birth of a galaxy – II. The role of radiation pressure: Radiation pressure in early dwarf galaxies Wise, John H.; Abel, Tom; Turk, Matthew J. Monthly Notices of the Royal Astronomical Society, Vol. 427, Issue 1 https://doi.org/10.1111/j.1365-2966.2012.21809.x	journal	October 2012
A parallel hashed Oct-Tree N-body algorithm Warren, M. S.; Salmon, J. K. Proceedings of the 1993 ACM/IEEE conference on Supercomputing - Supercomputing '93 https://doi.org/10.1145/169627.169640	conference	January 1993
Computer Simulations of Cosmic Reionization Trac, Hy; Gnedin, Nickolay Y. Advanced Science Letters, Vol. 4, Issue 2 https://doi.org/10.1166/asl.2011.1214	journal	February 2011

Cited By (15)

An Introductory Review on Cosmic Reionization Wise, John H. arXiv https://doi.org/10.48550/arxiv.1907.06653	text	January 2019
Impact of the reduced speed of light approximation on the post-overlap neutral hydrogen fraction in numerical simulations of the epoch of reionization Ocvirk, P.; Aubert, D.; Chardin, J. Astronomy & Astrophysics, Vol. 626 https://doi.org/10.1051/0004-6361/201832923	journal	June 2019
Cosmic reionisation Wise, John H. Contemporary Physics, Vol. 60, Issue 2 https://doi.org/10.1080/00107514.2019.1631548	journal	April 2019
Cosmic Dawn (CoDa): the first radiation-hydrodynamics simulation of reionization and galaxy formation in the Local Universe Ocvirk, Pierre; Gillet, Nicolas; Shapiro, Paul R. Monthly Notices of the Royal Astronomical Society, Vol. 463, Issue 2 https://doi.org/10.1093/mnras/stw2036	journal	August 2016
Simulating an isolated dwarf galaxy with multichannel feedback and chemical yields from individual stars Emerick, Andrew; Bryan, Greg L.; Mac Low, Mordecai-Mark Monthly Notices of the Royal Astronomical Society, Vol. 482, Issue 1 https://doi.org/10.1093/mnras/sty2689	journal	October 2018
The Goldilocks problem of the quasar contribution to reionization Garaldi, Enrico; Compostella, Michele; Porciani, Cristiano Monthly Notices of the Royal Astronomical Society, Vol. 483, Issue 4 https://doi.org/10.1093/mnras/sty3414	journal	December 2018
Inferring the astrophysics of reionization and cosmic dawn from galaxy luminosity functions and the 21-cm signal Park, Jaehong; Mesinger, Andrei; Greig, Bradley Monthly Notices of the Royal Astronomical Society, Vol. 484, Issue 1 https://doi.org/10.1093/mnras/stz032	journal	January 2019
A deep learning model to emulate simulations of cosmic reionization Chardin, Jonathan; Uhlrich, Grégoire; Aubert, Dominique Monthly Notices of the Royal Astronomical Society, Vol. 490, Issue 1 https://doi.org/10.1093/mnras/stz2605	journal	September 2019
Properties of reionization-era galaxies from JWST luminosity functions and 21-cm interferometry Park, Jaehong; Gillet, Nicolas; Mesinger, Andrei Monthly Notices of the Royal Astronomical Society, Vol. 491, Issue 3 https://doi.org/10.1093/mnras/stz3278	journal	November 2019
Challenges and Techniques for Simulating Line Emission Olsen, Karen; Pallottini, Andrea; Wofford, Aida Galaxies, Vol. 6, Issue 4 https://doi.org/10.3390/galaxies6040100	journal	September 2018
The Inhomogeneous Reionization Times of Present-day Galaxies Aubert, Dominique; Deparis, Nicolas; Ocvirk, Pierre The Astrophysical Journal, Vol. 856, Issue 2 https://doi.org/10.3847/2041-8213/aab14d	journal	March 2018
Impact of the reduced speed of light approximation on the post-overlap neutral hydrogen fraction in numerical simulations of the epoch of reionization Ocvirk, P.; Aubert, D.; Chardin, J. arXiv https://doi.org/10.48550/arxiv.1803.02434	text	January 2018
Inferring the astrophysics of reionization and cosmic dawn from galaxy luminosity functions and the 21-cm signal Park, Jaehong; Mesinger, Andrei; Greig, Bradley arXiv https://doi.org/10.48550/arxiv.1809.08995	text	January 2018
Cosmic Dawn (CoDa): the first radiation-hydrodynamics simulation of reionization and galaxy formation in the Local Universe Ocvirk, Pierre; Gillet, Nicolas; Shapiro, Paul R. Oxford University Press https://doi.org/10.5167/uzh-129969	text	January 2016
The Inhomogeneous Reionization Times of Present-day Galaxies Aubert, Dominique; Deparis, Nicolas; Ocvirk, Pierre IOP Publishing https://doi.org/10.5167/uzh-152904	text	January 2018

Similar Records

REIONIZATION SIMULATIONS POWERED BY GRAPHICS PROCESSING UNITS. I. ON THE STRUCTURE OF THE ULTRAVIOLET RADIATION FIELD

Journal Article · Fri Nov 19 23:00:00 EST 2010 · Astrophysical Journal · OSTI ID:21471196

THE EMMA LATTICE.

Conference · Mon Aug 06 00:00:00 EDT 2007 · OSTI ID:922981

The EMMA Lattice

Journal Article · Wed Feb 20 23:00:00 EST 2008 · AIP Conference Proceedings · OSTI ID:21055357

Related Subjects

79 ASTRONOMY AND ASTROPHYSICS
dark ages
first stars
methods: numerical
reionization

EMMA: an adaptive mesh refinement cosmological simulation code with radiative transfer

Citation Formats

References (54)

Cited By (15)

Similar Records

Related Subjects