In situ and in-transit analysis of cosmological simulations

Friesen, Brian; Almgren, Ann; Lukić, Zarija; Weber, Gunther; Morozov, Dmitriy; Beckner, Vincent; Day, Marcus

doi:10.1186/s40668-016-0017-2

Title: In situ and in-transit analysis of cosmological simulations

Authors:

; Almgren, Ann; Lukić, Zarija; Weber, Gunther; Morozov, Dmitriy; Beckner, Vincent; Day, Marcus

Publication Date:: Wed Aug 24 00:00:00 EDT 2016

Research Org.:: Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)

Sponsoring Org.:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR); USDOE Office of Science (SC), High Energy Physics (HEP)

OSTI Identifier:: 1328630

Alternate Identifier(s):: OSTI ID: 1342534; OSTI ID: 1379575

Report Number(s):: LBNL-1006104
Journal ID: ISSN 2197-7909; 4; PII: 17

Grant/Contract Number:: AC02-05CH11231

Resource Type:: Published Article

Journal Name:: Computational Astrophysics and Cosmology

Additional Journal Information:: Journal Name: Computational Astrophysics and Cosmology Journal Volume: 3 Journal Issue: 1; Journal ID: ISSN 2197-7909

Publisher:: Springer Science + Business Media

Country of Publication:: Germany

Language:: English

Subject:: 79 ASTRONOMY AND ASTROPHYSICS; cosmology; post-processing; halo-finding; power spectra; in situ in-transit

Citation Formats


                    Friesen, Brian, Almgren, Ann, Lukić, Zarija, Weber, Gunther, Morozov, Dmitriy, Beckner, Vincent, and Day, Marcus. In situ and in-transit analysis of cosmological simulations.  Germany: N. p., 2016. 
Web.  doi:10.1186/s40668-016-0017-2.

Copy to clipboard


                    Friesen, Brian, Almgren, Ann, Lukić, Zarija, Weber, Gunther, Morozov, Dmitriy, Beckner, Vincent, & Day, Marcus. In situ and in-transit analysis of cosmological simulations.  Germany.  https://doi.org/10.1186/s40668-016-0017-2

Copy to clipboard


                    Friesen, Brian, Almgren, Ann, Lukić, Zarija, Weber, Gunther, Morozov, Dmitriy, Beckner, Vincent, and Day, Marcus. Wed .  
"In situ and in-transit analysis of cosmological simulations".  Germany.  https://doi.org/10.1186/s40668-016-0017-2.

Copy to clipboard


                    
@article{osti_1328630,

  title        = {In situ and in-transit analysis of cosmological simulations},

  author       = {Friesen, Brian and Almgren, Ann and Lukić, Zarija and Weber, Gunther and Morozov, Dmitriy and Beckner, Vincent and Day, Marcus},

  abstractNote = {},

  doi          = {10.1186/s40668-016-0017-2},

  journal      = {Computational Astrophysics and Cosmology},

  number       = 1,

  volume       = 3,

  place        = {Germany},

  year         = {Wed Aug 24 00:00:00 EDT 2016},

  month        = {Wed Aug 24 00:00:00 EDT 2016}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Publisher's Version of Record
https://doi.org/10.1186/s40668-016-0017-2

Other availability

Search WorldCat to find libraries that may hold this journal

Save / Share:

Export Metadata

Save to My Library

Works referenced in this record:

Nucleosynthesis in multi-dimensional SN Ia explosions
journal, September 2004

Travaglio, C.; Hillebrandt, W.; Reinecke, M.
Astronomy & Astrophysics, Vol. 425, Issue 3
DOI: 10.1051/0004-6361:20041108

Search for TeV Gamma-ray Emission from GRB 100621A, an extremely bright GRB in X-rays, with H.E.S.S.
journal, April 2014

Abramowski, A.; Aharonian, F.; Ait Benkhali, F.
Astronomy & Astrophysics, Vol. 565
DOI: 10.1051/0004-6361/201322984

The clustering of galaxies in the SDSS-III Baryon Oscillation Spectroscopic Survey: baryon acoustic oscillations in the Data Releases 10 and 11 Galaxy samples
journal, April 2014

Anderson, Lauren; Aubourg, Éric; Bailey, Stephen
Monthly Notices of the Royal Astronomical Society, Vol. 441, Issue 1
DOI: 10.1093/mnras/stu523

The Universe at extreme scale: Multi-petaflop sky simulation on the BG/Q
conference, November 2012

Habib, Salman; Morozov, Vitali; Finkel, Hal
2012 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis
DOI: 10.1109/SC.2012.106

Radiative Transfer in a Clumpy Universe. iv. new Synthesis Models of the Cosmic Uv/X-Ray Background
journal, February 2012

Haardt, Francesco; Madau, Piero
The Astrophysical Journal, Vol. 746, Issue 2
DOI: 10.1088/0004-637X/746/2/125

Formation of Galaxies and Clusters of Galaxies by Self-Similar Gravitational Condensation
journal, February 1974

Press, William H.; Schechter, Paul
The Astrophysical Journal, Vol. 187
DOI: 10.1086/152650

Warm dark matter as a solution to the small scale crisis: New constraints from high redshift Lyman- $α$ forest data
journal, August 2013

Viel, Matteo; Becker, George D.; Bolton, James S.
Physical Review D, Vol. 88, Issue 4
DOI: 10.1103/PhysRevD.88.043502

Improved post hoc flow analysis via Lagrangian representations
conference, November 2014

Agranovsky, Alexy; Camp, David; Garth, Christoph
2014 IEEE 4th Symposium on Large Data Analysis and Visualization (LDAV)
DOI: 10.1109/LDAV.2014.7013206

Sussing Merger Trees: The Merger Trees Comparison Project
journal, October 2013

Srisawat, Chaichalit; Knebe, Alexander; Pearce, Frazer R.
Monthly Notices of the Royal Astronomical Society, Vol. 436, Issue 1
DOI: 10.1093/mnras/stt1545

Structure finding in cosmological simulations: the state of affairs
journal, August 2013

Knebe, Alexander; Pearce, Frazer R.; Lux, Hanni
Monthly Notices of the Royal Astronomical Society, Vol. 435, Issue 2
DOI: 10.1093/mnras/stt1403

The Lyman α forest in optically thin hydrodynamical simulations
journal, December 2014

Lukić, Zarija; Stark, Casey W.; Nugent, Peter
Monthly Notices of the Royal Astronomical Society, Vol. 446, Issue 4
DOI: 10.1093/mnras/stu2377

Type Ia supernova diversity in three-dimensional models
journal, June 2006

Röpke, F. K.; Gieseler, M.; Reinecke, M.
Astronomy & Astrophysics, Vol. 453, Issue 1
DOI: 10.1051/0004-6361:20053430

A piecewise parabolic method for cosmological hydrodynamics
journal, August 1995

Bryan, Greg L.; Norman, Michael L.; Stone, James M.
Computer Physics Communications, Vol. 89, Issue 1-3
DOI: 10.1016/0010-4655(94)00191-4

HACC: Simulating sky surveys on state-of-the-art supercomputing architectures
journal, January 2016

Habib, Salman; Pope, Adrian; Finkel, Hal
New Astronomy, Vol. 42
DOI: 10.1016/j.newast.2015.06.003

Visualization and parallel I/O at extreme scale
journal, July 2008

Ross, R. B.; Peterka, T.; Shen, H-W
Journal of Physics: Conference Series, Vol. 125
DOI: 10.1088/1742-6596/125/1/012099

The Santa Barbara Cluster Comparison Project: A Comparison of Cosmological Hydrodynamics Solutions
journal, November 1999

Frenk, C. S.; White, S. D. M.; Bode, P.
The Astrophysical Journal, Vol. 525, Issue 2
DOI: 10.1086/307908

Efficient solution algorithms for the Riemann problem for real gases
journal, June 1985

Colella, Phillip; Glaz, Harland M.
Journal of Computational Physics, Vol. 59, Issue 2
DOI: 10.1016/0021-9991(85)90146-9

Galaxy Formation and Evolution
book, January 2010

Mo, Houjun; van den Bosch, Frank; White, Simon
Cambridge University Press
DOI: 10.1017/CBO9780511807244

Matplotlib: A 2D Graphics Environment
journal, January 2007

Hunter, John D.
Computing in Science & Engineering, Vol. 9, Issue 3
DOI: 10.1109/MCSE.2007.55

Large-scale compute-intensive analysis via a combined in-situ and co-scheduling workflow approach
conference, January 2015

Sewell, Christopher; Messer, Bronson; Habib, Salman
Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '15
DOI: 10.1145/2807591.2807663

Multidimensional upwind methods for hyperbolic conservation laws
journal, March 1990

Colella, Phillip
Journal of Computational Physics, Vol. 87, Issue 1
DOI: 10.1016/0021-9991(90)90233-Q

PHEW: a parallel segmentation algorithm for three-dimensional AMR datasets: Application to structure detection in self-gravitating flows
journal, June 2015

Bleuler, Andreas; Teyssier, Romain; Carassou, Sébastien
Computational Astrophysics and Cosmology, Vol. 2, Issue 1
DOI: 10.1186/s40668-015-0009-7

The Design and Implementation of FFTW3
journal, February 2005

Frigo, M.; Johnson, S. G.
Proceedings of the IEEE, Vol. 93, Issue 2
DOI: 10.1109/JPROC.2004.840301

Large-Scale Simulations of Sky Surveys
journal, September 2014

Heitmann, Katrin; Habib, Salman; Finkel, Hal
Computing in Science & Engineering, Vol. 16, Issue 5
DOI: 10.1109/MCSE.2014.49

Distributed merge trees
conference, January 2013

Morozov, Dmitriy; Weber, Gunther
Proceedings of the 18th ACM SIGPLAN symposium on Principles and practice of parallel programming - PPoPP '13
DOI: 10.1145/2442516.2442526

Roofline: an insightful visual performance model for multicore architectures
journal, April 2009

Williams, Samuel; Waterman, Andrew; Patterson, David
Communications of the ACM, Vol. 52, Issue 4
DOI: 10.1145/1498765.1498785

The one-dimensional Ly α forest power spectrum from BOSS
journal, November 2013

Palanque-Delabrouille, Nathalie; Yèche, Christophe; Borde, Arnaud
Astronomy & Astrophysics, Vol. 559
DOI: 10.1051/0004-6361/201322130

The q Continuum Simulation: Harnessing the Power of gpu Accelerated Supercomputers
journal, August 2015

Heitmann, Katrin; Frontiere, Nicholas; Sewell, Chris
The Astrophysical Journal Supplement Series, Vol. 219, Issue 2
DOI: 10.1088/0067-0049/219/2/34

Castro: a new Compressible Astrophysical Solver. i. Hydrodynamics and Self-Gravity
journal, May 2010

Almgren, A. S.; Beckner, V. E.; Bell, J. B.
The Astrophysical Journal, Vol. 715, Issue 2
DOI: 10.1088/0004-637X/715/2/1221

Nyx: A MASSIVELY PARALLEL AMR CODE FOR COMPUTATIONAL COSMOLOGY
journal, February 2013

Almgren, Ann S.; Bell, John B.; Lijewski, Mike J.
The Astrophysical Journal, Vol. 765, Issue 1
DOI: 10.1088/0004-637X/765/1/39

The evolution of large-scale structure in a universe dominated by cold dark matter
journal, May 1985

Davis, M.; Efstathiou, G.; Frenk, C. S.
The Astrophysical Journal, Vol. 292
DOI: 10.1086/163168

The Structure of Halos: Implications for Group and Cluster Cosmology
journal, February 2009

Lukić, Zarija; Reed, Darren; Habib, Salman
The Astrophysical Journal, Vol. 692, Issue 1
DOI: 10.1088/0004-637X/692/1/217

Similar Records in DOE PAGES and OSTI.GOV collections:

SNIa Cosmology Analysis Results from Simulated LSST Images: From Difference Imaging to Constraints on Dark Energy

Journal Article Sánchez, B. O. ; Kessler, R. ; Scolnic, D. ; ... - The Astrophysical Journal

Abstract The Vera Rubin Observatory Legacy Survey of Space and Time (LSST) is expected to process ∼10 ⁶ transient detections per night. For precision measurements of cosmological parameters and rates, it is critical to understand the detection efficiency, magnitude limits, artifact contamination levels, and biases in the selection and photometry. Here we rigorously test the LSST Difference Image Analysis (DIA) pipeline using simulated images from the Rubin Observatory LSST Dark Energy Science Collaboration Data Challenge (DC2) simulation for the Wide-Fast-Deep survey area. DC2 is the first large-scale (300 deg ² ) image simulation of a transient survey that includes realisticmore »« less
https://doi.org/10.3847/1538-4357/ac7a37
WarpIV: In situ visualization and analysis of ion accelerator simulations

Journal Article Rubel, Oliver ; Loring, Burlen ; Vay, Jean -Luc ; ... - IEEE Computer Graphics and Applications

© 2016 IEEE. The generation of short pulses of ion beams through the interaction of an intense laser with a plasma sheath offers the possibility of compact and cheaper ion sources for many applications - from fast ignition and radiography of dense targets to hadron therapy and injection into conventional accelerators. To enable the efficient analysis of large-scale, high-fidelity particle accelerator simulations using the Warp simulation suite, the authors introduce the Warp In situ Visualization Toolkit (WarpIV). WarpIV integrates state-of-the-art in situ visualization and analysis using VisIt with Warp, supports management and control of complex in situ visualization and analysismore »« less
https://doi.org/10.1109/MCG.2016.62

Full Text Available
In-Situ Feature Extraction of Large Scale Combustion Simulations Using Segmented Merge Trees, In: SC '14: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

Conference Landge, Aaditya G. ; Pascucci, Valerio ; Gyulassy, Attila ; ... - SC14: INTERNATIONAL CONFERENCE FOR HIGH PERFORMANCE COMPUTING, NETWORKING, STORAGE AND ANALYSIS

The ever increasing amount of data generated by scientific simulations coupled with system I/O constraints are fueling a need for in-situ analysis techniques. Of particular interest are approaches that produce reduced data representations while maintaining the ability to redefine, extract, and study features in a post-process to obtain scientific insights. This paper presents two variants of in-situ feature extraction techniques using segmented merge trees, which encode a wide range of threshold based features. The first approach is a fast, low communication cost technique that generates an exact solution but has limited scalability. The second is a scalable, local approximation thatmore »« less
https://doi.org/10.1109/SC.2014.88
Feature Analysis, Tracking, and Data Reduction: An Application to Multiphase Reactor Simulation MFiX-Exa for In-Situ Use Case

Journal Article Biswas, Ayan ; Ahrens, James P. ; Dutta, Soumya ; ... - Computing in Science and Engineering

As we enter the exascale computing regime, powerful supercomputers continue to produce much higher amounts of data than what can be stored for offline data processing. To utilize such high compute capabilities on these machines, much of the data processing needs to happen in situ, when the full high-resolution data is available at the supercomputer memory. In this article, we discuss our MFiX-Exa simulation, which models multiphase flow by tracking a very large number of particles through the simulation domain. In one of the use cases, the carbon particles interact with air to produce carbon dioxide bubbles from the reactor.more »« less
https://doi.org/10.1109/mcse.2020.3016927

Full Text Available
PDACS - A Portal for Data Analysis Services for Cosmological Simulations

Conference Chard, Ryan ; Madduri, Ravi ; Heimann, Katrin ; ...
Full Text Available

Similar Records

Title: In situ and in-transit analysis of cosmological simulations

Citation Formats

Nucleosynthesis in multi-dimensional SN Ia explosions journal, September 2004

Search for TeV Gamma-ray Emission from GRB 100621A, an extremely bright GRB in X-rays, with H.E.S.S. journal, April 2014

The clustering of galaxies in the SDSS-III Baryon Oscillation Spectroscopic Survey: baryon acoustic oscillations in the Data Releases 10 and 11 Galaxy samples journal, April 2014

The Universe at extreme scale: Multi-petaflop sky simulation on the BG/Q conference, November 2012

Radiative Transfer in a Clumpy Universe. iv. new Synthesis Models of the Cosmic Uv/X-Ray Background journal, February 2012

Formation of Galaxies and Clusters of Galaxies by Self-Similar Gravitational Condensation journal, February 1974

Warm dark matter as a solution to the small scale crisis: New constraints from high redshift Lyman- α forest data journal, August 2013

Improved post hoc flow analysis via Lagrangian representations conference, November 2014

Sussing Merger Trees: The Merger Trees Comparison Project journal, October 2013

Structure finding in cosmological simulations: the state of affairs journal, August 2013

The Lyman α forest in optically thin hydrodynamical simulations journal, December 2014

Type Ia supernova diversity in three-dimensional models journal, June 2006

A piecewise parabolic method for cosmological hydrodynamics journal, August 1995

HACC: Simulating sky surveys on state-of-the-art supercomputing architectures journal, January 2016

Visualization and parallel I/O at extreme scale journal, July 2008

The Santa Barbara Cluster Comparison Project: A Comparison of Cosmological Hydrodynamics Solutions journal, November 1999

Efficient solution algorithms for the Riemann problem for real gases journal, June 1985

Galaxy Formation and Evolution book, January 2010

Matplotlib: A 2D Graphics Environment journal, January 2007

Large-scale compute-intensive analysis via a combined in-situ and co-scheduling workflow approach conference, January 2015

Multidimensional upwind methods for hyperbolic conservation laws journal, March 1990

PHEW: a parallel segmentation algorithm for three-dimensional AMR datasets: Application to structure detection in self-gravitating flows journal, June 2015

The Design and Implementation of FFTW3 journal, February 2005

Large-Scale Simulations of Sky Surveys journal, September 2014

Distributed merge trees conference, January 2013

Roofline: an insightful visual performance model for multicore architectures journal, April 2009

The one-dimensional Ly α forest power spectrum from BOSS journal, November 2013

The q Continuum Simulation: Harnessing the Power of gpu Accelerated Supercomputers journal, August 2015

Castro: a new Compressible Astrophysical Solver. i. Hydrodynamics and Self-Gravity journal, May 2010

Nyx: A MASSIVELY PARALLEL AMR CODE FOR COMPUTATIONAL COSMOLOGY journal, February 2013

The evolution of large-scale structure in a universe dominated by cold dark matter journal, May 1985

The Structure of Halos: Implications for Group and Cluster Cosmology journal, February 2009

Nucleosynthesis in multi-dimensional SN Ia explosions
journal, September 2004

Search for TeV Gamma-ray Emission from GRB 100621A, an extremely bright GRB in X-rays, with H.E.S.S.
journal, April 2014

The clustering of galaxies in the SDSS-III Baryon Oscillation Spectroscopic Survey: baryon acoustic oscillations in the Data Releases 10 and 11 Galaxy samples
journal, April 2014

The Universe at extreme scale: Multi-petaflop sky simulation on the BG/Q
conference, November 2012

Radiative Transfer in a Clumpy Universe. iv. new Synthesis Models of the Cosmic Uv/X-Ray Background
journal, February 2012

Formation of Galaxies and Clusters of Galaxies by Self-Similar Gravitational Condensation
journal, February 1974

Warm dark matter as a solution to the small scale crisis: New constraints from high redshift Lyman- $α$ forest data
journal, August 2013

Improved post hoc flow analysis via Lagrangian representations
conference, November 2014

Sussing Merger Trees: The Merger Trees Comparison Project
journal, October 2013

Structure finding in cosmological simulations: the state of affairs
journal, August 2013

The Lyman α forest in optically thin hydrodynamical simulations
journal, December 2014

Type Ia supernova diversity in three-dimensional models
journal, June 2006

A piecewise parabolic method for cosmological hydrodynamics
journal, August 1995

HACC: Simulating sky surveys on state-of-the-art supercomputing architectures
journal, January 2016

Visualization and parallel I/O at extreme scale
journal, July 2008

The Santa Barbara Cluster Comparison Project: A Comparison of Cosmological Hydrodynamics Solutions
journal, November 1999

Efficient solution algorithms for the Riemann problem for real gases
journal, June 1985

Galaxy Formation and Evolution
book, January 2010

Matplotlib: A 2D Graphics Environment
journal, January 2007

Large-scale compute-intensive analysis via a combined in-situ and co-scheduling workflow approach
conference, January 2015

Multidimensional upwind methods for hyperbolic conservation laws
journal, March 1990

PHEW: a parallel segmentation algorithm for three-dimensional AMR datasets: Application to structure detection in self-gravitating flows
journal, June 2015

The Design and Implementation of FFTW3
journal, February 2005

Large-Scale Simulations of Sky Surveys
journal, September 2014

Distributed merge trees
conference, January 2013

Roofline: an insightful visual performance model for multicore architectures
journal, April 2009

The one-dimensional Ly α forest power spectrum from BOSS
journal, November 2013

The q Continuum Simulation: Harnessing the Power of gpu Accelerated Supercomputers
journal, August 2015

Castro: a new Compressible Astrophysical Solver. i. Hydrodynamics and Self-Gravity
journal, May 2010

Nyx: A MASSIVELY PARALLEL AMR CODE FOR COMPUTATIONAL COSMOLOGY
journal, February 2013

The evolution of large-scale structure in a universe dominated by cold dark matter
journal, May 1985

The Structure of Halos: Implications for Group and Cluster Cosmology
journal, February 2009