skip to main content

DOE PAGESDOE PAGES

Title: HACC: Simulating sky surveys on state-of-the-art supercomputing architectures

Current and future surveys of large-scale cosmic structure are associated with a massive and complex datastream to study, characterize, and ultimately understand the physics behind the two major components of the ‘Dark Universe’, dark energy and dark matter. In addition, the surveys also probe primordial perturbations and carry out fundamental measurements, such as determining the sum of neutrino masses. Large-scale simulations of structure formation in the Universe play a critical role in the interpretation of the data and extraction of the physics of interest. Just as survey instruments continue to grow in size and complexity, so do the supercomputers that enable these simulations. In this paper, we report on HACC (Hardware/Hybrid Accelerated Cosmology Code), a recently developed and evolving cosmology N-body code framework, designed to run efficiently on diverse computing architectures and to scale to millions of cores and beyond. HACC can run on all current supercomputer architectures and supports a variety of programming models and algorithms. It has been demonstrated at scale on Cell- and GPU-accelerated systems, standard multi-core node clusters, and Blue Gene systems. HACC’s design allows for ease of portability, and at the same time, high levels of sustained performance on the fastest supercomputers available. We presentmore » a description of the design philosophy of HACC, the underlying algorithms and code structure, and outline implementation details for several specific architectures. We show selected accuracy and performance results from some of the largest high resolution cosmological simulations so far performed, including benchmarks evolving more than 3.6 trillion particles.« less
Authors:
 [1] ;  [2] ;  [3] ;  [4] ;  [1] ; ORCiD logo [5] ;  [5] ;  [6] ;  [7] ;  [8] ;  [9] ;  [10] ;  [11] ;  [11]
  1. Argonne National Lab. (ANL), Argonne, IL (United States). High Energy Physics Division and Mathematics and Computer Science Division; Univ. of Chicago, IL (United States). Kavli Inst. for Cosmological Physics and Computation Inst.
  2. Argonne National Lab. (ANL), Argonne, IL (United States). High Energy Physics Division
  3. Argonne National Lab. (ANL), Argonne, IL (United States). High Energy Physics Division and Argonne Leadership Computing Facility (ALCF)
  4. Argonne National Lab. (ANL), Argonne, IL (United States). High Energy Physics Division; Univ. of Chicago, IL (United States). Dept. of Physics
  5. Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
  6. Argonne National Lab. (ANL), Argonne, IL (United States). Argonne Leadership Computing Facility (ALCF)
  7. Kitware, Inc., Clifton Park, NY (United States)
  8. Argonne National Lab. (ANL), Argonne, IL (United States). Mathematics and Computer Science Division; Univ. of Chicago, IL (United States). Computation Inst.
  9. Univ. of Chicago, IL (United States). Computation Inst.; Argonne National Lab. (ANL), Argonne, IL (United States). Argonne Leadership Computing Facility (ALCF)
  10. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). Computational Research Division
  11. Northwestern Univ., Evanston, IL (United States). Dept. of Electrical Engineering and Computer Science
Publication Date:
Report Number(s):
LA-UR-17-27368
Journal ID: ISSN 1384-1076
Grant/Contract Number:
AC52-06NA25396; AC02-06CH11357; W-7405-ENG-36; AC05-00OR22725
Type:
Accepted Manuscript
Journal Name:
New Astronomy
Additional Journal Information:
Journal Volume: 42; Journal Issue: C; Journal ID: ISSN 1384-1076
Research Org:
Los Alamos National Lab. (LANL), Los Alamos, NM (United States); Argonne National Lab. (ANL), Argonne, IL (United States); Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Sponsoring Org:
USDOE Laboratory Directed Research and Development (LDRD) Program; USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21). Scientific Discovery through Advanced Computing (SciDAC); USDOE Office of Science (SC), High Energy Physics (HEP) (SC-25)
Country of Publication:
United States
Language:
English
Subject:
79 ASTRONOMY AND ASTROPHYSICS; 97 MATHEMATICS AND COMPUTING; Cosmology–large scale structure of the Universe; N-body simulations
OSTI Identifier:
1473805
Alternate Identifier(s):
OSTI ID: 1359616

Habib, Salman, Pope, Adrian, Finkel, Hal, Frontiere, Nicholas, Heitmann, Katrin, Daniel, David, Fasel, Patricia, Morozov, Vitali, Zagaris, George, Peterka, Tom, Vishwanath, Venkatram, Lukić, Zarija, Sehrish, Saba, and Liao, Wei-keng. HACC: Simulating sky surveys on state-of-the-art supercomputing architectures. United States: N. p., Web. doi:10.1016/j.newast.2015.06.003.
Habib, Salman, Pope, Adrian, Finkel, Hal, Frontiere, Nicholas, Heitmann, Katrin, Daniel, David, Fasel, Patricia, Morozov, Vitali, Zagaris, George, Peterka, Tom, Vishwanath, Venkatram, Lukić, Zarija, Sehrish, Saba, & Liao, Wei-keng. HACC: Simulating sky surveys on state-of-the-art supercomputing architectures. United States. doi:10.1016/j.newast.2015.06.003.
Habib, Salman, Pope, Adrian, Finkel, Hal, Frontiere, Nicholas, Heitmann, Katrin, Daniel, David, Fasel, Patricia, Morozov, Vitali, Zagaris, George, Peterka, Tom, Vishwanath, Venkatram, Lukić, Zarija, Sehrish, Saba, and Liao, Wei-keng. 2015. "HACC: Simulating sky surveys on state-of-the-art supercomputing architectures". United States. doi:10.1016/j.newast.2015.06.003. https://www.osti.gov/servlets/purl/1473805.
@article{osti_1473805,
title = {HACC: Simulating sky surveys on state-of-the-art supercomputing architectures},
author = {Habib, Salman and Pope, Adrian and Finkel, Hal and Frontiere, Nicholas and Heitmann, Katrin and Daniel, David and Fasel, Patricia and Morozov, Vitali and Zagaris, George and Peterka, Tom and Vishwanath, Venkatram and Lukić, Zarija and Sehrish, Saba and Liao, Wei-keng},
abstractNote = {Current and future surveys of large-scale cosmic structure are associated with a massive and complex datastream to study, characterize, and ultimately understand the physics behind the two major components of the ‘Dark Universe’, dark energy and dark matter. In addition, the surveys also probe primordial perturbations and carry out fundamental measurements, such as determining the sum of neutrino masses. Large-scale simulations of structure formation in the Universe play a critical role in the interpretation of the data and extraction of the physics of interest. Just as survey instruments continue to grow in size and complexity, so do the supercomputers that enable these simulations. In this paper, we report on HACC (Hardware/Hybrid Accelerated Cosmology Code), a recently developed and evolving cosmology N-body code framework, designed to run efficiently on diverse computing architectures and to scale to millions of cores and beyond. HACC can run on all current supercomputer architectures and supports a variety of programming models and algorithms. It has been demonstrated at scale on Cell- and GPU-accelerated systems, standard multi-core node clusters, and Blue Gene systems. HACC’s design allows for ease of portability, and at the same time, high levels of sustained performance on the fastest supercomputers available. We present a description of the design philosophy of HACC, the underlying algorithms and code structure, and outline implementation details for several specific architectures. We show selected accuracy and performance results from some of the largest high resolution cosmological simulations so far performed, including benchmarks evolving more than 3.6 trillion particles.},
doi = {10.1016/j.newast.2015.06.003},
journal = {New Astronomy},
number = C,
volume = 42,
place = {United States},
year = {2015},
month = {7}
}