Title: HACC: Simulating sky surveys on state-of-the-art supercomputing architectures

Abstract

Current and future surveys of large-scale cosmic structure are associated with a massive and complex datastream to study, characterize, and ultimately understand the physics behind the two major components of the ‘Dark Universe’, dark energy and dark matter. In addition, the surveys also probe primordial perturbations and carry out fundamental measurements, such as determining the sum of neutrino masses. Large-scale simulations of structure formation in the Universe play a critical role in the interpretation of the data and extraction of the physics of interest. Just as survey instruments continue to grow in size and complexity, so do the supercomputers that enable these simulations. In this paper, we report on HACC (Hardware/Hybrid Accelerated Cosmology Code), a recently developed and evolving cosmology N-body code framework, designed to run efficiently on diverse computing architectures and to scale to millions of cores and beyond. HACC can run on all current supercomputer architectures and supports a variety of programming models and algorithms. It has been demonstrated at scale on Cell- and GPU-accelerated systems, standard multi-core node clusters, and Blue Gene systems. HACC’s design allows for ease of portability, and at the same time, high levels of sustained performance on the fastest supercomputers available. We presentmore » a description of the design philosophy of HACC, the underlying algorithms and code structure, and outline implementation details for several specific architectures. We show selected accuracy and performance results from some of the largest high resolution cosmological simulations so far performed, including benchmarks evolving more than 3.6 trillion particles.« less

Authors:

Habib, Salman ^[1]; Pope, Adrian ^[2]; Finkel, Hal ^[3]; Frontiere, Nicholas ^[4]; Heitmann, Katrin ^[1]; Daniel, David  ^[5];

• Search DOE PAGES for author "Daniel, David"
• Search DOE PAGES for ORCID "0000-0002-4873-6856"

Fasel, Patricia ^[5]; Morozov, Vitali ^[6]; Zagaris, George ^[7]; Peterka, Tom ^[8]; Vishwanath, Venkatram ^[9]; Lukić, Zarija ^[10]; Sehrish, Saba ^[11]; Liao, Wei-keng ^[11]

---

+ Show Author Affiliations

1. Argonne National Lab. (ANL), Argonne, IL (United States). High Energy Physics Division and Mathematics and Computer Science Division; Univ. of Chicago, IL (United States). Kavli Inst. for Cosmological Physics and Computation Inst.
2. Argonne National Lab. (ANL), Argonne, IL (United States). High Energy Physics Division
3. Argonne National Lab. (ANL), Argonne, IL (United States). High Energy Physics Division and Argonne Leadership Computing Facility (ALCF)
4. Argonne National Lab. (ANL), Argonne, IL (United States). High Energy Physics Division; Univ. of Chicago, IL (United States). Dept. of Physics
5. Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
6. Argonne National Lab. (ANL), Argonne, IL (United States). Argonne Leadership Computing Facility (ALCF)
7. Kitware, Inc., Clifton Park, NY (United States)
8. Argonne National Lab. (ANL), Argonne, IL (United States). Mathematics and Computer Science Division; Univ. of Chicago, IL (United States). Computation Inst.
9. Univ. of Chicago, IL (United States). Computation Inst.; Argonne National Lab. (ANL), Argonne, IL (United States). Argonne Leadership Computing Facility (ALCF)
10. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). Computational Research Division
11. Northwestern Univ., Evanston, IL (United States). Dept. of Electrical Engineering and Computer Science

Publication Date:

Sat Jul 11 00:00:00 EDT 2015

Research Org.:

Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF); Los Alamos National Laboratory (LANL), Los Alamos, NM (United States); Argonne National Laboratory (ANL), Argonne, IL (United States)

Sponsoring Org.:

USDOE Laboratory Directed Research and Development (LDRD) Program; USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR). Scientific Discovery through Advanced Computing (SciDAC); USDOE Office of Science (SC), High Energy Physics (HEP)

OSTI Identifier:

1473805

Alternate Identifier(s):

OSTI ID: 1359616

Report Number(s):

LA-UR-17-27368
Journal ID: ISSN 1384-1076

Grant/Contract Number:  

AC52-06NA25396; AC02-06CH11357; W-7405-ENG-36; AC05-00OR22725

Resource Type:

Accepted Manuscript

Journal Name:

New Astronomy

Additional Journal Information:

Journal Volume: 42; Journal Issue: C; Journal ID: ISSN 1384-1076

Country of Publication:

United States

Language:

English

Subject:

79 ASTRONOMY AND ASTROPHYSICS; 97 MATHEMATICS AND COMPUTING; Cosmology–large scale structure of the Universe; N-body simulations