Cataloging the visible universe through Bayesian inference in Julia at petascale

Journal Article · Wed May 01 00:00:00 EDT 2019 · Journal of Parallel and Distributed Computing

DOI:https://doi.org/10.1016/j.jpdc.2018.12.008· OSTI ID:1527362

Regier, Jeffrey  ^[1]; Fischer, Keno  ^[2]; Pamnany, Kiran ^[3]; Noack, Andreas  ^[4]; Revels, Jarrett ^[4]; Lam, Maximilian ^[1]; Howard, Steve  ^[5]; Giordano, Ryan ^[5]; Schlegel, David ^[6]; McAuliffe, Jon  ^[5]; Thomas, Rollin ^[6]; Prabhat, N/A  ^[6]

---

1. Univ. of California, Berkeley, CA (United States). Dept. of Electrical Engineering and Computer Sciences
2. Julia Computing (United States)
3. Intel Corporation (United States). Parallel Computing Lab.
4. Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States). Computer Science and AI Laboratories
5. Univ. of California, Berkeley, CA (United States). Dept. of Statistics
6. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)

A key task in astronomy is to locate astronomical objects in images and to characterize them according to physical parameters such as brightness, color, and morphology. This task, known as cataloging, is challenging for several reasons: many astronomical objects are much dimmer than the sky background, labeled data is generally unavailable, overlapping astronomical objects must be resolved collectively, and the datasets are enormous – terabytes now, petabytes soon. In this work, we infer an astronomical catalog from 55 TB of imaging data using Celeste, a Bayesian variational inference code written entirely in the high-productivity programming language Julia. Using over 1.3 million threads on 650,000 Intel Xeon Phi cores of the Cori Phase II supercomputer, Celeste achieves a peak rate of 1.54 DP PFLOP/s. Celeste is able to jointly optimize parameters for 188 M stars and galaxies, loading and processing 178 TB across 8192 nodes in 14.6 min. To achieve this, Celeste exploits parallelism at multiple levels (cluster, node, and thread) and accelerates I/O through Cori’s burst buffer. Julia’s native performance enables Celeste to employ high-level constructs without resorting to hand-written or generated low-level code (C/C++/Fortran) while still achieving petascale performance.

• Research Organization: Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). National Energy Research Scientific Computing Center (NERSC)
• Sponsoring Organization: USDOE Office of Science (SC)
• DOE Contract Number: AC02-05CH11231
• OSTI ID: 1527362
• Journal Information: Journal of Parallel and Distributed Computing, Vol. 127, Issue C; ISSN 0743-7315
• Country of Publication: United States
• Language: English