Cataloging the visible universe through Bayesian inference in Julia at petascale

Regier, Jeffrey; Fischer, Keno; Pamnany, Kiran; Noack, Andreas; Revels, Jarrett; Lam, Maximilian; Howard, Steve; Giordano, Ryan; Schlegel, David; McAuliffe, Jon; Thomas, Rollin; Prabhat, Prabhat

doi:10.1016/j.jpdc.2018.12.008

Cataloging the visible universe through Bayesian inference in Julia at petascale

Journal Article · Mon Jan 21 00:00:00 EST 2019 · Journal of Parallel and Distributed Computing

DOI:https://doi.org/10.1016/j.jpdc.2018.12.008· OSTI ID:1656511

^[1]; ^[2]; Pamnany, Kiran ^[3]; ^[4]; Revels, Jarrett ^[4]; Lam, Maximilian ^[1]; ^[1]; Giordano, Ryan ^[1]; Schlegel, David ^[5]; ^[1]; Thomas, Rollin ^[5]; ^[5]

Univ. of California, Berkeley, CA (United States)
Julia Computing, Boston, MA (United States)
Intel Corp. (United States). Parallel Computing Lab.
Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States). Computer Science and AI Labs.
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)

A key task in astronomy is to locate astronomical objects in images and to characterize them according to physical parameters such as brightness, color, and morphology. This task, known as cataloging, is challenging for several reasons: many astronomical objects are much dimmer than the sky background, labeled data is generally unavailable, overlapping astronomical objects must be resolved collectively, and the datasets are enormous – terabytes now, petabytes soon. In this work, we infer an astronomical catalog from 55 TB of imaging data using Celeste, a Bayesian variational inference code written entirely in the high-productivity programming language Julia. Additionally, using over 1.3 million threads on 650,000 Intel Xeon Phi cores of the Cori Phase II supercomputer, Celeste achieves a peak rate of 1.54 DP PFLOP/s. Furthermore, Celeste is able to jointly optimize parameters for 188 M stars and galaxies, loading and processing 178 TB across 8192 nodes in 14.6 min. To achieve this, Celeste exploits parallelism at multiple levels (cluster, node, and thread) and accelerates I/O through Cori's burst buffer. Julia's native performance enables Celeste to employ high-level constructs without resorting to hand-written or generated low-level code (C/C++/Fortran) while still achieving petascale performance.

Research Organization:: Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)

Sponsoring Organization:: National Science Foundation (NSF); USDOE; USDOE Office of Science (SC)

Grant/Contract Number:: AC02-05CH11231

OSTI ID:: 1656511

Alternate ID(s):: OSTI ID: 1527362
OSTI ID: 1547840

Journal Information:: Journal of Parallel and Distributed Computing, Journal Name: Journal of Parallel and Distributed Computing Vol. 127; ISSN 0743-7315

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

References (15)

SExtractor: Software for source extraction Bertin, E.; Arnouts, S. Astronomy and Astrophysics Supplement Series, Vol. 117, Issue 2 https://doi.org/10.1051/aas:1996164	journal	June 1996
Variational Inference: A Review for Statisticians Blei, David M.; Kucukelbir, Alp; McAuliffe, Jon D. Journal of the American Statistical Association, Vol. 112, Issue 518 https://doi.org/10.1080/01621459.2017.1285773	journal	July 2016
Probabilistic Catalogs for Crowded Stellar Fields Brewer, Brendon J.; Foreman-Mackey, Daniel; Hogg, David W. The Astronomical Journal, Vol. 146, Issue 1 https://doi.org/10.1088/0004-6256/146/1/7	journal	June 2013
The Eleventh and Twelfth data Releases of the Sloan Digital sky Survey: Final data from Sdss-Iii Alam, Shadab; Albareti, Franco D.; Prieto, Carlos Allende The Astrophysical Journal Supplement Series, Vol. 219, Issue 1 https://doi.org/10.1088/0067-0049/219/1/12	journal	July 2015
Scripting: higher level programming for the 21st Century Ousterhout, J. K. Computer, Vol. 31, Issue 3 https://doi.org/10.1109/2.660187	journal	March 1998
Seeing the Earth in the Cloud: Processing one petabyte of satellite imagery in one day Warren, Michael S.; Brumby, Steven P.; Skillman, Samuel W. 2015 IEEE Applied Imagery Pattern Recognition Workshop (AIPR) https://doi.org/10.1109/AIPR.2015.7444536	conference	October 2015
Cataloging the Visible Universe Through Bayesian Inference at Petascale Regier, Jeffrey; McAuliffe, Jon; Thomas, Rollin 2018 IEEE International Parallel and Distributed Processing Symposium (IPDPS) https://doi.org/10.1109/IPDPS.2018.00015	conference	May 2018
Cython: The Best of Both Worlds Behnel, Stefan; Bradshaw, Robert; Citro, Craig Computing in Science & Engineering, Vol. 13, Issue 2 https://doi.org/10.1109/MCSE.2010.118	journal	March 2011
The NumPy Array: A Structure for Efficient Numerical Computation van der Walt, Stéfan; Colbert, S. Chris; Varoquaux, Gaël Computing in Science & Engineering, Vol. 13, Issue 2 https://doi.org/10.1109/MCSE.2011.37	journal	March 2011
Towards Green Aviation with Python at Petascale Vincent, Peter; Witherden, Freddie; Vermeire, Brian SC16: International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1109/SC.2016.1	conference	November 2016
A New Boson with a Mass of 125 GeV Observed with the CMS Experiment at the Large Hadron Collider Abbaneo, D.; Abbiendi, G.; Abbrescia, M. Science, Vol. 338, Issue 6114 https://doi.org/10.1126/science.1230816	journal	December 2012
Julia: A Fresh Approach to Numerical Computing Bezanson, Jeff; Edelman, Alan; Karpinski, Stefan SIAM Review, Vol. 59, Issue 1 https://doi.org/10.1137/141000671	journal	January 2017
Remote Memory Access Programming in MPI-3 Hoefler, Torsten; Dinan, James; Thakur, Rajeev ACM Transactions on Parallel Computing, Vol. 2, Issue 2 https://doi.org/10.1145/2780584	journal	July 2015
Advances, Applications and Performance of the Global Arrays Shared Memory Programming Toolkit Nieplocha, Jarek; Palmer, Bruce; Tipparaju, Vinod The International Journal of High Performance Computing Applications, Vol. 20, Issue 2 https://doi.org/10.1177/1094342006064503	journal	May 2006
Improved Point-source Detection in Crowded Fields Using Probabilistic Cataloging Portillo, Stephen K. N.; Lee, Benjamin C. G.; Daylan, Tansu The Astronomical Journal, Vol. 154, Issue 4 https://doi.org/10.3847/1538-3881/aa8565	journal	September 2017

Cited By (1)

Analyzing, Exploring, and Visualizing Complex Networks via Hypergraphs using SimpleHypergraphs.jl Antelmi, Alessia; Cordasco, Gennaro; Kamiński, Bogumił arXiv https://doi.org/10.48550/arxiv.2002.04654	text	January 2020

Similar Records

Cataloging the visible universe through Bayesian inference in Julia at petascale

Journal Article · Wed May 01 00:00:00 EDT 2019 · Journal of Parallel and Distributed Computing · OSTI ID:1527362

CosmoFlow: Using Deep Learning to Learn the Universe at Scale

Journal Article · Wed Mar 13 20:00:00 EDT 2019 · International Conference for High Performance Computing, Networking, Storage and Analysis · OSTI ID:1510756

Computational Chemistry at the Petascale: Are We There Yet?

Journal Article · Sat Jan 31 23:00:00 EST 2009 · Journal of Physics. Conference Series · OSTI ID:982191

Related Subjects

79 ASTRONOMY AND ASTROPHYSICS
97 MATHEMATICS AND COMPUTING
Astronomy
Bayesian
Julia
distributed optimization
high-performance computing
variational interference

Cataloging the visible universe through Bayesian inference in Julia at petascale

Citation Formats

References (15)

Cited By (1)

Similar Records

Related Subjects