Reconstructing probability distributions with Gaussian processes
Abstract
Modern cosmological analyses constrain physical parameters using Markov Chain Monte Carlo (MCMC) or similar sampling techniques. Oftentimes, these techniques are computationally expensive to run and require up to thousands of CPU hours to complete. Here we present a method for reconstructing the log-probability distributions of completed experiments from an existing chain (or any set of posterior samples). Here, the reconstruction is performed using Gaussian process regression for interpolating the log-probability. This allows for easy resampling, importance sampling, marginalization, testing different samplers, investigating chain convergence, and other operations. As an example use case, we reconstruct the posterior distribution of the most recent Planck 2018 analysis. We then resample the posterior, and generate a new chain with 40 times as many points in only 30 min. Our likelihood reconstruction tool is made publicly available online.
- Authors:
-
[1];
- Brookhaven National Lab. (BNL), Upton, NY (United States). Physics Dept.
- Univ. of Arizona, Tuscon, AZ (United States). Dept. of Physics
- Publication Date:
- Research Org.:
- Brookhaven National Lab. (BNL), Upton, NY (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC), High Energy Physics (HEP)
- OSTI Identifier:
- 1573465
- Report Number(s):
- BNL-212314-2019-JAAM
Journal ID: ISSN 0035-8711; TRN: US2001369
- Grant/Contract Number:
- SC0012704
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Monthly Notices of the Royal Astronomical Society
- Additional Journal Information:
- Journal Volume: 489; Journal Issue: 3; Journal ID: ISSN 0035-8711
- Publisher:
- Royal Astronomical Society
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS; methods; data analysis; statistical
Citation Formats
McClintock, Thomas, and Rozo, Eduardo. Reconstructing probability distributions with Gaussian processes. United States: N. p., 2019.
Web. doi:10.1093/mnras/stz2426.
McClintock, Thomas, & Rozo, Eduardo. Reconstructing probability distributions with Gaussian processes. United States. https://doi.org/10.1093/mnras/stz2426
McClintock, Thomas, and Rozo, Eduardo. Mon .
"Reconstructing probability distributions with Gaussian processes". United States. https://doi.org/10.1093/mnras/stz2426. https://www.osti.gov/servlets/purl/1573465.
@article{osti_1573465,
title = {Reconstructing probability distributions with Gaussian processes},
author = {McClintock, Thomas and Rozo, Eduardo},
abstractNote = {Modern cosmological analyses constrain physical parameters using Markov Chain Monte Carlo (MCMC) or similar sampling techniques. Oftentimes, these techniques are computationally expensive to run and require up to thousands of CPU hours to complete. Here we present a method for reconstructing the log-probability distributions of completed experiments from an existing chain (or any set of posterior samples). Here, the reconstruction is performed using Gaussian process regression for interpolating the log-probability. This allows for easy resampling, importance sampling, marginalization, testing different samplers, investigating chain convergence, and other operations. As an example use case, we reconstruct the posterior distribution of the most recent Planck 2018 analysis. We then resample the posterior, and generate a new chain with 40 times as many points in only 30 min. Our likelihood reconstruction tool is made publicly available online.},
doi = {10.1093/mnras/stz2426},
journal = {Monthly Notices of the Royal Astronomical Society},
number = 3,
volume = 489,
place = {United States},
year = {2019},
month = {9}
}
Free Publicly Available Full Text
Publisher's Version of Record
Other availability
Search WorldCat to find libraries that may hold this journal
Cited by: 2 works
Citation information provided by
Web of Science
Web of Science
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
ChainConsumer
journal, August 2016
- Hinton, Samuel
- The Journal of Open Source Software, Vol. 1, Issue 4
Dark Energy Survey Year 1 Results: A Precise H0 Estimate from DES Y1, BAO, and D/H Data
journal, July 2018
- Abbott, T. M. C.; Abdalla, F. B.; Annis, J.
- Monthly Notices of the Royal Astronomical Society, Vol. 480, Issue 3
emcee : The MCMC Hammer
journal, March 2013
- Foreman-Mackey, Daniel; Hogg, David W.; Lang, Dustin
- Publications of the Astronomical Society of the Pacific, Vol. 125, Issue 925
The Mira-Titan Universe. II. Matter Power Spectrum Emulation
journal, September 2017
- Lawrence, Earl; Heitmann, Katrin; Kwan, Juliana
- The Astrophysical Journal, Vol. 847, Issue 1
Hunting high and low: disentangling primordial and late-time non-Gaussianity with cosmic densities in spheres
journal, October 2017
- Uhlemann, C.; Pajer, E.; Pichon, C.
- Monthly Notices of the Royal Astronomical Society, Vol. 474, Issue 3
The clustering of galaxies in the completed SDSS-III Baryon Oscillation Spectroscopic Survey: cosmological analysis of the DR12 galaxy sample
journal, March 2017
- Alam, Shadab; Ata, Metin; Bailey, Stephen
- Monthly Notices of the Royal Astronomical Society, Vol. 470, Issue 3
Fast Direct Methods for Gaussian Processes
journal, February 2016
- Ambikasaran, Sivaram; Foreman-Mackey, Daniel; Greengard, Leslie
- IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 38, Issue 2
Conservative constraints on early cosmology with M ONTE P YTHON
journal, February 2013
- Audren, Benjamin; Lesgourgues, Julien; Benabed, Karim
- Journal of Cosmology and Astroparticle Physics, Vol. 2013, Issue 02
CosmoSIS: Modular cosmological parameter estimation
journal, September 2015
- Zuntz, J.; Paterno, M.; Jennings, E.
- Astronomy and Computing, Vol. 12
Gaussian Processes for Machine Learning
book, January 2005
- Rasmussen, Carl Edward; Williams, Christopher K. I.
- The MIT Press
The Coyote Universe. ii. Cosmological Models and Precision Emulation of the Nonlinear Matter Power Spectrum
journal, October 2009
- Heitmann, Katrin; Higdon, David; White, Martin
- The Astrophysical Journal, Vol. 705, Issue 1
Dark Energy Survey Year 1 results: Methodology and projections for joint analysis of galaxy clustering, galaxy lensing, and CMB lensing two-point functions
journal, January 2019
- Baxter, E. J.; Omori, Y.; Chang, C.
- Physical Review D, Vol. 99, Issue 2
Matplotlib: A 2D Graphics Environment
journal, January 2007
- Hunter, John D.
- Computing in Science & Engineering, Vol. 9, Issue 3
Cosmological parameters from CMB and other data: A Monte Carlo approach
journal, November 2002
- Lewis, Antony; Bridle, Sarah
- Physical Review D, Vol. 66, Issue 10
Observational Evidence from Supernovae for an Accelerating Universe and a Cosmological Constant
journal, September 1998
- Riess, Adam G.; Filippenko, Alexei V.; Challis, Peter
- The Astronomical Journal, Vol. 116, Issue 3
The 6dF Galaxy Survey: baryon acoustic oscillations and the local Hubble constant: 6dFGS: BAOs and the local Hubble constant
journal, July 2011
- Beutler, Florian; Blake, Chris; Colless, Matthew
- Monthly Notices of the Royal Astronomical Society, Vol. 416, Issue 4
A Micromechanical Method for the Analysis of Three-Dimensional Smart Composites
journal, March 2018
- Ye, J. J.; Chu, Ch. Ch.; Wang, Y. K.
- Mechanics of Composite Materials, Vol. 54, Issue 1
Nuisance hardened data compression for fast likelihood-free inference
journal, July 2019
- Alsing, Justin; Wandelt, Benjamin
- Monthly Notices of the Royal Astronomical Society, Vol. 488, Issue 4
Measurements of Ω and Λ from 42 High‐Redshift Supernovae
journal, June 1999
- Perlmutter, S.; Aldering, G.; Goldhaber, G.
- The Astrophysical Journal, Vol. 517, Issue 2
The NumPy Array: A Structure for Efficient Numerical Computation
journal, March 2011
- van der Walt, Stéfan; Colbert, S. Chris; Varoquaux, Gaël
- Computing in Science & Engineering, Vol. 13, Issue 2
The Aemulus Project. II. Emulating the Halo Mass Function
journal, February 2019
- McClintock, Thomas; Rozo, Eduardo; Becker, Matthew R.
- The Astrophysical Journal, Vol. 872, Issue 1
Cosmology from cosmic shear power spectra with Subaru Hyper Suprime-Cam first-year data
journal, March 2019
- Hikage, Chiaki; Oguri, Masamune; Hamana, Takashi
- Publications of the Astronomical Society of Japan, Vol. 71, Issue 2
Generalized Subset Designs in Analytical Chemistry
journal, May 2017
- Surowiec, Izabella; Vikström, Ludvig; Hector, Gustaf
- Analytical Chemistry, Vol. 89, Issue 12
CASSPER is a semantic segmentation-based particle picking algorithm for single-particle cryo-electron microscopy
journal, February 2021
- George, Blesson; Assaiya, Anshul; Roy, Robin J.
- Communications Biology, Vol. 4, Issue 1