skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

This content will become publicly available on September 2, 2020

Title: Reconstructing probability distributions with Gaussian processes

Abstract

Modern cosmological analyses constrain physical parameters using Markov Chain Monte Carlo (MCMC) or similar sampling techniques. Oftentimes, these techniques are computationally expensive to run and require up to thousands of CPU hours to complete. Here we present a method for reconstructing the log-probability distributions of completed experiments from an existing chain (or any set of posterior samples). Here, the reconstruction is performed using Gaussian process regression for interpolating the log-probability. This allows for easy resampling, importance sampling, marginalization, testing different samplers, investigating chain convergence, and other operations. As an example use case, we reconstruct the posterior distribution of the most recent Planck 2018 analysis. We then resample the posterior, and generate a new chain with 40 times as many points in only 30 min. Our likelihood reconstruction tool is made publicly available online.

Authors:
ORCiD logo [1];  [2]
  1. Brookhaven National Lab. (BNL), Upton, NY (United States). Physics Dept.
  2. Univ. of Arizona, Tuscon, AZ (United States). Dept. of Physics
Publication Date:
Research Org.:
Brookhaven National Lab. (BNL), Upton, NY (United States)
Sponsoring Org.:
USDOE Office of Science (SC), High Energy Physics (HEP) (SC-25)
OSTI Identifier:
1573465
Report Number(s):
BNL-212314-2019-JAAM
Journal ID: ISSN 0035-8711
Grant/Contract Number:  
SC0012704
Resource Type:
Accepted Manuscript
Journal Name:
Monthly Notices of the Royal Astronomical Society
Additional Journal Information:
Journal Volume: 489; Journal Issue: 3; Journal ID: ISSN 0035-8711
Publisher:
Royal Astronomical Society
Country of Publication:
United States
Language:
English
Subject:
72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS; methods; data analysis; statistical

Citation Formats

McClintock, Thomas, and Rozo, Eduardo. Reconstructing probability distributions with Gaussian processes. United States: N. p., 2019. Web. doi:10.1093/mnras/stz2426.
McClintock, Thomas, & Rozo, Eduardo. Reconstructing probability distributions with Gaussian processes. United States. doi:10.1093/mnras/stz2426.
McClintock, Thomas, and Rozo, Eduardo. Mon . "Reconstructing probability distributions with Gaussian processes". United States. doi:10.1093/mnras/stz2426.
@article{osti_1573465,
title = {Reconstructing probability distributions with Gaussian processes},
author = {McClintock, Thomas and Rozo, Eduardo},
abstractNote = {Modern cosmological analyses constrain physical parameters using Markov Chain Monte Carlo (MCMC) or similar sampling techniques. Oftentimes, these techniques are computationally expensive to run and require up to thousands of CPU hours to complete. Here we present a method for reconstructing the log-probability distributions of completed experiments from an existing chain (or any set of posterior samples). Here, the reconstruction is performed using Gaussian process regression for interpolating the log-probability. This allows for easy resampling, importance sampling, marginalization, testing different samplers, investigating chain convergence, and other operations. As an example use case, we reconstruct the posterior distribution of the most recent Planck 2018 analysis. We then resample the posterior, and generate a new chain with 40 times as many points in only 30 min. Our likelihood reconstruction tool is made publicly available online.},
doi = {10.1093/mnras/stz2426},
journal = {Monthly Notices of the Royal Astronomical Society},
number = 3,
volume = 489,
place = {United States},
year = {2019},
month = {9}
}

Journal Article:
Free Publicly Available Full Text
This content will become publicly available on September 2, 2020
Publisher's Version of Record

Save / Share:

Works referenced in this record:

Dark Energy Survey Year 1 Results: A Precise H0 Estimate from DES Y1, BAO, and D/H Data
journal, July 2018

  • Abbott, T. M. C.; Abdalla, F. B.; Annis, J.
  • Monthly Notices of the Royal Astronomical Society, Vol. 480, Issue 3
  • DOI: 10.1093/mnras/sty1939

The clustering of galaxies in the completed SDSS-III Baryon Oscillation Spectroscopic Survey: cosmological analysis of the DR12 galaxy sample
journal, March 2017

  • Alam, Shadab; Ata, Metin; Bailey, Stephen
  • Monthly Notices of the Royal Astronomical Society, Vol. 470, Issue 3
  • DOI: 10.1093/mnras/stx721

Nuisance hardened data compression for fast likelihood-free inference
journal, July 2019

  • Alsing, Justin; Wandelt, Benjamin
  • Monthly Notices of the Royal Astronomical Society, Vol. 488, Issue 4
  • DOI: 10.1093/mnras/stz1900

Fast Direct Methods for Gaussian Processes
journal, February 2016

  • Ambikasaran, Sivaram; Foreman-Mackey, Daniel; Greengard, Leslie
  • IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 38, Issue 2
  • DOI: 10.1109/TPAMI.2015.2448083

Conservative constraints on early cosmology with M ONTE P YTHON
journal, February 2013

  • Audren, Benjamin; Lesgourgues, Julien; Benabed, Karim
  • Journal of Cosmology and Astroparticle Physics, Vol. 2013, Issue 02
  • DOI: 10.1088/1475-7516/2013/02/001

The 6dF Galaxy Survey: baryon acoustic oscillations and the local Hubble constant: 6dFGS: BAOs and the local Hubble constant
journal, July 2011


emcee : The MCMC Hammer
journal, March 2013

  • Foreman-Mackey, Daniel; Hogg, David W.; Lang, Dustin
  • Publications of the Astronomical Society of the Pacific, Vol. 125, Issue 925
  • DOI: 10.1086/670067

The Coyote Universe. ii. Cosmological Models and Precision Emulation of the Nonlinear Matter Power Spectrum
journal, October 2009


Cosmology from cosmic shear power spectra with Subaru Hyper Suprime-Cam first-year data
journal, March 2019

  • Hikage, Chiaki; Oguri, Masamune; Hamana, Takashi
  • Publications of the Astronomical Society of Japan, Vol. 71, Issue 2
  • DOI: 10.1093/pasj/psz010

ChainConsumer
journal, August 2016


Matplotlib: A 2D Graphics Environment
journal, January 2007


The Mira-Titan Universe. II. Matter Power Spectrum Emulation
journal, September 2017

  • Lawrence, Earl; Heitmann, Katrin; Kwan, Juliana
  • The Astrophysical Journal, Vol. 847, Issue 1
  • DOI: 10.3847/1538-4357/aa86a9

Cosmological parameters from CMB and other data: A Monte Carlo approach
journal, November 2002


The Aemulus Project. II. Emulating the Halo Mass Function
journal, February 2019

  • McClintock, Thomas; Rozo, Eduardo; Becker, Matthew R.
  • The Astrophysical Journal, Vol. 872, Issue 1
  • DOI: 10.3847/1538-4357/aaf568

Hunting high and low: disentangling primordial and late-time non-Gaussianity with cosmic densities in spheres
journal, October 2017

  • Uhlemann, C.; Pajer, E.; Pichon, C.
  • Monthly Notices of the Royal Astronomical Society, Vol. 474, Issue 3
  • DOI: 10.1093/mnras/stx2623

Measurements of Ω and Λ from 42 High‐Redshift Supernovae
journal, June 1999

  • Perlmutter, S.; Aldering, G.; Goldhaber, G.
  • The Astrophysical Journal, Vol. 517, Issue 2
  • DOI: 10.1086/307221

Observational Evidence from Supernovae for an Accelerating Universe and a Cosmological Constant
journal, September 1998

  • Riess, Adam G.; Filippenko, Alexei V.; Challis, Peter
  • The Astronomical Journal, Vol. 116, Issue 3
  • DOI: 10.1086/300499

Generalized Subset Designs in Analytical Chemistry
journal, May 2017


The NumPy Array: A Structure for Efficient Numerical Computation
journal, March 2011

  • van der Walt, Stéfan; Colbert, S. Chris; Varoquaux, Gaël
  • Computing in Science & Engineering, Vol. 13, Issue 2
  • DOI: 10.1109/MCSE.2011.37

A Micromechanical Method for the Analysis of Three-Dimensional Smart Composites
journal, March 2018


CosmoSIS: Modular cosmological parameter estimation
journal, September 2015