skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

This content will become publicly available on February 26, 2020

Title: Likelihood non-Gaussianity in large-scale structure analyses

Abstract

The standard present-day large-scale structure (LSS) analyses make a major assumption in their Bayesian parameter inference – that the likelihood has a Gaussian form. For summary statistics currently used in LSS, this assumption, even if the underlying density field is Gaussian, cannot be correct in detail. We investigate the impact of this assumption on two recent LSS analyses: the Beutler et al. power spectrum multipole ($$P_ℓ$$) analysis and the Sinha et al. group multiplicity function ($ζ$) analysis. Using non-parametric divergence estimators on mock catalogues originally constructed for covariance matrix estimation, we identify significant non-Gaussianity in both the $$P_ℓ$$ and $ζ$ likelihoods. We then use Gaussian mixture density estimation and independent component analysis on the same mocks to construct likelihood estimates that approximate the true likelihood better than the Gaussian pseudo-likelihood. Using these likelihood estimates, we accurately estimate the true posterior probability distribution of the Beutler et al. and Sinha et al. parameters. Likelihood non-Gaussianity shifts the fσ8 constraint by -0.44σ, but otherwise does not significantly impact the overall parameter constraints of Beutler et al. For the $ζ$ analysis, using the pseudo-likelihood significantly underestimates the uncertainties and biases the constraints of the Sinha et al. halo occupation parameters. For log $$M_1$$ and α, the posteriors are shifted by +0.43σ and -0.51σ and broadened by 42 per cent and 66 per cent⁠, respectively. The divergence and likelihood estimation methods we present provide a straightforward framework for quantifying the impact of likelihood non-Gaussianity and deriving more accurate parameter constraints.

Authors:
ORCiD logo [1]; ORCiD logo [2]; ORCiD logo [3];  [4];  [5];  [6]
  1. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States); Univ. of California, Berkeley, CA (United States)
  2. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States) ; Univ. of Portsmouth (United Kingdom)
  3. Swinburne Univ. of Technology, Hawthorn, VIC (Australia); Vanderbilt Univ., Nashville, TN (United States)
  4. Vanderbilt Univ., Nashville, TN (United States)
  5. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States); Univ. of California, Berkeley, CA (United States); Carnegie Mellon Univ., Pittsburgh, PA (United States)
  6. Center for Cosmology and Particle Physics, New York University, New York, NY 10003, USA; Flatiron Institute, 162 Fifth Avenue, New York, NY 10010, USA
Publication Date:
Research Org.:
Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). National Energy Research Scientific Computing Center (NERSC)
Sponsoring Org.:
USDOE Office of Science (SC)
OSTI Identifier:
1577636
Grant/Contract Number:  
AC02-05CH11231; CE170100013
Resource Type:
Accepted Manuscript
Journal Name:
Monthly Notices of the Royal Astronomical Society
Additional Journal Information:
Journal Volume: 485; Journal Issue: 2; Journal ID: ISSN 0035-8711
Publisher:
Royal Astronomical Society
Country of Publication:
United States
Language:
English
Subject:
79 ASTRONOMY AND ASTROPHYSICS; Astronomy & Astrophysics; methods: data analysis; methods: statistical; galaxies: statistics; cosmology: observations; cosmological parameters; large-scale structure of Universe

Citation Formats

Hahn, ChangHoon, Beutler, Florian, Sinha, Manodeep, Berlind, Andreas, Ho, Shirley, and Hogg, David W. Likelihood non-Gaussianity in large-scale structure analyses. United States: N. p., 2019. Web. doi:10.1093/mnras/stz558.
Hahn, ChangHoon, Beutler, Florian, Sinha, Manodeep, Berlind, Andreas, Ho, Shirley, & Hogg, David W. Likelihood non-Gaussianity in large-scale structure analyses. United States. doi:10.1093/mnras/stz558.
Hahn, ChangHoon, Beutler, Florian, Sinha, Manodeep, Berlind, Andreas, Ho, Shirley, and Hogg, David W. Tue . "Likelihood non-Gaussianity in large-scale structure analyses". United States. doi:10.1093/mnras/stz558.
@article{osti_1577636,
title = {Likelihood non-Gaussianity in large-scale structure analyses},
author = {Hahn, ChangHoon and Beutler, Florian and Sinha, Manodeep and Berlind, Andreas and Ho, Shirley and Hogg, David W.},
abstractNote = {The standard present-day large-scale structure (LSS) analyses make a major assumption in their Bayesian parameter inference – that the likelihood has a Gaussian form. For summary statistics currently used in LSS, this assumption, even if the underlying density field is Gaussian, cannot be correct in detail. We investigate the impact of this assumption on two recent LSS analyses: the Beutler et al. power spectrum multipole ($P_ℓ$) analysis and the Sinha et al. group multiplicity function ($ζ$) analysis. Using non-parametric divergence estimators on mock catalogues originally constructed for covariance matrix estimation, we identify significant non-Gaussianity in both the $P_ℓ$ and $ζ$ likelihoods. We then use Gaussian mixture density estimation and independent component analysis on the same mocks to construct likelihood estimates that approximate the true likelihood better than the Gaussian pseudo-likelihood. Using these likelihood estimates, we accurately estimate the true posterior probability distribution of the Beutler et al. and Sinha et al. parameters. Likelihood non-Gaussianity shifts the fσ8 constraint by -0.44σ, but otherwise does not significantly impact the overall parameter constraints of Beutler et al. For the $ζ$ analysis, using the pseudo-likelihood significantly underestimates the uncertainties and biases the constraints of the Sinha et al. halo occupation parameters. For log $M_1$ and α, the posteriors are shifted by +0.43σ and -0.51σ and broadened by 42 per cent and 66 per cent⁠, respectively. The divergence and likelihood estimation methods we present provide a straightforward framework for quantifying the impact of likelihood non-Gaussianity and deriving more accurate parameter constraints.},
doi = {10.1093/mnras/stz558},
journal = {Monthly Notices of the Royal Astronomical Society},
number = 2,
volume = 485,
place = {United States},
year = {2019},
month = {2}
}

Journal Article:
Free Publicly Available Full Text
This content will become publicly available on February 26, 2020
Publisher's Version of Record

Citation Metrics:
Cited by: 1 work
Citation information provided by
Web of Science

Save / Share:

Works referenced in this record:

Estimating the Dimension of a Model
journal, March 1978


On estimating cosmology-dependent covariance matrices
journal, November 2013

  • Morrison, Christopher B.; Schneider, Michael D.
  • Journal of Cosmology and Astroparticle Physics, Vol. 2013, Issue 11
  • DOI: 10.1088/1475-7516/2013/11/009

Massive data compression for parameter-dependent covariance matrices
journal, September 2017

  • Heavens, Alan F.; Sellentin, Elena; de Mijolla, Damien
  • Monthly Notices of the Royal Astronomical Society, Vol. 472, Issue 4
  • DOI: 10.1093/mnras/stx2326

Parameter inference with estimated covariance matrices
journal, December 2015

  • Sellentin, Elena; Heavens, Alan F.
  • Monthly Notices of the Royal Astronomical Society: Letters, Vol. 456, Issue 1
  • DOI: 10.1093/mnrasl/slv190

Consistent Estimation of a Mixing Distribution
journal, September 1992


Least squares quantization in PCM
journal, March 1982


Fast and robust fixed-point algorithms for independent component analysis
journal, May 1999

  • Hyvarinen, A.
  • IEEE Transactions on Neural Networks, Vol. 10, Issue 3
  • DOI: 10.1109/72.761722

Towards accurate modelling of galaxy clustering on small scales: testing the standard ΛCDM + halo model
journal, April 2018

  • Sinha, Manodeep; Berlind, Andreas A.; McBride, Cameron K.
  • Monthly Notices of the Royal Astronomical Society, Vol. 478, Issue 1
  • DOI: 10.1093/mnras/sty967

A new Method to Correct for Fiber Collisions in Galaxy Two-Point Statistics
journal, August 2012


Halo mass distribution reconstruction across the cosmic web
journal, June 2015

  • Zhao, Cheng; Kitaura, Francisco-Shu; Chuang, Chia-Hsun
  • Monthly Notices of the Royal Astronomical Society, Vol. 451, Issue 4
  • DOI: 10.1093/mnras/stv1262

nbodykit: An Open-source, Massively Parallel Toolkit for Large-scale Structure
journal, September 2018


Cosmology and fundamental physics with the Euclid satellite
journal, April 2018

  • Amendola, Luca; Appleby, Stephen; Avgoustidis, Anastasios
  • Living Reviews in Relativity, Vol. 21, Issue 1
  • DOI: 10.1007/s41114-017-0010-3

Galaxy And Mass Assembly (GAMA): deconstructing bimodality – I. Red ones and blue ones
journal, November 2014

  • Taylor, Edward N.; Hopkins, Andrew M.; Baldry, Ivan K.
  • Monthly Notices of the Royal Astronomical Society, Vol. 446, Issue 2
  • DOI: 10.1093/mnras/stu1900

Dynamical mass Measurements of Contaminated Galaxy Clusters Using Machine Learning
journal, November 2016


The non-Gaussianity of the cosmic shear likelihood or how odd is the Chandra Deep Field South?
journal, July 2009


The three-point function in large-scale structure: redshift distortions and galaxy bias
journal, August 2005


Power Spectra Estimation for Weak Lensing
journal, June 2001

  • Hu, Wayne; White, Martin
  • The Astrophysical Journal, Vol. 554, Issue 1
  • DOI: 10.1086/321380

The Bispectrum: From Theory to Observations
journal, December 2000

  • Scoccimarro, Roman
  • The Astrophysical Journal, Vol. 544, Issue 2
  • DOI: 10.1086/317248

The clustering of galaxies in the completed SDSS-III Baryon Oscillation Spectroscopic Survey: anisotropic galaxy clustering in Fourier space
journal, December 2016

  • Beutler, Florian; Seo, Hee-Jong; Saito, Shun
  • Monthly Notices of the Royal Astronomical Society, Vol. 466, Issue 2
  • DOI: 10.1093/mnras/stw3298

Detection of baryon acoustic oscillation features in the large-scale three-point correlation function of SDSS BOSS DR12 CMASS galaxies
journal, March 2017

  • Slepian, Zachary; Eisenstein, Daniel J.; Brownstein, Joel R.
  • Monthly Notices of the Royal Astronomical Society, Vol. 469, Issue 2
  • DOI: 10.1093/mnras/stx488

The cosmological simulation code gadget-2
journal, December 2005


Myths and truths concerning estimation of power spectra: the case for a hybrid estimator
journal, April 2004


Imprints of primordial non-Gaussianities on large-scale structure: Scale-dependent bias and abundance of virialized objects
journal, June 2008


The clustering of galaxies in the completed SDSS-III Baryon Oscillation Spectroscopic Survey: cosmological analysis of the DR12 galaxy sample
journal, March 2017

  • Alam, Shadab; Ata, Metin; Bailey, Stephen
  • Monthly Notices of the Royal Astronomical Society, Vol. 470, Issue 3
  • DOI: 10.1093/mnras/stx721

Planck 2013 results. XX. Cosmology from Sunyaev–Zeldovich cluster counts
journal, October 2014


P-MaNGA: full spectral fitting and stellar population maps from prototype observations
journal, March 2015

  • Wilkinson, David M.; Maraston, Claudia; Thomas, Daniel
  • Monthly Notices of the Royal Astronomical Society, Vol. 449, Issue 1
  • DOI: 10.1093/mnras/stv301

The WiggleZ Dark Energy Survey: improved distance measurements to z = 1 with reconstruction of the baryonic acoustic feature
journal, May 2014

  • Kazin, Eyal A.; Koda, Jun; Blake, Chris
  • Monthly Notices of the Royal Astronomical Society, Vol. 441, Issue 4
  • DOI: 10.1093/mnras/stu778

Planck 2015 results : XI. CMB power spectra, likelihoods, and robustness of parameters
journal, September 2016


Imprint of DESI fiber assignment on the anisotropic power spectrum of emission line galaxies
journal, April 2017

  • Pinol, Lucas; Cahn, Robert N.; Hand, Nick
  • Journal of Cosmology and Astroparticle Physics, Vol. 2017, Issue 04
  • DOI: 10.1088/1475-7516/2017/04/008

Massive optimal data compression and density estimation for scalable, likelihood-free inference in cosmology
journal, March 2018

  • Alsing, Justin; Wandelt, Benjamin; Feeney, Stephen
  • Monthly Notices of the Royal Astronomical Society, Vol. 477, Issue 3
  • DOI: 10.1093/mnras/sty819

Divergence Estimation for Multidimensional Densities Via $k$-Nearest-Neighbor Distances
journal, May 2009

  • Wang, Qing; Kulkarni, Sanjeev R.; Verdu, Sergio
  • IEEE Transactions on Information Theory, Vol. 55, Issue 5
  • DOI: 10.1109/TIT.2009.2016060

Planck 2015 results : XXIV. Cosmology from Sunyaev-Zeldovich cluster counts
journal, September 2016


Constraining primordial non-Gaussianity with future galaxy surveys: Primordial non-Gaussianity with galaxy surveys
journal, April 2012

  • Giannantonio, Tommaso; Porciani, Cristiano; Carron, Julien
  • Monthly Notices of the Royal Astronomical Society, Vol. 422, Issue 4
  • DOI: 10.1111/j.1365-2966.2012.20604.x

A First look at Creating mock Catalogs with Machine Learning Techniques
journal, July 2013


Independent component analysis, A new concept?
journal, April 1994


A Machine Learning Approach for Dynamical mass Measurements of Galaxy Clusters
journal, April 2015


Transients from initial conditions: a perturbative analysis
journal, October 1998


How Many Clusters? Which Clustering Method? Answers Via Model-Based Cluster Analysis
journal, August 1998


Accelerating Approximate Bayesian Computation with Quantile Regression: application to cosmological redshift distributions
journal, February 2018


The clustering of galaxies in the SDSS-III Baryon Oscillation Spectroscopic Survey: RSD measurement from the power spectrum and bispectrum of the DR12 BOSS galaxies
journal, October 2016

  • Gil-Marín, Héctor; Percival, Will J.; Verde, Licia
  • Monthly Notices of the Royal Astronomical Society, Vol. 465, Issue 2
  • DOI: 10.1093/mnras/stw2679

Large covariance matrices: smooth models from the two-point correlation function
journal, July 2016

  • O'Connell, Ross; Eisenstein, Daniel; Vargas, Mariana
  • Monthly Notices of the Royal Astronomical Society, Vol. 462, Issue 3
  • DOI: 10.1093/mnras/stw1821

On the Convergence Properties of the EM Algorithm
journal, March 1983


Constraints on local primordial non-Gaussianity from large scale structure
journal, August 2008

  • Slosar, Anže; Hirata, Christopher; Seljak, Uroš
  • Journal of Cosmology and Astroparticle Physics, Vol. 2008, Issue 08
  • DOI: 10.1088/1475-7516/2008/08/031

Extreme deconvolution: Inferring complete distribution functions from noisy, heterogeneous and incomplete observations
journal, June 2011

  • Bovy, Jo; Hogg, David W.; Roweis, Sam T.
  • The Annals of Applied Statistics, Vol. 5, Issue 2B
  • DOI: 10.1214/10-AOAS439

Fast estimators for redshift-space clustering
journal, October 2015


Evidence for low Black hole spin and Physically Motivated Accretion Models from Millimeter-Vlbi Observations of Sagittarius a*
journal, June 2011

  • Broderick, Avery E.; Fish, Vincent L.; Doeleman, Sheperd S.
  • The Astrophysical Journal, Vol. 735, Issue 2
  • DOI: 10.1088/0004-637X/735/2/110

Planck 2013 results. XV. CMB power spectra and likelihood
journal, October 2014


Approximate Bayesian computation in large-scale structure: constraining the galaxy–halo connection
journal, April 2017

  • Hahn, ChangHoon; Vakili, Mohammadjavad; Walsh, Kilian
  • Monthly Notices of the Royal Astronomical Society, Vol. 469, Issue 3
  • DOI: 10.1093/mnras/stx894

MultiDark simulations: the story of dark matter halo concentrations and density profiles
journal, February 2016

  • Klypin, Anatoly; Yepes, Gustavo; Gottlöber, Stefan
  • Monthly Notices of the Royal Astronomical Society, Vol. 457, Issue 4
  • DOI: 10.1093/mnras/stw248

Measuring line-of-sight-dependent Fourier-space clustering using FFTs
journal, August 2015

  • Bianchi, Davide; Gil-Marín, Héctor; Ruggeri, Rossana
  • Monthly Notices of the Royal Astronomical Society: Letters, Vol. 453, Issue 1
  • DOI: 10.1093/mnrasl/slv090

Karhunen‐Loeve Eigenvalue Problems in Cosmology: How Should We Tackle Large Data Sets?
journal, May 1997

  • Tegmark, Max; Taylor, Andy N.; Heavens, Alan F.
  • The Astrophysical Journal, Vol. 480, Issue 1
  • DOI: 10.1086/303939

Information criteria for astrophysical model selection
journal, May 2007


An optimal FFT-based anisotropic power spectrum estimator
journal, July 2017


Dependence of cosmic shear covariances on cosmology: Impact on parameter estimation
journal, June 2009


Cosmological structure formation with augmented Lagrangian perturbation theory
journal, August 2013

  • Kitaura, Francisco-Shu; Heß, Steffen
  • Monthly Notices of the Royal Astronomical Society: Letters, Vol. 435, Issue 1
  • DOI: 10.1093/mnrasl/slt101

Practical Bayesian Density Estimation Using Mixtures of Normals
journal, September 1997


The clustering of galaxies in the SDSS-III Baryon Oscillation Spectroscopic Survey: modelling the clustering and halo occupation distribution of BOSS CMASS galaxies in the Final Data Release
journal, April 2016

  • Rodríguez-Torres, Sergio A.; Chuang, Chia-Hsun; Prada, Francisco
  • Monthly Notices of the Royal Astronomical Society, Vol. 460, Issue 2
  • DOI: 10.1093/mnras/stw1014

Estimating mutual information
journal, June 2004


Transients from initial conditions in cosmological simulations
journal, November 2006


Hybrid estimation of cosmic microwave background polarization power spectra
journal, July 2006


Improved primordial non-Gaussianity constraints from measurements of galaxy clustering and the integrated Sachs-Wolfe effect
journal, January 2014


Automatic physical inference with information maximizing neural networks
journal, April 2018


Perturbative approach to covariance matrix of the matter power spectrum
journal, December 2016

  • Mohammed, Irshad; Seljak, Uroš; Vlah, Zvonimir
  • Monthly Notices of the Royal Astronomical Society, Vol. 466, Issue 1
  • DOI: 10.1093/mnras/stw3196

The clustering of Galaxies in the SDSS-III Baryon Oscillation Spectroscopic Survey: potential systematics in fitting of baryon acoustic feature
journal, September 2014

  • Vargas-Magaña, Mariana; Ho, Shirley; Xu, Xiaoying
  • Monthly Notices of the Royal Astronomical Society, Vol. 445, Issue 1
  • DOI: 10.1093/mnras/stu1681

The WiggleZ Dark Energy Survey: Final data release and cosmological results
journal, November 2012


The clustering of galaxies in the SDSS-III DR9 Baryon Oscillation Spectroscopic Survey: constraints on primordial non-Gaussianity
journal, November 2012

  • Ross, Ashley J.; Percival, Will J.; Carnero, Aurelio
  • Monthly Notices of the Royal Astronomical Society, Vol. 428, Issue 2
  • DOI: 10.1093/mnras/sts094

The clustering of galaxies in the SDSS-III Baryon Oscillation Spectroscopic Survey: mock galaxy catalogues for the BOSS Final Data Release
journal, January 2016

  • Kitaura, Francisco-Shu; Rodríguez-Torres, Sergio; Chuang, Chia-Hsun
  • Monthly Notices of the Royal Astronomical Society, Vol. 456, Issue 4
  • DOI: 10.1093/mnras/stv2826

Parameter estimation in astronomy through application of the likelihood ratio
journal, March 1979


Statistical analysis of galaxy surveys - I. Robust error estimation for two-point clustering statistics
journal, June 2009


Percolation Galaxy Groups and Clusters in the SDSS Redshift Survey: Identification, Catalogs, and the Multiplicity Function
journal, November 2006

  • Berlind, Andreas A.; Frieman, Joshua; Weinberg, David H.
  • The Astrophysical Journal Supplement Series, Vol. 167, Issue 1
  • DOI: 10.1086/508170

On the insufficiency of arbitrarily precise covariance matrices: non-Gaussian weak-lensing likelihoods
journal, September 2017

  • Sellentin, Elena; Heavens, Alan F.
  • Monthly Notices of the Royal Astronomical Society, Vol. 473, Issue 2
  • DOI: 10.1093/mnras/stx2491

Constraining the halo bispectrum in real and redshift space from perturbation theory and non-linear stochastic bias
journal, April 2015

  • Kitaura, F. -S.; Gil-Marin, H.; Scoccola, C. G.
  • Monthly Notices of the Royal Astronomical Society, Vol. 450, Issue 2
  • DOI: 10.1093/mnras/stv645

Nonparametric kernel estimators for image classification
conference, June 2012

  • Poczos, B.; Sutherland, D. J.
  • 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • DOI: 10.1109/CVPR.2012.6248028

Generalized massive optimal data compression
journal, February 2018

  • Alsing, Justin; Wandelt, Benjamin
  • Monthly Notices of the Royal Astronomical Society: Letters, Vol. 476, Issue 1
  • DOI: 10.1093/mnrasl/sly029

Independent component analysis: algorithms and applications
journal, June 2000


The Bispectrum as a Signature of Gravitational Instability in Redshift Space
journal, June 1999

  • Scoccimarro, Roman; Couchman, H. M. P.; Frieman, Joshua A.
  • The Astrophysical Journal, Vol. 517, Issue 2
  • DOI: 10.1086/307220

Correlations in the Spatial Power Spectra Inferred from Angular Clustering: Methods and Application to the Automated Plate Measuring Survey
journal, January 2001

  • Eisenstein, Daniel J.; Zaldarriaga, Matias
  • The Astrophysical Journal, Vol. 546, Issue 1
  • DOI: 10.1086/318226

The clustering of galaxies in the completed SDSS-III Baryon Oscillation Spectroscopic Survey: observational systematics and baryon acoustic oscillations in the correlation function
journal, September 2016

  • Ross, Ashley J.; Beutler, Florian; Chuang, Chia-Hsun
  • Monthly Notices of the Royal Astronomical Society, Vol. 464, Issue 1
  • DOI: 10.1093/mnras/stw2372

Modelling baryon acoustic oscillations with perturbation theory and stochastic halo biasing
journal, December 2013

  • Kitaura, Francisco-Shu; Yepes, Gustavo; Prada, Francisco
  • Monthly Notices of the Royal Astronomical Society: Letters, Vol. 439, Issue 1
  • DOI: 10.1093/mnrasl/slt172

Breaking the Degeneracies between Cosmology and Galaxy Bias
journal, April 2007

  • Zheng, Zheng; Weinberg, David H.
  • The Astrophysical Journal, Vol. 659, Issue 1
  • DOI: 10.1086/512151

Including parameter dependence in the data and covariance for cosmological inference
journal, December 2015


The evolution of large-scale structure in a universe dominated by cold dark matter
journal, May 1985

  • Davis, M.; Efstathiou, G.; Frenk, C. S.
  • The Astrophysical Journal, Vol. 292
  • DOI: 10.1086/163168