skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Improving galaxy clustering measurements with deep learning: analysis of the DECaLS DR7 data

Abstract

ABSTRACT Robust measurements of cosmological parameters from galaxy surveys rely on our understanding of systematic effects that impact the observed galaxy density field. In this paper, we present, validate, and implement the idea of adopting the systematics mitigation method of artificial neural networks for modelling the relationship between the target galaxy density field and various observational realities including but not limited to Galactic extinction, seeing, and stellar density. Our method by construction allows a wide class of models and alleviates overtraining by performing k-fold cross-validation and dimensionality reduction via backward feature elimination. By permuting the choice of the training, validation, and test sets, we construct a selection mask for the entire footprint. We apply our method on the extended Baryon Oscillation Spectroscopic Survey (eBOSS) Emission Line Galaxies (ELGs) selection from the Dark Energy Camera Legacy Survey (DECaLS) Data Release 7 and show that the spurious large-scale contamination due to imaging systematics can be significantly reduced by up-weighting the observed galaxy density using the selection mask from the neural network and that our method is more effective than the conventional linear and quadratic polynomial functions. We perform extensive analyses on simulated mock data sets with and without systematic effects. Our analysesmore » indicate that our methodology is more robust to overfitting compared to the conventional methods. This method can be utilized in the catalogue generation of future spectroscopic galaxy surveys such as eBOSS and Dark Energy Spectroscopic Instrument (DESI) to better mitigate observational systematics.« less

Authors:
ORCiD logo [1]; ORCiD logo [1]; ORCiD logo [2];  [3]
  1. Department of Physics and Astronomy, Ohio University, Athens, OH 45701, USA
  2. The Center of Cosmology and Astro Particle Physics, the Ohio State University, Columbus, OH 43210, USA
  3. School of Electrical Engineering and Computer Science, Ohio University, Athens, OH 45701, USA
Publication Date:
Sponsoring Org.:
USDOE
OSTI Identifier:
1630781
Grant/Contract Number:  
SC0014329; AC02-05CH11231
Resource Type:
Published Article
Journal Name:
Monthly Notices of the Royal Astronomical Society
Additional Journal Information:
Journal Name: Monthly Notices of the Royal Astronomical Society Journal Volume: 495 Journal Issue: 2; Journal ID: ISSN 0035-8711
Publisher:
Oxford University Press
Country of Publication:
United Kingdom
Language:
English

Citation Formats

Rezaie, Mehdi, Seo, Hee-Jong, Ross, Ashley J., and Bunescu, Razvan C. Improving galaxy clustering measurements with deep learning: analysis of the DECaLS DR7 data. United Kingdom: N. p., 2020. Web. doi:10.1093/mnras/staa1231.
Rezaie, Mehdi, Seo, Hee-Jong, Ross, Ashley J., & Bunescu, Razvan C. Improving galaxy clustering measurements with deep learning: analysis of the DECaLS DR7 data. United Kingdom. doi:https://doi.org/10.1093/mnras/staa1231
Rezaie, Mehdi, Seo, Hee-Jong, Ross, Ashley J., and Bunescu, Razvan C. Mon . "Improving galaxy clustering measurements with deep learning: analysis of the DECaLS DR7 data". United Kingdom. doi:https://doi.org/10.1093/mnras/staa1231.
@article{osti_1630781,
title = {Improving galaxy clustering measurements with deep learning: analysis of the DECaLS DR7 data},
author = {Rezaie, Mehdi and Seo, Hee-Jong and Ross, Ashley J. and Bunescu, Razvan C.},
abstractNote = {ABSTRACT Robust measurements of cosmological parameters from galaxy surveys rely on our understanding of systematic effects that impact the observed galaxy density field. In this paper, we present, validate, and implement the idea of adopting the systematics mitigation method of artificial neural networks for modelling the relationship between the target galaxy density field and various observational realities including but not limited to Galactic extinction, seeing, and stellar density. Our method by construction allows a wide class of models and alleviates overtraining by performing k-fold cross-validation and dimensionality reduction via backward feature elimination. By permuting the choice of the training, validation, and test sets, we construct a selection mask for the entire footprint. We apply our method on the extended Baryon Oscillation Spectroscopic Survey (eBOSS) Emission Line Galaxies (ELGs) selection from the Dark Energy Camera Legacy Survey (DECaLS) Data Release 7 and show that the spurious large-scale contamination due to imaging systematics can be significantly reduced by up-weighting the observed galaxy density using the selection mask from the neural network and that our method is more effective than the conventional linear and quadratic polynomial functions. We perform extensive analyses on simulated mock data sets with and without systematic effects. Our analyses indicate that our methodology is more robust to overfitting compared to the conventional methods. This method can be utilized in the catalogue generation of future spectroscopic galaxy surveys such as eBOSS and Dark Energy Spectroscopic Instrument (DESI) to better mitigate observational systematics.},
doi = {10.1093/mnras/staa1231},
journal = {Monthly Notices of the Royal Astronomical Society},
number = 2,
volume = 495,
place = {United Kingdom},
year = {2020},
month = {5}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
DOI: https://doi.org/10.1093/mnras/staa1231

Save / Share:

Works referenced in this record:

Systematic Effects in Large-Scale Angular Power Spectra of Photometric Quasars and Implications for Constraining Primordial Non-Gaussianity
journal, June 2013

  • Pullen, Anthony R.; Hirata, Christopher M.
  • Publications of the Astronomical Society of the Pacific, Vol. 125, Issue 928
  • DOI: 10.1086/671189

No galaxy left behind: accurate measurements with the faintest objects in the Dark Energy Survey
journal, January 2016

  • Suchyta, E.; Huff, E. M.; Aleksić, J.
  • Monthly Notices of the Royal Astronomical Society, Vol. 457, Issue 1
  • DOI: 10.1093/mnras/stv2953

Measurement of the cosmic microwave background spectrum by the COBE FIRAS instrument
journal, January 1994

  • Mather, J. C.; Cheng, E. S.; Cottingham, D. A.
  • The Astrophysical Journal, Vol. 420
  • DOI: 10.1086/173574

Maps of Dust Infrared Emission for Use in Estimation of Reddening and Cosmic Microwave Background Radiation Foregrounds
journal, June 1998

  • Schlegel, David J.; Finkbeiner, Douglas P.; Davis, Marc
  • The Astrophysical Journal, Vol. 500, Issue 2
  • DOI: 10.1086/305772

Mock galaxy catalogues using the quick particle mesh method
journal, November 2013

  • White, Martin; Tinker, Jeremy L.; McBride, Cameron K.
  • Monthly Notices of the Royal Astronomical Society, Vol. 437, Issue 3
  • DOI: 10.1093/mnras/stt2071

Statistical Analysis of Catalogs of Extragalactic Objects. I. Theory
journal, October 1973

  • Peebles, P. J. E.
  • The Astrophysical Journal, Vol. 185
  • DOI: 10.1086/152431

Multiclass cancer diagnosis using tumor gene expression signatures
journal, December 2001

  • Ramaswamy, S.; Tamayo, P.; Rifkin, R.
  • Proceedings of the National Academy of Sciences, Vol. 98, Issue 26
  • DOI: 10.1073/pnas.211566398

The SDSS-IV extended Baryon Oscillation Spectroscopic Survey: final emission line galaxy target selection
journal, July 2017

  • Raichoor, A.; Comparat, J.; Delubac, T.
  • Monthly Notices of the Royal Astronomical Society, Vol. 471, Issue 4
  • DOI: 10.1093/mnras/stx1790

Probing Dark Energy with Baryonic Acoustic Oscillations from Future Large Galaxy Redshift Surveys
journal, December 2003

  • Seo, Hee‐Jong; Eisenstein, Daniel J.
  • The Astrophysical Journal, Vol. 598, Issue 2
  • DOI: 10.1086/379122

Gaia Data Release 2 : Summary of the contents and survey properties
journal, August 2018


Theory of cosmological perturbations
journal, June 1992


Project Overview of the Beijing–Arizona Sky Survey
journal, April 2017

  • Zou, Hu; Zhou, Xu; Fan, Xiaohui
  • Publications of the Astronomical Society of the Pacific, Vol. 129, Issue 976
  • DOI: 10.1088/1538-3873/aa65ba

nbodykit: An Open-source, Massively Parallel Toolkit for Large-scale Structure
journal, September 2018


Galaxy clustering, photometric redshifts and diagnosis of systematics in the DES Science Verification data
journal, December 2015

  • Crocce, M.; Carretero, J.; Bauer, A. H.
  • Monthly Notices of the Royal Astronomical Society, Vol. 455, Issue 4
  • DOI: 10.1093/mnras/stv2590

Cosmic Complementarity: [ITAL]H[/ITAL][TINF]0[/TINF] and Ω[TINF][CLC][ITAL]m[/ITAL][/CLC][/TINF] from Combining Cosmic Microwave Background Experiments and Redshift Surveys
journal, September 1998

  • Eisenstein, Daniel J.; Hu, Wayne; Tegmark, Max
  • The Astrophysical Journal, Vol. 504, Issue 2
  • DOI: 10.1086/311582

Unbiased contaminant removal for 3D galaxy power spectrum measurements
journal, August 2016

  • Kalus, B.; Percival, W. J.; Bacon, D. J.
  • Monthly Notices of the Royal Astronomical Society, Vol. 463, Issue 1
  • DOI: 10.1093/mnras/stw2008

How to measure CMB power spectra without losing information
journal, May 1997


Dark energy and cosmic sound
journal, November 2005


HEALPix: A Framework for High‐Resolution Discretization and Fast Analysis of Data Distributed on the Sphere
journal, April 2005

  • Gorski, K. M.; Hivon, E.; Banday, A. J.
  • The Astrophysical Journal, Vol. 622, Issue 2
  • DOI: 10.1086/427976

A map-based method for eliminating systematic modes from galaxy clustering power spectra with application to BOSS
journal, October 2018

  • Kalus, B.; Percival, W. J.; Bacon, D. J.
  • Monthly Notices of the Royal Astronomical Society, Vol. 482, Issue 1
  • DOI: 10.1093/mnras/sty2655

Fast Cosmic Microwave Background Analyses via Correlation Functions
journal, February 2001

  • Szapudi, István; Prunet, Simon; Pogosyan, Dmitry
  • The Astrophysical Journal, Vol. 548, Issue 2
  • DOI: 10.1086/319105

HI4PI: a full-sky H i survey based on EBHIS and GASS
journal, October 2016


Approximation by superpositions of a sigmoidal function
journal, December 1989

  • Cybenko, G.
  • Mathematics of Control, Signals, and Systems, Vol. 2, Issue 4
  • DOI: 10.1007/BF02551274

The clustering of the SDSS-IV extended Baryon Oscillation Spectroscopic Survey DR14 quasar sample: first measurement of baryon acoustic oscillations between redshift 0.8 and 2.2
journal, October 2017

  • Ata, Metin; Baumgarten, Falk; Bautista, Julian
  • Monthly Notices of the Royal Astronomical Society, Vol. 473, Issue 4
  • DOI: 10.1093/mnras/stx2630

Correlation of CMB with large-scale structure. I. Integrated Sachs-Wolfe tomography and cosmological implications
journal, August 2008


Clustering Analyses of 300,000 Photometrically Classified Quasars. II. The Excess on Very Small Scales
journal, March 2007

  • Myers, Adam D.; Brunner, Robert J.; Richards, Gordon T.
  • The Astrophysical Journal, Vol. 658, Issue 1
  • DOI: 10.1086/511520

Imprints of primordial non-Gaussianities on large-scale structure: Scale-dependent bias and abundance of virialized objects
journal, June 2008


The clustering of galaxies in the completed SDSS-III Baryon Oscillation Spectroscopic Survey: cosmological analysis of the DR12 galaxy sample
journal, March 2017

  • Alam, Shadab; Ata, Metin; Bailey, Stephen
  • Monthly Notices of the Royal Astronomical Society, Vol. 470, Issue 3
  • DOI: 10.1093/mnras/stx721

The clustering of galaxies in the SDSS-III Baryon Oscillation Spectroscopic Survey: analysis of potential systematics: Systematic analysis of BOSS 3D clustering
journal, June 2012

  • Ross, Ashley J.; Percival, Will J.; Sánchez, Ariel G.
  • Monthly Notices of the Royal Astronomical Society, Vol. 424, Issue 1
  • DOI: 10.1111/j.1365-2966.2012.21235.x

Analysis of a complete galaxy redshift survey - II. The field-galaxy luminosity function
journal, May 1988

  • Efstathiou, G.; Ellis, R. S.; Peterson, B. A.
  • Monthly Notices of the Royal Astronomical Society, Vol. 232, Issue 2
  • DOI: 10.1093/mnras/232.2.431

Ridge Regression: Biased Estimation for Nonorthogonal Problems
journal, February 1970


Measuring the Galaxy Power Spectrum with Future Redshift Surveys
journal, June 1998

  • Tegmark, Max; Hamilton, Andrew J. S.; Strauss, Michael A.
  • The Astrophysical Journal, Vol. 499, Issue 2
  • DOI: 10.1086/305663

Planck 2015 results : XIII. Cosmological parameters
journal, September 2016


Excess Clustering on Large Scales in the MegaZ DR7 Photometric Redshift Survey
journal, June 2011


The 2dF Galaxy Redshift Survey: spectra and redshifts
journal, December 2001


Observational Evidence from Supernovae for an Accelerating Universe and a Cosmological Constant
journal, September 1998

  • Riess, Adam G.; Filippenko, Alexei V.; Challis, Peter
  • The Astronomical Journal, Vol. 116, Issue 3
  • DOI: 10.1086/300499

The angular power spectra of photometric Sloan Digital Sky Survey luminous red galaxies: The angular power spectra of photometric LRGs
journal, February 2011


Exploiting the full potential of photometric quasar surveys: optimal power spectra through blind mitigation of systematics
journal, August 2014

  • Leistedt, Boris; Peiris, Hiranya V.
  • Monthly Notices of the Royal Astronomical Society, Vol. 444, Issue 1
  • DOI: 10.1093/mnras/stu1439

The SDSS-IV Extended Baryon Oscillation Spectroscopic Survey: Baryon Acoustic Oscillations at Redshift of 0.72 with the DR14 Luminous Red Galaxy Sample
journal, August 2018

  • Bautista, Julian E.; Vargas-Magaña, Mariana; Dawson, Kyle S.
  • The Astrophysical Journal, Vol. 863, Issue 1
  • DOI: 10.3847/1538-4357/aacea5

An Ultra Fast Image Generator (UFig) for wide-field astronomy
journal, February 2013


Estimating the large-scale angular power spectrum in the presence of systematics: a case study of Sloan Digital Sky Survey quasars
journal, September 2013

  • Leistedt, Boris; Peiris, Hiranya V.; Mortlock, Daniel J.
  • Monthly Notices of the Royal Astronomical Society, Vol. 435, Issue 3
  • DOI: 10.1093/mnras/stt1359

A lognormal model for the cosmological mass distribution
journal, January 1991

  • Coles, Peter; Jones, Bernard
  • Monthly Notices of the Royal Astronomical Society, Vol. 248, Issue 1
  • DOI: 10.1093/mnras/248.1.1

The Dark Energy Survey
journal, June 2005


Fast estimation of polarization power spectra using correlation functions
journal, May 2004


The Sloan Digital Sky Survey: Technical Summary
journal, September 2000

  • York, Donald G.; Adelman, J.; Anderson, Jr., John E.
  • The Astronomical Journal, Vol. 120, Issue 3
  • DOI: 10.1086/301513

Structure in the COBE differential microwave radiometer first-year maps
journal, September 1992

  • Smoot, G. F.; Bennett, C. L.; Kogut, A.
  • The Astrophysical Journal, Vol. 396
  • DOI: 10.1086/186504

On the approximate realization of continuous mappings by neural networks
journal, January 1989


Calibration errors unleashed: effects on cosmological parameters and requirements for large-scale structure surveys
journal, May 2013

  • Huterer, Dragan; Cunha, Carlos E.; Fang, Wenjuan
  • Monthly Notices of the Royal Astronomical Society, Vol. 432, Issue 4
  • DOI: 10.1093/mnras/stt653

Unbiased methods for removing systematics from galaxy clustering measurements
journal, December 2015

  • Elsner, Franz; Leistedt, Boris; Peiris, Hiranya V.
  • Monthly Notices of the Royal Astronomical Society, Vol. 456, Issue 2
  • DOI: 10.1093/mnras/stv2777

Extremely randomized trees
journal, March 2006


The WiggleZ Dark Energy Survey: survey design and first data release
journal, January 2010

  • Drinkwater, Michael J.; Jurek, Russell J.; Blake, Chris
  • Monthly Notices of the Royal Astronomical Society, Vol. 401, Issue 3
  • DOI: 10.1111/j.1365-2966.2009.15754.x

Clustering of Sloan Digital sky Survey iii Photometric Luminous Galaxies: the Measurement, Systematics, and Cosmological Implications
journal, November 2012


Multilayer feedforward networks are universal approximators
journal, January 1989


Analysis of Systematic Effects and Statistical Uncertainties in Angular Clustering of Galaxies from Early Sloan Digital Sky Survey Data
journal, November 2002

  • Scranton, Ryan; Johnston, David; Dodelson, Scott
  • The Astrophysical Journal, Vol. 579, Issue 1
  • DOI: 10.1086/342786

Exact likelihood evaluations and foreground marginalization in low resolution WMAP data
journal, June 2004


What is the best way to measure baryonic acoustic oscillations?
journal, October 2008


The SDSS-IV eBOSS: emission line galaxy catalogues at z ≈ 0.8 and study of systematic errors in the angular clustering
journal, October 2016

  • Delubac, T.; Raichoor, A.; Comparat, J.
  • Monthly Notices of the Royal Astronomical Society, Vol. 465, Issue 2
  • DOI: 10.1093/mnras/stw2741

MASTER of the Cosmic Microwave Background Anisotropy Power Spectrum: A Fast Method for Statistical Analysis of Large and Complex Cosmic Microwave Background Data Sets
journal, March 2002

  • Hivon, Eric; Gorski, Krzysztof M.; Netterfield, C. Barth
  • The Astrophysical Journal, Vol. 567, Issue 1
  • DOI: 10.1086/338126

Interpolation, realization, and reconstruction of noisy, irregularly sampled data
journal, October 1992

  • Rybicki, George B.; Press, William H.
  • The Astrophysical Journal, Vol. 398
  • DOI: 10.1086/171845

POKER: estimating the power spectrum of diffuse emission with complex masks and at high angular resolution
journal, November 2011


Why Does Deep and Cheap Learning Work So Well?
journal, July 2017

  • Lin, Henry W.; Tegmark, Max; Rolnick, David
  • Journal of Statistical Physics, Vol. 168, Issue 6
  • DOI: 10.1007/s10955-017-1836-5

Wrappers for feature subset selection
journal, December 1997


Ameliorating systematic uncertainties in the angular clustering of galaxies: a study using the SDSS-III: Ameliorating systematic uncertainties in w(θ)
journal, September 2011


The clustering of galaxies in the SDSS-III DR9 Baryon Oscillation Spectroscopic Survey: constraints on primordial non-Gaussianity
journal, November 2012

  • Ross, Ashley J.; Percival, Will J.; Carnero, Aurelio
  • Monthly Notices of the Royal Astronomical Society, Vol. 428, Issue 2
  • DOI: 10.1093/mnras/sts094

LSST: From Science Drivers to Reference Design and Anticipated Data Products
journal, March 2019

  • Ivezić, Željko; Kahn, Steven M.; Tyson, J. Anthony
  • The Astrophysical Journal, Vol. 873, Issue 2
  • DOI: 10.3847/1538-4357/ab042c

The Sdss-Iv Extended Baryon Oscillation Spectroscopic Survey: Luminous red Galaxy Target Selection
journal, June 2016

  • Prakash, Abhishek; Licquia, Timothy C.; Newman, Jeffrey A.
  • The Astrophysical Journal Supplement Series, Vol. 224, Issue 2
  • DOI: 10.3847/0067-0049/224/2/34

Capabilities of a four-layered feedforward neural network: four layers versus three
journal, March 1997

  • Tamura, S.; Tateishi, M.
  • IEEE Transactions on Neural Networks, Vol. 8, Issue 2
  • DOI: 10.1109/72.557662

Measuring Dark Energy Properties with Photometrically Classified Pan-STARRS Supernovae. II. Cosmological Parameters
journal, April 2018


Clustering of quasars in SDSS-IV eBOSS: study of potential systematics and bias determination
journal, July 2017

  • Laurent, Pierre; Eftekharzadeh, Sarah; Goff, Jean-Marc Le
  • Journal of Cosmology and Astroparticle Physics, Vol. 2017, Issue 07
  • DOI: 10.1088/1475-7516/2017/07/017

The power spectrum of IRAS galaxies
journal, January 1993

  • Fisher, Karl B.; Davis, Marc; Strauss, Michael A.
  • The Astrophysical Journal, Vol. 402
  • DOI: 10.1086/172110

Measuring Reddening with Sloan Digital sky Survey Stellar Spectra and Recalibrating sfd
journal, August 2011


Mapping and Simulating Systematics due to Spatially Varying Observing Conditions in des Science Verification data
journal, October 2016

  • Leistedt, B.; Peiris, H. V.; Elsner, F.
  • The Astrophysical Journal Supplement Series, Vol. 226, Issue 2
  • DOI: 10.3847/0067-0049/226/2/24

Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
conference, December 2015

  • He, Kaiming; Zhang, Xiangyu; Ren, Shaoqing
  • 2015 IEEE International Conference on Computer Vision (ICCV)
  • DOI: 10.1109/ICCV.2015.123

Clustering in real space and in redshift space
journal, July 1987

  • Kaiser, Nick
  • Monthly Notices of the Royal Astronomical Society, Vol. 227, Issue 1
  • DOI: 10.1093/mnras/227.1.1

Higher Order Angular Galaxy Correlations in the SDSS: Redshift and Color Dependence of Nonlinear Bias
journal, August 2007

  • Ross, Ashley J.; Brunner, Robert J.; Myers, Adam D.
  • The Astrophysical Journal, Vol. 665, Issue 1
  • DOI: 10.1086/519020

Measurements of Ω and Λ from 42 High‐Redshift Supernovae
journal, June 1999

  • Perlmutter, S.; Aldering, G.; Goldhaber, G.
  • The Astrophysical Journal, Vol. 517, Issue 2
  • DOI: 10.1086/307221

Improving deep neural networks for LVCSR using rectified linear units and dropout
conference, May 2013

  • Dahl, George E.; Sainath, Tara N.; Hinton, Geoffrey E.
  • ICASSP 2013 - 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
  • DOI: 10.1109/ICASSP.2013.6639346

The clustering of galaxies in the completed SDSS-III Baryon Oscillation Spectroscopic Survey: observational systematics and baryon acoustic oscillations in the correlation function
journal, September 2016

  • Ross, Ashley J.; Beutler, Florian; Chuang, Chia-Hsun
  • Monthly Notices of the Royal Astronomical Society, Vol. 464, Issue 1
  • DOI: 10.1093/mnras/stw2372

The Wide-Field Infrared Survey Explorer (Wise): Mission Description and Initial On-Orbit Performance
journal, November 2010


Random Forests
journal, January 2001


The Ninth data Release of the Sloan Digital sky Survey: First Spectroscopic data from the Sdss-Iii Baryon Oscillation Spectroscopic Survey
journal, November 2012

  • Ahn, Christopher P.; Alexandroff, Rachael; Allende Prieto, Carlos
  • The Astrophysical Journal Supplement Series, Vol. 203, Issue 2
  • DOI: 10.1088/0067-0049/203/2/21