skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: The Hyper Suprime-Cam software pipeline

Abstract

Here in this article, we describe the optical imaging data processing pipeline developed for the Subaru Telescope’s Hyper Suprime-Cam (HSC) instrument. The HSC Pipeline builds on the prototype pipeline being developed by the Large Synoptic Survey Telescope’s Data Management system, adding customizations for HSC, large-scale processing capabilities, and novel algorithms that have since been reincorporated into the LSST codebase. While designed primarily to reduce HSC Subaru Strategic Program (SSP) data, it is also the recommended pipeline for reducing general-observer HSC data. The HSC pipeline includes high-level processing steps that generate coadded images and science-ready catalogs as well as low-level detrending and image characterizations.

Authors:
 [1];  [1];  [2];  [3];  [3];  [3];  [1];  [3];  [1];  [4];  [3];  [5];  [1];  [6];  [7];  [8];  [9];  [10];  [6];  [11] more »;  [12];  [10];  [1];  [1];  [13];  [14];  [4];  [15];  [5];  [16];  [6];  [1];  [1];  [3];  [3] « less
  1. Princeton Univ., NJ (United States). Dept. of Astrophysical Sciences
  2. Orbital Insight, Mountain View, CA (United States); Univ. of Tokyo (Japan). Kavli Inst. for the Physics and Mathematics of the Universe (Kavli IPMU, WPI)
  3. National Astronomical Observatory of Japan, Mitaka, Tokyo (Japan)
  4. National Astronomical Observatory of Japan, Mitaka, Tokyo (Japan); Graduate Univ. for Advanced Studies (SOKENDAI), Mitaka, Tokyo (Japan). Dept. of Astronomy, School of Science
  5. Univ. of Tokyo (Japan). Kavli Inst. for the Physics and Mathematics of the Universe (Kavli IPMU, WPI)
  6. Univ. of Washington, Seattle, WA (United States). Dept. of Astronomy
  7. Princeton Univ., NJ (United States). Dept. of Physics
  8. Univ. of Geneva (Switzerland). Dept. of Astronomy
  9. Princeton Univ., NJ (United States). Dept. of Astrophysical Sciences; Massive Dynamics, Princeton, NJ (United States)
  10. SLAC National Accelerator Lab., Menlo Park, CA (United States)
  11. Univ. of Toronto, ON (Canada). Dunlap Inst. and Dept. of Astronomy and Astrophysics
  12. Univ. of Tokyo (Japan). Kavli Inst. for the Physics and Mathematics of the Universe (Kavli IPMU, WPI); Massive Dynamics, Princeton, NJ (United States)
  13. Carnegie Mellon Univ., Pittsburgh, PA (United States). McWilliams Center for Cosmology, Dept. of Physics
  14. Univ. of Tokyo (Japan). Kavli Inst. for the Physics and Mathematics of the Universe (Kavli IPMU, WPI); California Inst. of Technology (CalTech), Pasadena, CA (United States). Jet Propulsion Lab.
  15. Univ. of Tokyo (Japan). Kavli Inst. for the Physics and Mathematics of the Universe (Kavli IPMU, WPI); Univ. of Tokyo (Japan). Dept. of Physics
  16. Inst. of Physical and Chemical Research (RIKEN), Wako (Japan). High Energy Astrophysics Lab.; Brookhaven National Lab. (BNL), Upton, NY (United States). RIKEN Research Center
Publication Date:
Research Org.:
SLAC National Accelerator Lab., Menlo Park, CA (United States)
Sponsoring Org.:
USDOE; Japan Society for the Promotion of Science (JSPS); Japan Science and Technology Agency (JST)
OSTI Identifier:
1425668
Grant/Contract Number:  
AC02-76SF00515; NNX08AR22G; AST-1238877; JP15K17600; JP16H01089
Resource Type:
Accepted Manuscript
Journal Name:
Publications of the Astronomical Society of Japan
Additional Journal Information:
Journal Volume: 70; Journal Issue: SP1; Journal ID: ISSN 0004-6264
Publisher:
Astronomical Society of Japan
Country of Publication:
United States
Language:
English
Subject:
79 ASTRONOMY AND ASTROPHYSICS

Citation Formats

Bosch, James, Armstrong, Robert, Bickerton, Steven, Furusawa, Hisanori, Ikeda, Hiroyuki, Koike, Michitaro, Lupton, Robert, Mineo, Sogo, Price, Paul, Takata, Tadafumi, Tanaka, Masayuki, Yasuda, Naoki, AlSayyad, Yusra, Becker, Andrew C., Coulton, William, Coupon, Jean, Garmilla, Jose, Huang, Song, Krughoff, K. Simon, Lang, Dustin, Leauthaud, Alexie, Lim, Kian-Tat, Lust, Nate B., MacArthur, Lauren A., Mandelbaum, Rachel, Miyatake, Hironao, Miyazaki, Satoshi, Murata, Ryoma, More, Surhud, Okura, Yuki, Owen, Russell, Swinbank, John D., Strauss, Michael A., Yamada, Yoshihiko, and Yamanoi, Hitomi. The Hyper Suprime-Cam software pipeline. United States: N. p., 2017. Web. doi:10.1093/pasj/psx080.
Bosch, James, Armstrong, Robert, Bickerton, Steven, Furusawa, Hisanori, Ikeda, Hiroyuki, Koike, Michitaro, Lupton, Robert, Mineo, Sogo, Price, Paul, Takata, Tadafumi, Tanaka, Masayuki, Yasuda, Naoki, AlSayyad, Yusra, Becker, Andrew C., Coulton, William, Coupon, Jean, Garmilla, Jose, Huang, Song, Krughoff, K. Simon, Lang, Dustin, Leauthaud, Alexie, Lim, Kian-Tat, Lust, Nate B., MacArthur, Lauren A., Mandelbaum, Rachel, Miyatake, Hironao, Miyazaki, Satoshi, Murata, Ryoma, More, Surhud, Okura, Yuki, Owen, Russell, Swinbank, John D., Strauss, Michael A., Yamada, Yoshihiko, & Yamanoi, Hitomi. The Hyper Suprime-Cam software pipeline. United States. doi:10.1093/pasj/psx080.
Bosch, James, Armstrong, Robert, Bickerton, Steven, Furusawa, Hisanori, Ikeda, Hiroyuki, Koike, Michitaro, Lupton, Robert, Mineo, Sogo, Price, Paul, Takata, Tadafumi, Tanaka, Masayuki, Yasuda, Naoki, AlSayyad, Yusra, Becker, Andrew C., Coulton, William, Coupon, Jean, Garmilla, Jose, Huang, Song, Krughoff, K. Simon, Lang, Dustin, Leauthaud, Alexie, Lim, Kian-Tat, Lust, Nate B., MacArthur, Lauren A., Mandelbaum, Rachel, Miyatake, Hironao, Miyazaki, Satoshi, Murata, Ryoma, More, Surhud, Okura, Yuki, Owen, Russell, Swinbank, John D., Strauss, Michael A., Yamada, Yoshihiko, and Yamanoi, Hitomi. Thu . "The Hyper Suprime-Cam software pipeline". United States. doi:10.1093/pasj/psx080. https://www.osti.gov/servlets/purl/1425668.
@article{osti_1425668,
title = {The Hyper Suprime-Cam software pipeline},
author = {Bosch, James and Armstrong, Robert and Bickerton, Steven and Furusawa, Hisanori and Ikeda, Hiroyuki and Koike, Michitaro and Lupton, Robert and Mineo, Sogo and Price, Paul and Takata, Tadafumi and Tanaka, Masayuki and Yasuda, Naoki and AlSayyad, Yusra and Becker, Andrew C. and Coulton, William and Coupon, Jean and Garmilla, Jose and Huang, Song and Krughoff, K. Simon and Lang, Dustin and Leauthaud, Alexie and Lim, Kian-Tat and Lust, Nate B. and MacArthur, Lauren A. and Mandelbaum, Rachel and Miyatake, Hironao and Miyazaki, Satoshi and Murata, Ryoma and More, Surhud and Okura, Yuki and Owen, Russell and Swinbank, John D. and Strauss, Michael A. and Yamada, Yoshihiko and Yamanoi, Hitomi},
abstractNote = {Here in this article, we describe the optical imaging data processing pipeline developed for the Subaru Telescope’s Hyper Suprime-Cam (HSC) instrument. The HSC Pipeline builds on the prototype pipeline being developed by the Large Synoptic Survey Telescope’s Data Management system, adding customizations for HSC, large-scale processing capabilities, and novel algorithms that have since been reincorporated into the LSST codebase. While designed primarily to reduce HSC Subaru Strategic Program (SSP) data, it is also the recommended pipeline for reducing general-observer HSC data. The HSC pipeline includes high-level processing steps that generate coadded images and science-ready catalogs as well as low-level detrending and image characterizations.},
doi = {10.1093/pasj/psx080},
journal = {Publications of the Astronomical Society of Japan},
number = SP1,
volume = 70,
place = {United States},
year = {2017},
month = {10}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Citation Metrics:
Cited by: 65 works
Citation information provided by
Web of Science

Save / Share:

Works referenced in this record:

How to COAAD Images. II. A Coaddition Image that is Optimal for Any Purpose in the Background-dominated Noise Limit
journal, February 2017


Replacing Standard Galaxy Profiles with Mixtures of Gaussians
journal, June 2013

  • Hogg, David W.; Lang, Dustin
  • Publications of the Astronomical Society of the Pacific, Vol. 125, Issue 928
  • DOI: 10.1086/671228

Hyper Suprime-Cam: System design and verification of image quality
journal, September 2017

  • Miyazaki, Satoshi; Komiyama, Yutaka; Kawanomoto, Satoshi
  • Publications of the Astronomical Society of Japan, Vol. 70, Issue SP1
  • DOI: 10.1093/pasj/psx063

THE SLOAN DIGITAL SKY SURVEY COADD: 275 deg 2 OF DEEP SLOAN DIGITAL SKY SURVEY IMAGING ON STRIPE 82
journal, September 2014

  • Annis, James; Soares-Santos, Marcelle; Strauss, Michael A.
  • The Astrophysical Journal, Vol. 794, Issue 2
  • DOI: 10.1088/0004-637X/794/2/120

SExtractor: Software for source extraction
journal, June 1996

  • Bertin, E.; Arnouts, S.
  • Astronomy and Astrophysics Supplement Series, Vol. 117, Issue 2
  • DOI: 10.1051/aas:1996164

GalSim: The modular galaxy image simulation toolkit
journal, April 2015


The brighter-fatter effect and pixel correlations in CCD sensors
journal, March 2014


Photometry of a complete sample of faint galaxies
journal, June 1980

  • Kron, R. G.
  • The Astrophysical Journal Supplement Series, Vol. 43
  • DOI: 10.1086/190669

Toward Precision LSST Weak-Lensing Measurement. I. Impacts of Atmospheric Turbulence and Optical Aberration
journal, May 2011

  • Jee, M. James; Tyson, J. Anthony
  • Publications of the Astronomical Society of the Pacific, Vol. 123, Issue 903
  • DOI: 10.1086/660137

Characterization and correction of charge-induced pixel shifts in DECam
journal, May 2015


The bright-star masks for the HSC-SSP survey
journal, July 2017

  • Coupon, Jean; Czakon, Nicole; Bosch, James
  • Publications of the Astronomical Society of Japan, Vol. 70, Issue SP1
  • DOI: 10.1093/pasj/psx047

Hyper Suprime-Cam: Camera dewar design
journal, September 2017

  • Komiyama, Yutaka; Obuchi, Yoshiyuki; Nakaya, Hidehiko
  • Publications of the Astronomical Society of Japan, Vol. 70, Issue SP1
  • DOI: 10.1093/pasj/psx069

Photometric redshifts for Hyper Suprime-Cam Subaru Strategic Program Data Release 1
journal, October 2017

  • Tanaka, Masayuki; Coupon, Jean; Hsieh, Bau-Ching
  • Publications of the Astronomical Society of Japan, Vol. 70, Issue SP1
  • DOI: 10.1093/pasj/psx077

Weak Gravitational Lensing with COSMOS: Galaxy Selection and Shape Measurements
journal, September 2007

  • Leauthaud, Alexie; Massey, Richard; Kneib, Jean‐Paul
  • The Astrophysical Journal Supplement Series, Vol. 172, Issue 1
  • DOI: 10.1086/516598

The DES Science Verification weak lensing shear catalogues
journal, May 2016

  • Jarvis, M.; Sheldon, E.; Zuntz, J.
  • Monthly Notices of the Royal Astronomical Society, Vol. 460, Issue 2
  • DOI: 10.1093/mnras/stw990

An algorithm for precise aperture photometry of critically sampled images
journal, March 2013

  • Bickerton, S. J.; Lupton, R. H.
  • Monthly Notices of the Royal Astronomical Society, Vol. 431, Issue 2
  • DOI: 10.1093/mnras/stt244

Astrometric Calibration of the Sloan Digital Sky Survey
journal, March 2003

  • Pier, Jeffrey R.; Munn, Jeffrey A.; Hindsley, Robert B.
  • The Astronomical Journal, Vol. 125, Issue 3
  • DOI: 10.1086/346138

Fast Algorithms for Matching CCD Images to a Stellar Catalogue
journal, January 2007

  • Tabur, V.
  • Publications of the Astronomical Society of Australia, Vol. 24, Issue 4
  • DOI: 10.1071/AS07028

Bayesian galaxy shape measurement for weak lensing surveys – III. Application to the Canada–France–Hawaii Telescope Lensing Survey
journal, January 2013

  • Miller, L.; Heymans, C.; Kitching, T. D.
  • Monthly Notices of the Royal Astronomical Society, Vol. 429, Issue 4
  • DOI: 10.1093/mnras/sts454

Weak Gravitational Lensing and Its Cosmological Applications
journal, November 2008


Gravitational lensing analysis of the Kilo-Degree Survey
journal, October 2015

  • Kuijken, Konrad; Heymans, Catherine; Hildebrandt, Hendrik
  • Monthly Notices of the Royal Astronomical Society, Vol. 454, Issue 4
  • DOI: 10.1093/mnras/stv2140

The Kilo-Degree Survey
journal, August 2012

  • de Jong, Jelte T. A.; Verdoes Kleijn, Gijs A.; Kuijken, Konrad H.
  • Experimental Astronomy, Vol. 35, Issue 1-2
  • DOI: 10.1007/s10686-012-9306-1

Impact of Atmospheric Chromatic Effects on weak Lensing Measurements
journal, July 2015


Shear calibration biases in weak-lensing surveys
journal, August 2003


An implementation of Bayesian lensing shear measurement
journal, July 2014

  • Sheldon, Erin S.
  • Monthly Notices of the Royal Astronomical Society: Letters, Vol. 444, Issue 1
  • DOI: 10.1093/mnrasl/slu104

Galaxy Modeling with Compound Elliptical Shapelets
journal, August 2010


DAOPHOT - A computer program for crowded-field stellar photometry
journal, March 1987

  • Stetson, Peter B.
  • Publications of the Astronomical Society of the Pacific, Vol. 99
  • DOI: 10.1086/131977

THINK OUTSIDE THE COLOR BOX: PROBABILISTIC TARGET SELECTION AND THE SDSS - XDQSO QUASAR TARGETING CATALOG
journal, February 2011


The Second Data Release of the Sloan Digital Sky Survey
journal, July 2004

  • Abazajian, Kevork; Adelman-McCarthy, Jennifer K.; Agüeros, Marcel A.
  • The Astronomical Journal, Vol. 128, Issue 1
  • DOI: 10.1086/421365

The on-site quality-assurance system for Hyper Suprime-Cam: OSQAH†
journal, September 2017

  • Furusawa, Hisanori; Koike, Michitaro; Takata, Tadafumi
  • Publications of the Astronomical Society of Japan, Vol. 70, Issue SP1
  • DOI: 10.1093/pasj/psx079

Characterization and photometric performance of the Hyper Suprime-Cam Software Pipeline
journal, December 2017

  • Huang, Song; Leauthaud, Alexie; Murata, Ryoma
  • Publications of the Astronomical Society of Japan, Vol. 70, Issue SP1
  • DOI: 10.1093/pasj/psx126

The first-year shear catalog of the Subaru Hyper Suprime-Cam Subaru Strategic Program Survey
journal, December 2017

  • Mandelbaum, Rachel; Miyatake, Hironao; Hamana, Takashi
  • Publications of the Astronomical Society of Japan, Vol. 70, Issue SP1
  • DOI: 10.1093/pasj/psx130

    Works referencing / citing this record:

    Galaxy formation and evolution science in the era of the Large Synoptic Survey Telescope
    journal, June 2019


    The Hyper Suprime-Cam SSP transient survey in COSMOS: Overview
    journal, May 2019

    • Yasuda, Naoki; Tanaka, Masaomi; Tominaga, Nozomu
    • Publications of the Astronomical Society of Japan, Vol. 71, Issue 4
    • DOI: 10.1093/pasj/psz050

    Halo concentration, galaxy red fraction, and gas properties of optically defined merging clusters†
    journal, July 2019

    • Okabe, Nobuhiro; Oguri, Masamune; Akamatsu, Hiroki
    • Publications of the Astronomical Society of Japan, Vol. 71, Issue 4
    • DOI: 10.1093/pasj/psz059

    A Wide and Deep Exploration of Radio Galaxies with Subaru HSC (WERGS). II. Physical Properties Derived from the SED Fitting with Optical, Infrared, and Radio Data
    journal, July 2019

    • Toba, Yoshiki; Yamashita, Takuji; Nagao, Tohru
    • The Astrophysical Journal Supplement Series, Vol. 243, Issue 1
    • DOI: 10.3847/1538-4365/ab238d

    Galaxy formation and evolution science in the era of the Large Synoptic Survey Telescope
    journal, June 2019


    The Hyper Suprime-Cam SSP transient survey in COSMOS: Overview
    journal, May 2019

    • Yasuda, Naoki; Tanaka, Masaomi; Tominaga, Nozomu
    • Publications of the Astronomical Society of Japan, Vol. 71, Issue 4
    • DOI: 10.1093/pasj/psz050

    Halo concentration, galaxy red fraction, and gas properties of optically defined merging clusters†
    journal, July 2019

    • Okabe, Nobuhiro; Oguri, Masamune; Akamatsu, Hiroki
    • Publications of the Astronomical Society of Japan, Vol. 71, Issue 4
    • DOI: 10.1093/pasj/psz059

    A Wide and Deep Exploration of Radio Galaxies with Subaru HSC (WERGS). II. Physical Properties Derived from the SED Fitting with Optical, Infrared, and Radio Data
    journal, July 2019

    • Toba, Yoshiki; Yamashita, Takuji; Nagao, Tohru
    • The Astrophysical Journal Supplement Series, Vol. 243, Issue 1
    • DOI: 10.3847/1538-4365/ab238d