skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Automated transient identification in the Dark Energy Survey

Abstract

We describe an algorithm for identifying point-source transients and moving objects on reference-subtracted optical images containing artifacts of processing and instrumentation. The algorithm makes use of the supervised machine learning technique known as Random Forest. We present results from its use in the Dark Energy Survey Supernova program (DES-SN), where it was trained using a sample of 898,963 signal and background events generated by the transient detection pipeline. After reprocessing the data collected during the first DES-SN observing season (2013 September through 2014 February) using the algorithm, the number of transient candidates eligible for human scanning decreased by a factor of 13.4, while only 1.0% of the artificial Type Ia supernovae (SNe) injected into search images to monitor survey efficiency were lost, most of which were very faint events. Furthermore, we characterize the algorithm's performance in detail, and we discuss how it can inform pipeline design decisions for future time-domain imaging surveys, such as the Large Synoptic Survey Telescope and the Zwicky Transient Facility.

Authors:
 [1];  [2];  [3];  [4];  [5];  [6];  [7];  [2];  [7];  [2];  [3];  [8];  [8];  [7];  [9];  [3];  [10];  [11];  [10];  [12] more »;  [10];  [13];  [14];  [15];  [16];  [17];  [18];  [9];  [10];  [19];  [20];  [9];  [9];  [14];  [21];  [22];  [23];  [24];  [25];  [26];  [9];  [10];  [17];  [13];  [27];  [3];  [17];  [28];  [9];  [29];  [9];  [13];  [30];  [31];  [32];  [33];  [9];  [22];  [34];  [35];  [9];  [36];  [28];  [16];  [22];  [4];  [35] « less
  1. Univ. of California, Berkeley, CA (United States); Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
  2. Univ. of Portsmouth, Portsmouth (United Kingdom)
  3. Univ. of Pennsylvania, Philadelphia, PA (United States)
  4. Univ. of Illinois at Urbana-Champaign, Urbana, IL (United States)
  5. Argonne National Lab. (ANL), Argonne, IL (United States)
  6. Univ. of Chicago,Chicago, IL (United States)
  7. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
  8. Univ. of Southampton, Southampton (United Kingdom)
  9. Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
  10. Univ. College London, London (United Kingdom)
  11. Univ. of Cambridge, Cambridge (United Kingdom)
  12. Univ. Pierre et Marie Curie, and CNRS, Paris (France)
  13. Lab. Interinstitucional de e-Astronomia - LIneA, Rio de Janeiro (Brazil); Observatorio Nacional, Rio de Janieiro (Brazil)
  14. Institut de Ciencies de l'Espai, Barcelona (Spain)
  15. Lab. Interinstitucional de e-Astonomia, Rio de Janeiro (Brazil); Observatorio Nacional, Rio de Janeiro (Brazil)
  16. National Center for Supercomputing Applications, Urbana, IL (United States)
  17. Texas A & M Univ., College Station, TX (United States)
  18. Ludwig Maximilian Univ., Muenchen (Germany)
  19. Univ. of Pennsylvania, Philadelphia, PA (United States); California Inst. of Technology (CalTech), Pasadena, CA (United States)
  20. Lab. Interinstitucional de e-Astronomia LIneA, Rio de Janeiro (Brazil)
  21. Univ. of Chicago,Chicago, IL (United States); Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
  22. Univ. of Michigan, Ann Arbor, MI (United States)
  23. Max Planck Institute for Extraterrestrial Physics, Garching (Germany); Univ. Observatory Munich, Munich (Germany)
  24. Univ. of Illinois at Urbana-Champaign, Urbana, IL (United States); National Center for Supercomputing Applications, Urbana, IL (United States)
  25. National Optical Astronomy Observatory, La Serena (Chile)
  26. Austrilian Astronomical Observatory, North Ryde (Australia)
  27. Centro Brasileiro de Pesquisas Fisicas, Rio de Janeiro (Brazil)
  28. The Ohio State Univ., Columbus, OH (United States)
  29. Univ. Autonoma de Barcelona, Barcelona (Spain); Institucio Catalana de Recerca i Estudis, Barcelona (Spain)
  30. California Inst. of Technology (CalTech), Pasadena, CA (United States); Univ. of Michigan, Ann Arbor, MI (United States)
  31. Univ. of Sussex, Brighton (United Kingdom)
  32. Stanford Univ., Stanford, CA (United States); SLAC National Accelerator Lab., Menlo Park, CA (United States)
  33. Centro de Investigaciones Energeticas, Madrid (Spain)
  34. Univ. of Illinois at Urbana-Champaign, Urbana, IL (United States); Centro de Investigaciones Energeticas, Madrid (Spain)
  35. National Optical Astronomy Observatory, North Ryde (Australia)
  36. Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States); Lab. Interinstitucional de e-Astronomia LIneA, Rio de Janeiro (Brazil)
Publication Date:
Research Org.:
Brookhaven National Lab. (BNL), Upton, NY (United States); SLAC National Accelerator Lab., Menlo Park, CA (United States)
Sponsoring Org.:
USDOE Office of Science (SC), High Energy Physics (HEP) (SC-25)
OSTI Identifier:
1311819
Alternate Identifier(s):
OSTI ID: 1234374
Report Number(s):
BNL-111654-2015-JA; SLAC-PUB-16709
Journal ID: ISSN 1538-3881; KA2301020
Grant/Contract Number:
SC00112704; AC02-76SF00515
Resource Type:
Journal Article: Accepted Manuscript
Journal Name:
Astronomical Journal (Online)
Additional Journal Information:
Journal Name: Astronomical Journal (Online); Journal Volume: 150; Journal Issue: 3; Journal ID: ISSN 1538-3881
Country of Publication:
United States
Language:
English
Subject:
79 ASTRONOMY AND ASTROPHYSICS; transients; discovery, algorithms; statistical, random forest; machine learning; 97 MATHEMATICS AND COMPUTING; discovery; algorithms; statistical; random forest

Citation Formats

Goldstein, D. A., D'Andrea, C. B., Fischer, J. A., Foley, R. J., Gupta, R. R., Kessler, R., Kim, A. G., Nichol, R. C., Nungent, P., Papadopoulos, A., Sako, M., Smith, M., Sullivan, M., Thomas, R. C., Wester, W., Wolf, R. C., Abdalla, F. B., Banjeri, M., Benoit-Levy, A., Bertin, E., Brooks, D., Rosell, A. Carnero, Castander, F. J., da Costa, L. N., Covarrubias, R., DePoy, D. L., Desai, S., Diehl, H. T., Doel, P., Eifler, T. F., Neto, A. Fausti, Finley, D. A., Flaugher, B., Fosalba, P., Frieman, J., Gerdes, D., Gruen, D., Gruendl, R. A., James, D., Kuehn, K., Kuropatkin, N., Lahav, O., Li, T. S., Maia, M. A. G., Makler, M., March, M., Marshall, J. L., Martini, P., Merritt, K. W., Miquel, R., Nord, B., Ogando, R., Plazas, A. A., Romer, A. K., Roodman, A., Sanchez, E., Scarpine, V., Schubnell, M., Sevilla-Noarbe, I., Smith, R. C., Soares-Santos, M., Sobreira, F., Suchyta, E., Swanson, M. E. C., Tarle, G., Thaler, J., and Walker, A. R. Automated transient identification in the Dark Energy Survey. United States: N. p., 2015. Web. doi:10.1088/0004-6256/150/3/82.
Goldstein, D. A., D'Andrea, C. B., Fischer, J. A., Foley, R. J., Gupta, R. R., Kessler, R., Kim, A. G., Nichol, R. C., Nungent, P., Papadopoulos, A., Sako, M., Smith, M., Sullivan, M., Thomas, R. C., Wester, W., Wolf, R. C., Abdalla, F. B., Banjeri, M., Benoit-Levy, A., Bertin, E., Brooks, D., Rosell, A. Carnero, Castander, F. J., da Costa, L. N., Covarrubias, R., DePoy, D. L., Desai, S., Diehl, H. T., Doel, P., Eifler, T. F., Neto, A. Fausti, Finley, D. A., Flaugher, B., Fosalba, P., Frieman, J., Gerdes, D., Gruen, D., Gruendl, R. A., James, D., Kuehn, K., Kuropatkin, N., Lahav, O., Li, T. S., Maia, M. A. G., Makler, M., March, M., Marshall, J. L., Martini, P., Merritt, K. W., Miquel, R., Nord, B., Ogando, R., Plazas, A. A., Romer, A. K., Roodman, A., Sanchez, E., Scarpine, V., Schubnell, M., Sevilla-Noarbe, I., Smith, R. C., Soares-Santos, M., Sobreira, F., Suchyta, E., Swanson, M. E. C., Tarle, G., Thaler, J., & Walker, A. R. Automated transient identification in the Dark Energy Survey. United States. doi:10.1088/0004-6256/150/3/82.
Goldstein, D. A., D'Andrea, C. B., Fischer, J. A., Foley, R. J., Gupta, R. R., Kessler, R., Kim, A. G., Nichol, R. C., Nungent, P., Papadopoulos, A., Sako, M., Smith, M., Sullivan, M., Thomas, R. C., Wester, W., Wolf, R. C., Abdalla, F. B., Banjeri, M., Benoit-Levy, A., Bertin, E., Brooks, D., Rosell, A. Carnero, Castander, F. J., da Costa, L. N., Covarrubias, R., DePoy, D. L., Desai, S., Diehl, H. T., Doel, P., Eifler, T. F., Neto, A. Fausti, Finley, D. A., Flaugher, B., Fosalba, P., Frieman, J., Gerdes, D., Gruen, D., Gruendl, R. A., James, D., Kuehn, K., Kuropatkin, N., Lahav, O., Li, T. S., Maia, M. A. G., Makler, M., March, M., Marshall, J. L., Martini, P., Merritt, K. W., Miquel, R., Nord, B., Ogando, R., Plazas, A. A., Romer, A. K., Roodman, A., Sanchez, E., Scarpine, V., Schubnell, M., Sevilla-Noarbe, I., Smith, R. C., Soares-Santos, M., Sobreira, F., Suchyta, E., Swanson, M. E. C., Tarle, G., Thaler, J., and Walker, A. R. Tue . "Automated transient identification in the Dark Energy Survey". United States. doi:10.1088/0004-6256/150/3/82. https://www.osti.gov/servlets/purl/1311819.
@article{osti_1311819,
title = {Automated transient identification in the Dark Energy Survey},
author = {Goldstein, D. A. and D'Andrea, C. B. and Fischer, J. A. and Foley, R. J. and Gupta, R. R. and Kessler, R. and Kim, A. G. and Nichol, R. C. and Nungent, P. and Papadopoulos, A. and Sako, M. and Smith, M. and Sullivan, M. and Thomas, R. C. and Wester, W. and Wolf, R. C. and Abdalla, F. B. and Banjeri, M. and Benoit-Levy, A. and Bertin, E. and Brooks, D. and Rosell, A. Carnero and Castander, F. J. and da Costa, L. N. and Covarrubias, R. and DePoy, D. L. and Desai, S. and Diehl, H. T. and Doel, P. and Eifler, T. F. and Neto, A. Fausti and Finley, D. A. and Flaugher, B. and Fosalba, P. and Frieman, J. and Gerdes, D. and Gruen, D. and Gruendl, R. A. and James, D. and Kuehn, K. and Kuropatkin, N. and Lahav, O. and Li, T. S. and Maia, M. A. G. and Makler, M. and March, M. and Marshall, J. L. and Martini, P. and Merritt, K. W. and Miquel, R. and Nord, B. and Ogando, R. and Plazas, A. A. and Romer, A. K. and Roodman, A. and Sanchez, E. and Scarpine, V. and Schubnell, M. and Sevilla-Noarbe, I. and Smith, R. C. and Soares-Santos, M. and Sobreira, F. and Suchyta, E. and Swanson, M. E. C. and Tarle, G. and Thaler, J. and Walker, A. R.},
abstractNote = {We describe an algorithm for identifying point-source transients and moving objects on reference-subtracted optical images containing artifacts of processing and instrumentation. The algorithm makes use of the supervised machine learning technique known as Random Forest. We present results from its use in the Dark Energy Survey Supernova program (DES-SN), where it was trained using a sample of 898,963 signal and background events generated by the transient detection pipeline. After reprocessing the data collected during the first DES-SN observing season (2013 September through 2014 February) using the algorithm, the number of transient candidates eligible for human scanning decreased by a factor of 13.4, while only 1.0% of the artificial Type Ia supernovae (SNe) injected into search images to monitor survey efficiency were lost, most of which were very faint events. Furthermore, we characterize the algorithm's performance in detail, and we discuss how it can inform pipeline design decisions for future time-domain imaging surveys, such as the Large Synoptic Survey Telescope and the Zwicky Transient Facility.},
doi = {10.1088/0004-6256/150/3/82},
journal = {Astronomical Journal (Online)},
number = 3,
volume = 150,
place = {United States},
year = {Tue Sep 01 00:00:00 EDT 2015},
month = {Tue Sep 01 00:00:00 EDT 2015}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Citation Metrics:
Cited by: 29works
Citation information provided by
Web of Science

Save / Share:
  • We describe an algorithm for identifying point-source transients and moving objects on reference-subtracted optical images containing artifacts of processing and instrumentation. The algorithm makes use of the supervised machine learning technique known as Random Forest. We present results from its use in the Dark Energy Survey Supernova program (DES-SN), where it was trained using a sample of 898,963 signal and background events generated by the transient detection pipeline. After reprocessing the data collected during the first DES-SN observing season (2013 September through 2014 February) using the algorithm, the number of transient candidates eligible for human scanning decreased by a factormore » of 13.4, while only 1.0 percent of the artificial Type Ia supernovae (SNe) injected into search images to monitor survey efficiency were lost, most of which were very faint events. Here we characterize the algorithm's performance in detail, and we discuss how it can inform pipeline design decisions for future time-domain imaging surveys, such as the Large Synoptic Survey Telescope and the Zwicky Transient Facility.« less
  • We describe an algorithm for identifying point-source transients and moving objects on reference-subtracted optical images containing artifacts of processing and instrumentation. The algorithm makes use of the supervised machine learning technique known as Random Forest. We present results from its use in the Dark Energy Survey Supernova program (DES-SN), where it was trained using a sample of 898,963 signal and background events generated by the transient detection pipeline. After reprocessing the data collected during the first DES-SN observing season (2013 September through 2014 February) using the algorithm, the number of transient candidates eligible for human scanning decreased by a factormore » of 13.4, while only 1.0% of the artificial Type Ia supernovae (SNe) injected into search images to monitor survey efficiency were lost, most of which were very faint events. Here we characterize the algorithm's performance in detail, and we discuss how it can inform pipeline design decisions for future time-domain imaging surveys, such as the Large Synoptic Survey Telescope and the Zwicky Transient Facility. An implementation of the algorithm and the training data used in this paper are available at http://portal.nserc.gov/project/dessn/autoscan.« less
  • We describe an algorithm for identifying point-source transients and moving objects on reference-subtracted optical images containing artifacts of processing and instrumentation. The algorithm makes use of the supervised machine learning technique known as Random Forest. We present results from its use in the Dark Energy Survey Supernova program (DES-SN), where it was trained using a sample of 898,963 signal and background events generated by the transient detection pipeline. After reprocessing the data collected during the first DES-SN observing season (2013 September through 2014 February) using the algorithm, the number of transient candidates eligible for human scanning decreased by a factormore » of 13.4, while only 1.0% of the artificial Type Ia supernovae (SNe) injected into search images to monitor survey efficiency were lost, most of which were very faint events. Here we characterize the algorithm’s performance in detail, and we discuss how it can inform pipeline design decisions for future time-domain imaging surveys, such as the Large Synoptic Survey Telescope and the Zwicky Transient Facility. An implementation of the algorithm and the training data used in this paper are available at at http://portal.nersc.gov/project/dessn/autoscan.« less
  • We describe an algorithm for identifying point-source transients and moving objects on reference-subtracted optical images containing artifacts of processing and instrumentation. The algorithm makes use of the supervised machine learning technique known as Random Forest. We present results from its use in the Dark Energy Survey Supernova program (DES-SN), where it was trained using a sample of 898,963 signal and background events generated by the transient detection pipeline. After reprocessing the data collected during the first DES-SN observing season (2013 September through 2014 February) using the algorithm, the number of transient candidates eligible for human scanning decreased by a factormore » of 13.4, while only 1.0% of the artificial Type Ia supernovae (SNe) injected into search images to monitor survey efficiency were lost, most of which were very faint events. Furthermore, we characterize the algorithm's performance in detail, and we discuss how it can inform pipeline design decisions for future time-domain imaging surveys, such as the Large Synoptic Survey Telescope and the Zwicky Transient Facility.« less
  • We describe the operation and performance of the difference imaging pipeline (DiffImg) used to detect transients in deep images from the Dark Energy Survey Supernova program (DES-SN) in its first observing season from 2013 August through 2014 February. DES-SN is a search for transients in which ten 3 deg(2) fields are repeatedly observed in the g, r, i, z passbands with a cadence of about 1 week. The observing strategy has been optimized to measure high-quality light curves and redshifts for thousands of Type Ia supernovae (SNe Ia) with the goal of measuring dark energy parameters. The essential DiffImg functionsmore » are to align each search image to a deep reference image, do a pixel-by-pixel subtraction, and then examine the subtracted image for significant positive detections of point-source objects. The vast majority of detections are subtraction artifacts, but after selection requirements and image filtering with an automated scanning program, there are similar to 130 detections per deg(2) per observation in each band, of which only similar to 25% are artifacts. Of the similar to 7500 transients discovered by DES-SN in its first observing season, each requiring a detection on at least two separate nights, Monte Carlo (MC) simulations predict that 27% are expected to be SNe Ia or core-collapse SNe. Another similar to 30% of the transients are artifacts in which a small number of observations satisfy the selection criteria for a single-epoch detection. Spectroscopic analysis shows that most of the remaining transients are AGNs and variable stars. Fake SNe Ia are overlaid onto the images to rigorously evaluate detection efficiencies and to understand the DiffImg performance. The DiffImg efficiency measured with fake SNe agrees well with expectations from a MC simulation that uses analytical calculations of the fluxes and their uncertainties. In our 8 "shallow" fields with single-epoch 50% completeness depth similar to 23.5, the SN Ia efficiency falls to 1/2 at redshift z approximate to 0.7; in our 2 "deep" fields with mag-depth similar to 24.5, the efficiency falls to 1/2 at z approximate to 1.1. A remaining performance issue is that the measured fluxes have additional scatter (beyond Poisson fluctuations) that increases with the host galaxy surface brightness at the transient location. This bright-galaxy issue has minimal impact on the SNe Ia program, but it may lower the efficiency for finding fainter transients on bright galaxies.« less