Search Menu
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information
Search Scholarly Publications

Title: Machine Learning for the Zwicky Transient Facility

Journal Article · · Publications of the Astronomical Society of the Pacific
ORCiD logo [1];  [2];  [3];  [1];  [1];  [4];  [1];  [5];  [1];  [6];  [1];  [7];  [8];  [9];  [10];  [1];  [7];  [11];  [2];  [1] more »;  [7];  [3];  [5];  [10];  [12];  [10];  [1];  [10];  [1];  [1];  [1];  [1];  [13];  [14];  [15];  [5];  [7];  [1];  [3];  [1];  [7];  [16];  [1];  [17];  [18];  [7];  [19];  [1];  [10];  [12] « less
  1. California Inst. of Technology (CalTech), Pasadena, CA (United States)
  2. California Inst. of Technology (CalTech), Pasadena, CA (United States). Jet Propulsion Lab. (JPL)
  3. California Inst. of Technology (CalTech), Pasadena, CA (United States). Caltech Optical Observatories
  4. Radboud Univ., Nijmegen (Netherlands)
  5. Stockholm Univ. (Sweden)
  6. National Central Univ. (Taiwan)
  7. Univ. of Washington, Seattle, WA (United States)
  8. Northwestern Univ., Evanston, IL (United States); The Adler Planetarium, Chicago, IL (United States)
  9. Humboldt Univ. of Berlin (Germany)
  10. Univ. of Maryland, College Park, MD (United States)
  11. Space Telescope Science Inst., Baltimore, MD (United States)
  12. Univ. of Oxford (United Kingdom); Univ. of Minnesota, Minneapolis, MN (United States)
  13. California Inst. of Technology (CalTech), Pasadena, CA (United States); Univ. of California, Santa Barbara, CA (United States)
  14. National Central Univ. (Taiwan); Univ. of Michigan, Ann Arbor, MI (United States)
  15. Univ. of Oxford (United Kingdom)
  16. Univ. of Freiburg (Germany)
  17. NASA Goddard Space Flight Center (GSFC), Greenbelt, MD (United States); Univ. of Maryland, College Park, MD (United States). Joint Space-Science Inst.
  18. Weizmann Inst. of Science, Rehovot (Israel)
  19. California Inst. of Technology (CalTech), Pasadena, CA (United States); Tokyo Inst. of Technology, Meguro-ku, Tokyo (Japan)
+ Show Author Affiliations

The Zwicky Transient Facility is a large optical survey in multiple filters producing hundreds of thousands of transient alerts per night. We describe here various machine learning (ML) implementations and plans to make the maximal use of the large data set by taking advantage of the temporal nature of the data, and further combining it with other data sets. We start with the initial steps of separating bogus candidates from real ones, separating stars and galaxies, and go on to the classification of real objects into various classes. Besides the usual methods (e.g., based on features extracted from light curves) we also describe early plans for alternate methods including the use of domain adaptation, and deep learning. In a similar fashion we describe efforts to detect fast moving asteroids. We also describe the use of the Zooniverse platform for helping with classifications through the creation of training samples, and active learning. Finally we mention the synergistic aspects of ZTF and LSST from the ML perspective.

Research Organization:
Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)
Sponsoring Organization:
USDOE Office of Science (SC); National Science Foundation (NSF); Weizmann Institute of Science; National Aeronautics and Space Administration (NASA)
Grant/Contract Number:
AC02-05CH11231
OSTI ID:
1580382
Journal Information:
Publications of the Astronomical Society of the Pacific, Journal Name: Publications of the Astronomical Society of the Pacific Journal Issue: 997 Vol. 131; ISSN 0004-6280
Publisher:
Astronomical Society of the Pacific (ASP)Copyright Statement
Country of Publication:
United States
Language:
English

References (90)

The Palomar-Quest digital synoptic sky survey journal March 2008
Towards a real-time transient classification engine journal March 2008
Modeling lightcurves for improved classification of astronomical objects: Modeling Lightcurves
  • Faraway, Julian; Mahabal, Ashish; Sun, Jiayang
  • Statistical Analysis and Data Mining: The ASA Data Science Journal, Vol. 9, Issue 1 https://doi.org/10.1002/sam.11305
journal February 2016
Least-squares frequency analysis of unequally spaced data journal February 1976
Extremely randomized trees journal March 2006
Distributed asteroid discovery system for large astronomical data journal September 2017
Random Forests journal January 2001
Deep learning journal May 2015
A recurrent neural network for classification of unevenly sampled variable stars journal November 2017
miRNALoc: predicting miRNA subcellular localizations based on principal component scores of physico-chemical properties and pseudo compositions of di-nucleotides journal September 2020
A package for the automated classification of periodic variable stars journal February 2016
Gaia Data Release 1 : Summary of the astrometric, photometric, and survey properties journal November 2016
Gaia Data Release 1 : Testing parallaxes with local Cepheids and RR Lyrae stars⋆ journal September 2017
Gaia Data Release 2: Processing of the photometric data journal August 2018
Gaia Data Release 2 : The celestial reference frame ( journal August 2018
Gaia Data Release 2 : Summary of the contents and survey properties journal August 2018
Gaia Data Release 2: Short-timescale variability processing and analysis journal December 2018
Gaia Data Release 2: The first Gaia catalogue of long-period variable candidates journal October 2018
A box-fitting algorithm in the search for periodic transits journal July 2002
Studies in astronomical time series analysis. II - Statistical aspects of spectral analysis of unevenly spaced data journal December 1982
Weak Gravitational Lensing with COSMOS: Galaxy Selection and Shape Measurements
  • Leauthaud, Alexie; Massey, Richard; Kneib, Jean‐Paul
  • The Astrophysical Journal Supplement Series, Vol. 172, Issue 1 https://doi.org/10.1086/516598
journal September 2007
How to Find More Supernovae with Less Work: Object Classification Techniques for Difference Imaging journal August 2007
The Palomar Transient Factory: System Overview, Performance, and First Results
  • Law, Nicholas M.; Kulkarni, Shrinivas R.; Dekany, Richard G.
  • Publications of the Astronomical Society of the Pacific, Vol. 121, Issue 886 https://doi.org/10.1086/648598
journal December 2009
Improved Asteroid Astrometry and Photometry with Trail Fitting
  • Vereš, Peter; Jedicke, Robert; Denneau, Larry
  • Publications of the Astronomical Society of the Pacific, Vol. 124, Issue 921 https://doi.org/10.1086/668616
journal November 2012
Asteroid Light Curves from the Palomar Transient Factory Survey: Rotation Periods and Phase Functions from Sparse Photometry journal August 2015
Automated Transient Identification in the dark Energy Survey journal August 2015
First Results from the Catalina Real-Time Transient Survey journal April 2009
On Machine-Learned Classification of Variable Stars with Sparse and Noisy Time-Series data journal April 2011
Active Learning to Overcome Sample Selection Bias: Application to Photometric Variable star Classification journal December 2011
313 new Asteroid Rotation Periods from Palomar Transient Factory Observations journal May 2014
The man Behind the Curtain: X-Rays Drive the uv Through nir Variability in the 2013 Active Galactic Nucleus Outburst in ngc 2617 journal May 2014
The Catalina Surveys Periodic Variable star Catalog journal June 2014
Asteroid Spin-Rate Study Using the Intermediate Palomar Transient Factory journal August 2015
Volumetric Survey Speed: A Figure of Merit for Transient Surveys journal June 2016
Small Near-Earth Asteroids in the Palomar Transient Factory Survey: A Real-Time Streak-detection System journal February 2017
ATLAS: A High-cadence All-sky Survey System journal May 2018
A Morphological Classification Model to Identify Unresolved PanSTARRS1 Sources: Application in the ZTF Real-time Pipeline journal November 2018
The Zwicky Transient Facility: Data Processing, Products, and Archive journal December 2018
The Zwicky Transient Facility Alert Distribution System journal November 2018
The Zwicky Transient Facility: System Overview, Performance, and First Results journal December 2018
On the advantage of using analysis of variance for period search journal November 1989
Using conditional entropy to identify periodicity journal July 2013
Machine learning for transient discovery in Pan-STARRS1 difference imaging journal March 2015
Comparative performance of selected variability detection techniques in photometric time series data journal September 2016
Machine learning search for variable stars journal December 2017
Optimizing spectroscopic follow-up strategies for supernova photometric classification with active learning journal November 2018
Effective image differencing with convolutional neural networks for real-time transient hunting journal April 2018
The ASAS-SN catalogue of variable stars I: The Serendipitous Survey journal April 2018
Active learning* journal January 2007
Galaxy Zoo: morphologies derived from visual inspection of galaxies from the Sloan Digital Sky Survey ★ journal September 2008
ANTARES: a prototype transient broker system conference July 2014
Overview of the Nearby Supernova Factory conference December 2002
Pan-STARRS: a wide-field optical survey telescope array conference September 2004
XGBoost: A Scalable Tree Boosting System conference January 2016
Active Learning journal June 2012
The Zwicky Transient Facility: System Overview, Performance, and First Results text January 2019
Proper Image Subtraction—Optimal Transient Detection, Photometry, and Hypothesis Testing journal October 2016
Preparing for Advanced Ligo: a Star–Galaxy Separation Catalog for the Palomar Transient Factory journal January 2017
A First Catalog of Variable Stars Measured by the Asteroid Terrestrial-impact Last Alert System (ATLAS) journal November 2018
RATIR Follow-up of LIGO/Virgo Gravitational Wave Events journal April 2018
Machine-learning-based Brokers for Real-time Classification of the LSST Alert Stream journal May 2018
Deep Recurrent Neural Networks for Supernovae Classification journal March 2017
Gaia Data Release 2. Summary of the contents and survey properties text January 2018
How to Find More Supernovae with Less Work: Object Classification Techniques for Difference Imaging text January 2007
Towards a Real-time Transient Classification Engine text January 2008
Galaxy Zoo : Morphologies derived from visual inspection of galaxies from the Sloan Digital Sky Survey text January 2008
First Results from the Catalina Real-time Transient Survey text January 2008
The Palomar Transient Factory: System Overview, Performance and First Results text January 2009
On Machine-Learned Classification of Variable Stars with Sparse and Noisy Time-Series Data text January 2011
Active Learning to Overcome Sample Selection Bias: Application to Photometric Variable Star Classification text January 2011
313 new asteroid rotation periods from Palomar Transient Factory observations text January 2014
The Catalina Surveys Periodic Variable Star Catalog text January 2014
ANTARES: A Prototype Transient Broker System text January 2014
Machine learning for transient discovery in Pan-STARRS1 difference imaging text January 2015
Automated Transient Identification in the Dark Energy Survey text January 2015
Asteroid lightcurves from the Palomar Transient Factory survey: Rotation periods and phase functions from sparse photometry text January 2015
A Package for the Automated Classification of Periodic Variable Stars text January 2015
Proper image subtraction - optimal transient detection, photometry and hypothesis testing text January 2016
Volumetric Survey Speed: A Figure of Merit for Transient Surveys text January 2016
Deep Recurrent Neural Networks for Supernovae Classification text January 2016
Small near-Earth asteroids in the Palomar Transient Factory survey: A real-time streak-detection system text January 2016
RATIR Followup of LIGO/Virgo Gravitational Wave Events text January 2017
Machine Learning-based Brokers for Real-time Classification of the LSST Alert Stream text January 2018
The ASAS-SN Catalog of Variable Stars I: The Serendipitous Survey text January 2018
Optimizing spectroscopic follow-up strategies for supernova photometric classification with active learning text January 2018
The Zwicky Transient Facility: Data Processing, Products, and Archive text January 2019
The Zwicky Transient Facility: System Overview, Performance, and First Results text January 2019
The Zwicky Transient Facility Alert Distribution System text January 2019
A box-fitting algorithm in the search for periodic transits text January 2002
Deep Learning text January 2018

Cited By (17)

Surveying the reach and maturity of machine learning and artificial intelligence in astronomy journal September 2019
Transient processing and analysis using AMPEL: alert management, photometry, and evaluation of light curves journal November 2019
2900 Square Degree Search for the Optical Counterpart of Short Gamma-Ray Burst GRB 180523B with the Zwicky Transient Facility journal February 2019
Toward Efficient Detection of Small Near-Earth Asteroids Using the Zwicky Transient Facility (ZTF) journal May 2019
The Zwicky Transient Facility: Observing System journal January 2020
Palomar Gattini-IR: Survey Overview, Data Processing System, On-sky Performance and First Results journal January 2020
Transient processing and analysis using AMPEL: alert management, photometry, and evaluation of light curves text January 2019
GROWTH on S190425z: Searching Thousands of Square Degrees to Identify an Optical or Infrared Counterpart to a Binary Neutron Star Merger with the Zwicky Transient Facility and Palomar Gattini-IR text January 2019
A New Class of Changing-look LINERs journal September 2019
Evidence for Late-stage Eruptive Mass Loss in the Progenitor to SN2018gep, a Broad-lined Ic Supernova: Pre-explosion Emission and a Rapidly Rising Luminous Transient journal December 2019
Discovery of an Intermediate-luminosity Red Transient in M51 and Its Likely Dust-obscured, Infrared-variable Progenitor journal July 2019
GROWTH on S190425z: Searching Thousands of Square Degrees to Identify an Optical or Infrared Counterpart to a Binary Neutron Star Merger with the Zwicky Transient Facility and Palomar Gattini-IR journal October 2019
2900 square degree search for the optical counterpart of short gamma-ray burst GRB 180523B with the Zwicky Transient Facility text January 2019
Transient processing and analysis using $\texttt{AMPEL}$: Alert Management, Photometry and Evaluation of Lightcurves text January 2019
Discovery of an intermediate-luminosity red transient in M51 and its likely dust-obscured, infrared-variable progenitor text January 2019
Evidence for Late-stage Eruptive Mass-loss in the Progenitor to SN2018gep, a Broad-lined Ic Supernova: Pre-explosion Emission and a Rapidly Rising Luminous Transient text January 2019
Palomar Gattini-IR: Survey overview, data processing system, on-sky performance and first results text January 2019

Similar Records

The Zwicky Transient Facility: Science Objectives
Journal Article · 2019 · Publications of the Astronomical Society of the Pacific · OSTI ID:1567168
+117 more
Anomaly detection in the Zwicky Transient Facility DR3
Journal Article · 2021 · Monthly Notices of the Royal Astronomical Society · OSTI ID:1825075
+11 more
The Zwicky Transient Facility: System Overview, Performance, and First Results
Journal Article · 2018 · Publications of the Astronomical Society of the Pacific · OSTI ID:1494108
+113 more

Related Subjects

79 ASTRONOMY AND ASTROPHYSICS
Machine Learning
Sky Surveys
Time Domain