DMTN-219: Proposal and Prototype for Prompt Processing

Lim, Kian-Tat

doi:10.71929/rubin/2585429

Title: DMTN-219: Proposal and Prototype for Prompt Processing

Dataset
Other Related Research

Abstract

The Prompt Processing framework has long been a gray area in the overall Rubin Observatory architecture. This document attempts to fill that hole by proposing a concrete design with an accompanying prototype implementation that should meet minimum requirements. Additional features and optimizations that could be incorporated later are also described.

Authors:

SLAC National Accelerator Laboratory

Publication Date:: Mon Jul 10 00:00:00 EDT 2023

Other Number(s):: Vera C. Rubin Observatory Data Management Technical Note DMTN-219

DOE Contract Number:: AC02-76SF00515

Research Org.:: NSF-DOE Vera C. Rubin Observatory

Sponsoring Org.:: U.S. National Science Foundation; U.S. Department of Energy Office of Science

Subject:: 79 ASTRONOMY AND ASTROPHYSICS

OSTI Identifier:: 2585429

DOI:: https://doi.org/10.71929/rubin/2585429

Citation Formats


                    Lim, Kian-Tat. DMTN-219: Proposal and Prototype for Prompt Processing.  United States: N. p., 2023. 
        Web.  doi:10.71929/rubin/2585429.

Copy to clipboard


                    Lim, Kian-Tat. DMTN-219: Proposal and Prototype for Prompt Processing.  United States.  doi:https://doi.org/10.71929/rubin/2585429

Copy to clipboard


                    Lim, Kian-Tat. 2023.  
"DMTN-219: Proposal and Prototype for Prompt Processing".  United States.  doi:https://doi.org/10.71929/rubin/2585429.  https://www.osti.gov/servlets/purl/2585429. Pub date:Mon Jul 10 00:00:00 EDT 2023

Copy to clipboard


                    
@article{osti_2585429,

  title        = {DMTN-219: Proposal and Prototype for Prompt Processing},

  author       = {Lim, Kian-Tat},

  abstractNote = {The Prompt Processing framework has long been a gray area in the overall Rubin Observatory architecture. This document attempts to fill that hole by proposing a concrete design with an accompanying prototype implementation that should meet minimum requirements. Additional features and optimizations that could be incorporated later are also described.},

  doi          = {10.71929/rubin/2585429},

  journal      = {},

  number       = ,

  volume       = ,

  place        = {United States},

  year         = {Mon Jul 10 00:00:00 EDT 2023},

  month        = {Mon Jul 10 00:00:00 EDT 2023}

}

Copy to clipboard

Dataset:

View Dataset

DOI: https://doi.org/10.71929/rubin/2585429

Save / Share:

Export Metadata

Save to My Library

Similar records in DOE Data Explorer and OSTI.GOV collections:

DMTN-260: Failure Modes and Error Handling for Prompt Processing

Dataset Findeisen, Krzysztof ; Lim, Kian-Tat ; Bellm, Eric C. ; ...

The Prompt Processing system will be responsible for processing roughly a thousand visits per night, and distributing the results in near real time, for at least ten years of Rubin Observatory operations. As such, it must be highly robust to algorithmic, network, and infrastructure failures, ranging from momentary glitches to extended downtimes. DMTN-219 introduced the initial design for the Prompt Processing framework; this document expands on the design to address expected failure modes and recovery strategies for each.
DMTN-197: Streak Masking in DM Image Processing

Dataset Saunders, Clare

Streaks caused by satellites are a persistent problem in optical images, and their occurrence will likely increase in the coming years. Many Hyper Suprime-Cam images are already affected by streaks, and the same is expected for Rubin Observatory data. In the LSST Data Release Production (DRP), most streaks and other artifacts are already detected and masked by the algorithm CompareWarpAssembleCoadd. However, some streaks are not caught at this stage and can contaminate the final coadds and detection catalogs. To find and mask these, we adopt a morphologically-based method for detecting streaks, which uses the Kernel-Based Hough Transform to detect straightmore »« less
DMTN-015: Flavors of Coadds

Dataset Bosch, James F.

A glossary of different kinds of coadded images, with brief descriptions of the algorithms behind them.
DMTN-118: Review of Timeseries Features

Dataset Bellm, Eric C.

Rubin Observatory will compute timeseries variability features on lightcurves to aid users in identifying objects of interest, both during realtime Alert Production as well as in the annual Data Releases. The Data Products Definition Document (DPDD; LSE-163) allocates space for pre-computed timeseries features, and a sample set is baselined in LDM-151. However, in the subsequent decade the scientfic community has made a great deal of further progress in this area. This technote reviews the relevant literature, grouping related features where possible; discusses potential concerns and open questions; and proposes a new baseline feature set.
DMTN-171: Fall 2020 status of crowded field processing with the LSST Alert Production Pipelines

Dataset Sullivan, Ian S. ; Bellm, Eric C.

A report on the Fall 2020 state of the LSST Alert Production Pipelines' ability to process deep images in the galactic bulge.

Similar Records