AI Data Quality Monitoring with Hydra

Conference · Sat Oct 01 00:00:00 EDT 2022

DOI:https://doi.org/10.2172/1998578· OSTI ID:1998578

Britton, Thomas; Lawrence, David; Jeske, Torri; Rajput, Kishansingh

Hydra is an extensible framework for training and managing AI for near real time monitoring that aims to replace the tedious and repetitive data quality monitoring activities the shift crew and online monitoring coordinator typically perform. It continuously scans incoming data in the form of monitoring plots for signs of problems, flagging them for human review. A web app was developed such that experts can efficiently label images for training. Labels are stored in a database for use in training and model validation. Backed up by a comprehensive database, it utilizes an additional web based front-end for viewing the current monitoring status from anywhere in the world. The system has been in production use for the GlueX experiment at Jefferson Lab for more than 2 years with new features still under active development.

• Research Organization: Thomas Jefferson National Accelerator Facility, Newport News, VA (United States)
• Sponsoring Organization: USDOE Office of Science (SC), Nuclear Physics (NP)
• DOE Contract Number: AC05-06OR23177
• OSTI ID: 1998578
• Report Number(s): JLAB-CST-23-3918; DOE/OR/23177-7112
• Country of Publication: United States
• Language: English