Developing Robust Digital Twins and Reinforcement Learning for Accelerator Control Systems at the Fermilab Booster

Conference ·

DOI:https://doi.org/10.18429/JACoW-IPAC2021-TUPAB327· OSTI ID:1828299

Kafkes, D. L. ^[1]; Schram, M. ^[2]

---

1. Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
2. Thomas Jefferson National Accelerator Facility (TJNAF), Newport News, VA (United States)

We describe the offline machine learning (ML) development for an effort to precisely regulate the Gradient Magnet Power Supply (GMPS) at the Fermilab Booster accelerator complex via a Field-Programmable Gate Array (FPGA). As part of this effort, we created a digital twin of the Booster-GMPS control system by training a Long Short-Term Memory (LSTM) to capture its full dynamics. We outline the path we took to carefully validate our digital twin before deploying it as a reinforcement learning (RL) environment. Additionally, we demonstrate the use of a Deep Q-Network (DQN) policy model with the capability to regulate the GMPS against realistic time-varying perturbations.

• Research Organization: Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States); Thomas Jefferson National Accelerator Facility (TJNAF), Newport News, VA (United States)
• Sponsoring Organization: USDOE Office of Science (SC), Nuclear Physics (NP)
• DOE Contract Number: FNAL-LDRD-2019-027: Accelerator Control with Artificial Intelligence
• OSTI ID: 1828299
• Report Number(s): JLAB-CST-21-3502; DOE/OR/23177-5351
• Resource Relation: Conference: IPAC 21, 24-28 May 2021, Campinas, SP, Brazil
• Country of Publication: United States
• Language: English