Real-time artificial intelligence for accelerator control: A study at the Fermilab Booster
Abstract
We describe a method for precisely regulating the gradient magnet power supply (GMPS) at the Fermilab Booster accelerator complex using a neural network trained via reinforcement learning. We demonstrate preliminary results by training a surrogate machine-learning model on real accelerator data to emulate the GMPS, and using this surrogate model in turn to train the neural network for its regulation task. We additionally show how the neural networks to be deployed for control purposes may be compiled to execute on field-programmable gate arrays (FPGAs), and show the first machine-learning based control algorithm implemented on an FPGA for controls at the Fermilab accelerator complex. As there are no surprise latencies on an FPGA, this capability is important for operational stability in complicated environments such as an accelerator facility.
- Publication Date:
- Research Org.:
- Pacific Northwest National Lab. (PNNL), Richland, WA (United States); Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States); Thomas Jefferson National Accelerator Facility (TJNAF), Newport News, VA (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC), High Energy Physics (HEP); USDOE Laboratory Directed Research and Development (LDRD) Program; National Science Foundation (NSF)
- OSTI Identifier:
- 1826394
- Alternate Identifier(s):
- OSTI ID: 1826740; OSTI ID: 1828840; OSTI ID: 1886597
- Report Number(s):
- FERMILAB-PUB-20-565-AD-E-QIS-SCD; arXiv:2011.07371; PNNL-SA-157642; JLAB-CST-21-3606; DOE/OR/23177-5611
Journal ID: ISSN 2469-9888; PRABCJ; 104601
- Grant/Contract Number:
- AC02-07CH11359; SC0021187; FNAL-LDRD-2019-027; AC05-76RL01830; DGE-1644869
- Resource Type:
- Published Article
- Journal Name:
- Physical Review Accelerators and Beams
- Additional Journal Information:
- Journal Name: Physical Review Accelerators and Beams Journal Volume: 24 Journal Issue: 10; Journal ID: ISSN 2469-9888
- Publisher:
- American Physical Society
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 43 PARTICLE ACCELERATORS; artificial intelligence; machine learning; accelerator control; FPGA; reinforcement learning; neural network; artification intelligence; surrogate model; LSTM; DQN
Citation Formats
St. John, Jason, Herwig, Christian, Kafkes, Diana, Mitrevski, Jovan, Pellico, William A., Perdue, Gabriel N., Quintero-Parra, Andres, Schupbach, Brian A., Seiya, Kiyomi, Tran, Nhan, Schram, Malachi, Duarte, Javier M., Huang, Yunzhi, and Keller, Rachael. Real-time artificial intelligence for accelerator control: A study at the Fermilab Booster. United States: N. p., 2021.
Web. doi:10.1103/PhysRevAccelBeams.24.104601.
St. John, Jason, Herwig, Christian, Kafkes, Diana, Mitrevski, Jovan, Pellico, William A., Perdue, Gabriel N., Quintero-Parra, Andres, Schupbach, Brian A., Seiya, Kiyomi, Tran, Nhan, Schram, Malachi, Duarte, Javier M., Huang, Yunzhi, & Keller, Rachael. Real-time artificial intelligence for accelerator control: A study at the Fermilab Booster. United States. https://doi.org/10.1103/PhysRevAccelBeams.24.104601
St. John, Jason, Herwig, Christian, Kafkes, Diana, Mitrevski, Jovan, Pellico, William A., Perdue, Gabriel N., Quintero-Parra, Andres, Schupbach, Brian A., Seiya, Kiyomi, Tran, Nhan, Schram, Malachi, Duarte, Javier M., Huang, Yunzhi, and Keller, Rachael. Mon .
"Real-time artificial intelligence for accelerator control: A study at the Fermilab Booster". United States. https://doi.org/10.1103/PhysRevAccelBeams.24.104601.
@article{osti_1826394,
title = {Real-time artificial intelligence for accelerator control: A study at the Fermilab Booster},
author = {St. John, Jason and Herwig, Christian and Kafkes, Diana and Mitrevski, Jovan and Pellico, William A. and Perdue, Gabriel N. and Quintero-Parra, Andres and Schupbach, Brian A. and Seiya, Kiyomi and Tran, Nhan and Schram, Malachi and Duarte, Javier M. and Huang, Yunzhi and Keller, Rachael},
abstractNote = {We describe a method for precisely regulating the gradient magnet power supply (GMPS) at the Fermilab Booster accelerator complex using a neural network trained via reinforcement learning. We demonstrate preliminary results by training a surrogate machine-learning model on real accelerator data to emulate the GMPS, and using this surrogate model in turn to train the neural network for its regulation task. We additionally show how the neural networks to be deployed for control purposes may be compiled to execute on field-programmable gate arrays (FPGAs), and show the first machine-learning based control algorithm implemented on an FPGA for controls at the Fermilab accelerator complex. As there are no surprise latencies on an FPGA, this capability is important for operational stability in complicated environments such as an accelerator facility.},
doi = {10.1103/PhysRevAccelBeams.24.104601},
journal = {Physical Review Accelerators and Beams},
number = 10,
volume = 24,
place = {United States},
year = {2021},
month = {10}
}
Free Publicly Available Full Text
Publisher's Version of Record
https://doi.org/10.1103/PhysRevAccelBeams.24.104601
https://doi.org/10.1103/PhysRevAccelBeams.24.104601
Other availability
Search WorldCat to find libraries that may hold this journal
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
Machine learning for orders of magnitude speedup in multiobjective optimization of particle accelerator systems
journal, April 2020
- Edelen, Auralee; Neveu, Nicole; Frey, Matthias
- Physical Review Accelerators and Beams, Vol. 23, Issue 4
Fast inference of Boosted Decision Trees in FPGAs for particle physics
journal, May 2020
- Summers, S.; Guglielmo, G. Di; Duarte, J.
- Journal of Instrumentation, Vol. 15, Issue 05
Learning long-term dependencies with gradient descent is difficult
journal, March 1994
- Bengio, Y.; Simard, P.; Frasconi, P.
- IEEE Transactions on Neural Networks, Vol. 5, Issue 2
Deep learning
journal, May 2015
- LeCun, Yann; Bengio, Yoshua; Hinton, Geoffrey
- Nature, Vol. 521, Issue 7553
Sample-efficient reinforcement learning for CERN accelerator control
journal, December 2020
- Kain, Verena; Hirlander, Simon; Goddard, Brennan
- Physical Review Accelerators and Beams, Vol. 23, Issue 12
Policy gradient methods for free-electron laser and terahertz source optimization and stabilization at the FERMI free-electron laser at Elettra
journal, December 2020
- O’Shea, F. H.; Bruchon, N.; Gaio, G.
- Physical Review Accelerators and Beams, Vol. 23, Issue 12
Neuronlike adaptive elements that can solve difficult learning control problems
journal, September 1983
- Barto, Andrew G.; Sutton, Richard S.; Anderson, Charles W.
- IEEE Transactions on Systems, Man, and Cybernetics, Vol. SMC-13, Issue 5
Human-level control through deep reinforcement learning
journal, February 2015
- Mnih, Volodymyr; Kavukcuoglu, Koray; Silver, David
- Nature, Vol. 518, Issue 7540
Machine learning-based surrogate modeling for data-driven optimization: a comparison of subset selection for regression techniques
journal, May 2019
- Kim, Sun Hye; Boukouvala, Fani
- Optimization Letters, Vol. 14, Issue 4
Fast inference of deep neural networks in FPGAs for particle physics
journal, July 2018
- Duarte, J.; Han, S.; Harris, P.
- Journal of Instrumentation, Vol. 13, Issue 07
Investigating Causal Relations by Econometric Models and Cross-spectral Methods
journal, August 1969
- Granger, C. W. J.
- Econometrica, Vol. 37, Issue 3
Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors
journal, June 2021
- Coelho, Claudionor N.; Kuusela, Aki; Li, Shan
- Nature Machine Intelligence, Vol. 3, Issue 8
Toolflows for Mapping Convolutional Neural Networks on FPGAs: A Survey and Future Directions
journal, July 2018
- Venieris, Stylianos I.; Kouris, Alexandros; Bouganis, Christos-Savvas
- ACM Computing Surveys, Vol. 51, Issue 3
Compressing deep neural networks on FPGAs to binary and ternary precision with hls4ml
journal, December 2020
- Ngadiuba, Jennifer; Loncar, Vladimir; Pierini, Maurizio
- Machine Learning: Science and Technology, Vol. 2, Issue 1
Fast convolutional neural networks on FPGAs with hls4ml
journal, July 2021
- Aarrestad, Thea; Loncar, Vladimir; Ghielmetti, Nicolò
- Machine Learning: Science and Technology, Vol. 2, Issue 4
Directional Stability of Automatically Steered Bodies
journal, May 1922
- Minorsky., N.
- Journal of the American Society for Naval Engineers, Vol. 34, Issue 2
Bayesian Optimization of a Free-Electron Laser
journal, March 2020
- Duris, J.; Kennedy, D.; Hanuka, A.
- Physical Review Letters, Vol. 124, Issue 12
BOOSTR: A Dataset for Accelerator Control Systems
journal, April 2021
- Kafkes, Diana; St. John, Jason
- Data, Vol. 6, Issue 4
Demonstration of Model-Independent Control of the Longitudinal Phase Space of Electron Beams in the Linac-Coherent Light Source with Femtosecond Resolution
journal, July 2018
- Scheinker, Alexander; Edelen, Auralee; Bohler, Dorian
- Physical Review Letters, Vol. 121, Issue 4
Physics model-informed Gaussian process for online optimization of particle accelerators
journal, July 2021
- Hanuka, Adi; Huang, X.; Shtalenkova, J.
- Physical Review Accelerators and Beams, Vol. 24, Issue 7
Adaptive method for electron bunch profile prediction
journal, October 2015
- Scheinker, Alexander; Gessner, Spencer
- Physical Review Special Topics - Accelerators and Beams, Vol. 18, Issue 10
Machine learning and the physical sciences
journal, December 2019
- Carleo, Giuseppe; Cirac, Ignacio; Cranmer, Kyle
- Reviews of Modern Physics, Vol. 91, Issue 4
Optimum Settings for Automatic Controllers
journal, June 1993
- Ziegler, J. G.; Nichols, N. B.
- Journal of Dynamic Systems, Measurement, and Control, Vol. 115, Issue 2B
The Fermilab ACNET upgrade
journal, August 1990
- Briegel, Charlie; Johnson, Glenn; Winterowd, Lin
- Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, Vol. 293, Issue 1-2
On the Theory of Dynamic Programming
journal, August 1952
- Bellman, R.
- Proceedings of the National Academy of Sciences, Vol. 38, Issue 8
Neural Networks for Modeling and Control of Particle Accelerators
journal, April 2016
- Edelen, A. L.; Biedron, S. G.; Chase, B. E.
- IEEE Transactions on Nuclear Science, Vol. 63, Issue 2
Basic Reinforcement Learning Techniques to Control the Intensity of a Seeded Free-Electron Laser
journal, May 2020
- Bruchon, Niky; Fenu, Gianfranco; Gaio, Giulio
- Electronics, Vol. 9, Issue 5
An Introduction to Deep Reinforcement Learning
journal, January 2018
- François-Lavet, Vincent; Henderson, Peter; Islam, Riashat
- Foundations and Trends® in Machine Learning, Vol. 11, Issue 3-4
Distance-Weighted Graph Neural Networks on FPGAs for Real-Time Particle Reconstruction in High Energy Physics
journal, January 2021
- Iiyama, Yutaro; Cerminara, Gianluca; Gupta, Abhijay
- Frontiers in Big Data, Vol. 3
Superconducting radio-frequency cavity fault classification using machine learning at Jefferson Laboratory
journal, November 2020
- Tennant, Chris; Carpenter, Adam; Powers, Tom
- Physical Review Accelerators and Beams, Vol. 23, Issue 11
Long Short-Term Memory
journal, November 1997
- Hochreiter, Sepp; Schmidhuber, Jürgen
- Neural Computation, Vol. 9, Issue 8
[DL] A Survey of FPGA-based Neural Network Inference Accelerators
journal, April 2019
- Guo, Kaiyuan; Zeng, Shulin; Yu, Jincheng
- ACM Transactions on Reconfigurable Technology and Systems, Vol. 12, Issue 1
Apache Spark: a unified engine for big data processing
journal, October 2016
- Zaharia, Matei; Franklin, Michael J.; Ghodsi, Ali
- Communications of the ACM, Vol. 59, Issue 11
Machine learning for beam dynamics studies at the CERN Large Hadron Collider
journal, January 2021
- Arpaia, P.; Azzopardi, G.; Blanc, F.
- Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, Vol. 985
Surrogate modeling of the CLIC final-focus system using artificial neural networks
journal, May 2021
- Ögren, J.; Gohil, C.; Schulte, D.
- Journal of Instrumentation, Vol. 16, Issue 05
FINN- R: An End-to-End Deep-Learning Framework for Fast Exploration of Quantized Neural Networks
journal, December 2018
- Blott, Michaela; Preußer, Thomas B.; Fraser, Nicholas J.
- ACM Transactions on Reconfigurable Technology and Systems, Vol. 11, Issue 3
Machine learning-based longitudinal phase space prediction of particle accelerators
journal, November 2018
- Emma, C.; Edelen, A.; Hogan, M. J.
- Physical Review Accelerators and Beams, Vol. 21, Issue 11
FPGA-Based Accelerators of Deep Learning Networks for Learning and Classification: A Review
journal, January 2019
- Shawahna, Ahmad; Sait, Sadiq M.; El-Maleh, Aiman
- IEEE Access, Vol. 7