Distilling particle knowledge for fast reconstruction at high-energy physics experiments
[1];
[1];
[2];
[1];
[3];
[4];
[5]
- Karlsruhe Inst. of Technology (KIT) (Germany)
- Chinese Academy of Sciences (CAS), Beijing (China). Institute of High Energy Physics (IHEP)
- Karlsruhe Inst. of Technology (KIT) (Germany); Imperial College, London (United Kingdom)
- Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). National Energy Research Scientific Computing Center (NERSC)
- Eidgenoessische Technische Hochschule (ETH), Zurich (Switzerland)
Knowledge distillation is a form of model compression that allows artificial neural networks of different sizes to learn from one another. Its main application is the compactification of large deep neural networks to free up computational resources, in particular on edge devices. In this article, we consider proton-proton collisions at the High-Luminosity Large Hadron Collider (HL-LHC) and demonstrate a successful knowledge transfer from an event-level graph neural network (GNN) to a particle-level small deep neural network (DNN). Our algorithm, DistillNet, is a DNN that is trained to learn about the provenance of particles, as provided by the soft labels that are the GNN outputs, to predict whether or not a particle originates from the primary interaction vertex. The results indicate that for this problem, which is one of the main challenges at the HL-LHC, there is minimal loss during the transfer of knowledge to the small student network, while improving significantly the computational resource needs compared to the teacher. This is demonstrated for the distilled student network on a CPU, as well as for a quantized and pruned student network deployed on a field programmable gate array. Our study proves that knowledge transfer between networks of different complexity can be used for fast artificial intelligence (AI) in high-energy physics that improves the expressiveness of observables over non-AI-based reconstruction algorithms. Such an approach can become essential at the HL-LHC experiments, e.g. to comply with the resource budget of their trigger stages.
- Research Organization:
- Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). National Energy Research Scientific Computing Center (NERSC)
- Sponsoring Organization:
- USDOE Office of Science (SC), High Energy Physics (HEP); USDOE Office of Science (SC), Basic Energy Sciences (BES). Scientific User Facilities (SUF); Swiss National Science Foundation (SNSF)
- Grant/Contract Number:
- AC02-05CH11231
- OSTI ID:
- 2447590
- Journal Information:
- Machine Learning: Science and Technology, Journal Name: Machine Learning: Science and Technology Journal Issue: 2 Vol. 5; ISSN 2632-2153
- Publisher:
- IOP PublishingCopyright Statement
- Country of Publication:
- United States
- Language:
- English
Similar Records
Robustness of topological persistence in knowledge distillation for wearable sensor data
Graph Neural Networks for Charged Particle Tracking on FPGAs