skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

This content will become publicly available on October 14, 2020

Title: FPGA-Accelerated Machine Learning Inference as a Service for Particle Physics Computing

Abstract

Large-scale particle physics experiments face challenging demands for high-throughput computing resources both now and in the future. New heterogeneous computing paradigms on dedicated hardware with increased parallelization, such as Field Programmable Gate Arrays (FPGAs), offer exciting solutions with large potential gains. The growing applications of machine learning algorithms in particle physics for simulation, reconstruction, and analysis are naturally deployed on such platforms. We demonstrate that the acceleration of machine learning inference as a web service represents a heterogeneous computing solution for particle physics experiments that potentially requires minimal modification to the current computing model. As examples, we retrain the ResNet-50 convolutional neural network to demonstrate state-of-the-art performance for top quark jet tagging at the LHC and apply a ResNet-50 model with transfer learning for neutrino event classification. Using Project Brainwave by Microsoft to accelerate the ResNet-50 image classification model, we achieve average inference times of 60 (10) ms with our experimental physics software framework using Brainwave as a cloud (edge or on-premises) service, representing an improvement by a factor of approximately 30 (175) in model inference latency over traditional CPU inference in current experimental hardware. A single FPGA service accessed by many CPUs achieves a throughput of 600–700 inferences permore » second using an image batch of one, comparable to large batch-size GPU throughput and significantly better than small batch-size GPU throughput. Deployed as an edge or cloud service for the particle physics computing model, coprocessor accelerators can have a higher duty cycle and are potentially much more cost-effective.« less

Authors:
 [1];  [2];  [3];  [1];  [3];  [1];  [4];  [1];  [4];  [1];  [5];  [6];  [1];  [4];  [6];  [2]; ORCiD logo [1];  [3];  [1];  [4] more »;  [4];  [3];  [7] « less
  1. Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
  2. Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States)
  3. Univ. of Washington, Seattle, WA (United States)
  4. Microsoft, Redmond, WA (United States)
  5. European Organization for Nuclear Research (CERN), Geneva (Switzerland); Univ. of Belgrade (Serbia)
  6. European Organization for Nuclear Research (CERN), Geneva (Switzerland)
  7. Univ. of Illinois, Chicago, IL (United States)
Publication Date:
Research Org.:
Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
Sponsoring Org.:
USDOE Office of Science (SC), High Energy Physics (HEP) (SC-25)
OSTI Identifier:
1565955
Report Number(s):
arXiv:1904.08986; FERMILAB-PUB-19-170-CD-CMS-E-ND
Journal ID: ISSN 2510-2036; oai:inspirehep.net:1730403
Grant/Contract Number:  
AC02-07CH11359
Resource Type:
Accepted Manuscript
Journal Name:
Computing and Software for Big Science
Additional Journal Information:
Journal Volume: 3; Journal Issue: 1; Journal ID: ISSN 2510-2036
Publisher:
Springer
Country of Publication:
United States
Language:
English
Subject:
46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND TECHNOLOGY; 72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS

Citation Formats

Duarte, Javier, Harris, Philip, Hauck, Scott, Holzman, Burt, Hsu, Shih-Chieh, Jindariani, Sergo, Khan, Suffian, Kreis, Benjamin, Lee, Brian, Liu, Mia, Lončar, Vladimir, Ngadiuba, Jennifer, Pedro, Kevin, Perez, Brandon, Pierini, Maurizio, Rankin, Dylan, Tran, Nhan, Trahms, Matthew, Tsaris, Aristeidis, Versteeg, Colin, Way, Ted W., Werran, Dustin, and Wu, Zhenbin. FPGA-Accelerated Machine Learning Inference as a Service for Particle Physics Computing. United States: N. p., 2019. Web. doi:10.1007/s41781-019-0027-2.
Duarte, Javier, Harris, Philip, Hauck, Scott, Holzman, Burt, Hsu, Shih-Chieh, Jindariani, Sergo, Khan, Suffian, Kreis, Benjamin, Lee, Brian, Liu, Mia, Lončar, Vladimir, Ngadiuba, Jennifer, Pedro, Kevin, Perez, Brandon, Pierini, Maurizio, Rankin, Dylan, Tran, Nhan, Trahms, Matthew, Tsaris, Aristeidis, Versteeg, Colin, Way, Ted W., Werran, Dustin, & Wu, Zhenbin. FPGA-Accelerated Machine Learning Inference as a Service for Particle Physics Computing. United States. doi:10.1007/s41781-019-0027-2.
Duarte, Javier, Harris, Philip, Hauck, Scott, Holzman, Burt, Hsu, Shih-Chieh, Jindariani, Sergo, Khan, Suffian, Kreis, Benjamin, Lee, Brian, Liu, Mia, Lončar, Vladimir, Ngadiuba, Jennifer, Pedro, Kevin, Perez, Brandon, Pierini, Maurizio, Rankin, Dylan, Tran, Nhan, Trahms, Matthew, Tsaris, Aristeidis, Versteeg, Colin, Way, Ted W., Werran, Dustin, and Wu, Zhenbin. Mon . "FPGA-Accelerated Machine Learning Inference as a Service for Particle Physics Computing". United States. doi:10.1007/s41781-019-0027-2.
@article{osti_1565955,
title = {FPGA-Accelerated Machine Learning Inference as a Service for Particle Physics Computing},
author = {Duarte, Javier and Harris, Philip and Hauck, Scott and Holzman, Burt and Hsu, Shih-Chieh and Jindariani, Sergo and Khan, Suffian and Kreis, Benjamin and Lee, Brian and Liu, Mia and Lončar, Vladimir and Ngadiuba, Jennifer and Pedro, Kevin and Perez, Brandon and Pierini, Maurizio and Rankin, Dylan and Tran, Nhan and Trahms, Matthew and Tsaris, Aristeidis and Versteeg, Colin and Way, Ted W. and Werran, Dustin and Wu, Zhenbin},
abstractNote = {Large-scale particle physics experiments face challenging demands for high-throughput computing resources both now and in the future. New heterogeneous computing paradigms on dedicated hardware with increased parallelization, such as Field Programmable Gate Arrays (FPGAs), offer exciting solutions with large potential gains. The growing applications of machine learning algorithms in particle physics for simulation, reconstruction, and analysis are naturally deployed on such platforms. We demonstrate that the acceleration of machine learning inference as a web service represents a heterogeneous computing solution for particle physics experiments that potentially requires minimal modification to the current computing model. As examples, we retrain the ResNet-50 convolutional neural network to demonstrate state-of-the-art performance for top quark jet tagging at the LHC and apply a ResNet-50 model with transfer learning for neutrino event classification. Using Project Brainwave by Microsoft to accelerate the ResNet-50 image classification model, we achieve average inference times of 60 (10) ms with our experimental physics software framework using Brainwave as a cloud (edge or on-premises) service, representing an improvement by a factor of approximately 30 (175) in model inference latency over traditional CPU inference in current experimental hardware. A single FPGA service accessed by many CPUs achieves a throughput of 600–700 inferences per second using an image batch of one, comparable to large batch-size GPU throughput and significantly better than small batch-size GPU throughput. Deployed as an edge or cloud service for the particle physics computing model, coprocessor accelerators can have a higher duty cycle and are potentially much more cost-effective.},
doi = {10.1007/s41781-019-0027-2},
journal = {Computing and Software for Big Science},
number = 1,
volume = 3,
place = {United States},
year = {2019},
month = {10}
}

Journal Article:
Free Publicly Available Full Text
This content will become publicly available on October 14, 2020
Publisher's Version of Record

Save / Share:

Works referenced in this record:

Densely Connected Convolutional Networks
conference, July 2017

  • Huang, Gao; Liu, Zhuang; Maaten, Laurens van der
  • 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • DOI: 10.1109/CVPR.2017.243

The anti- k t jet clustering algorithm
journal, April 2008


Deep-learned Top Tagging with a Lorentz Layer
journal, January 2018


Fast inference of deep neural networks in FPGAs for particle physics
journal, July 2018


Reionization and the Cosmic Dawn with the Square Kilometre Array
journal, April 2013

  • Mellema, Garrelt; Koopmans, Léon V. E.; Abdalla, Filipe A.
  • Experimental Astronomy, Vol. 36, Issue 1-2
  • DOI: 10.1007/s10686-013-9334-5

Convolutional neural networks applied to neutrino events in a liquid argon time projection chamber
journal, March 2017


CaloGAN: Simulating 3D high energy particle showers in multilayer electromagnetic calorimeters with generative adversarial networks
journal, January 2018


Observation of a new boson at a mass of 125 GeV with the CMS experiment at the LHC
journal, September 2012


DELPHES 3: a modular framework for fast simulation of a generic collider experiment
journal, February 2014

  • de Favereau, J.; Delaere, C.; Demin, P.
  • Journal of High Energy Physics, Vol. 2014, Issue 2
  • DOI: 10.1007/JHEP02(2014)057

Observation of a new particle in the search for the Standard Model Higgs boson with the ATLAS detector at the LHC
journal, September 2012


A convolutional neural network neutrino event classifier
journal, September 2016


FastJet user manual: (for version 3.0.2)
journal, March 2012


Machine learning at the energy and intensity frontiers of particle physics
journal, August 2018


Constraints on Oscillation Parameters from ν e Appearance and ν μ Disappearance in NOvA
journal, June 2017


ImageNet: A large-scale hierarchical image database
conference, June 2009

  • Deng, Jia; Dong, Wei; Socher, Richard
  • 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPR Workshops), 2009 IEEE Conference on Computer Vision and Pattern Recognition
  • DOI: 10.1109/CVPR.2009.5206848

Deep Residual Learning for Image Recognition
conference, June 2016

  • He, Kaiming; Zhang, Xiangyu; Ren, Shaoqing
  • 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • DOI: 10.1109/CVPR.2016.90

The HEP.TrkX Project: deep neural networks for HL-LHC online and offline tracking
journal, January 2017


An introduction to PYTHIA 8.2
journal, June 2015

  • Sjöstrand, Torbjörn; Ask, Stefan; Christiansen, Jesper R.
  • Computer Physics Communications, Vol. 191
  • DOI: 10.1016/j.cpc.2015.01.024

A cloud-scale acceleration architecture
conference, October 2016

  • Caulfield, Adrian M.; Chung, Eric S.; Putnam, Andrew
  • 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)
  • DOI: 10.1109/MICRO.2016.7783710

Tuning PYTHIA 8.1: the Monash 2013 tune
journal, August 2014


Machine Learning in High Energy Physics Community White Paper
journal, September 2018


Dispelling the <mml:math altimg="si1.gif" overflow="scroll" xmlns:xocs="http://www.elsevier.com/xml/xocs/dtd" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.elsevier.com/xml/ja/dtd" xmlns:ja="http://www.elsevier.com/xml/ja/dtd" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:tb="http://www.elsevier.com/xml/common/table/dtd" xmlns:sb="http://www.elsevier.com/xml/common/struct-bib/dtd" xmlns:ce="http://www.elsevier.com/xml/common/dtd" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:cals="http://www.elsevier.com/xml/common/cals/dtd"><mml:msup><mml:mi>N</mml:mi><mml:mn>3</mml:mn></mml:msup></mml:math> myth for the <mml:math altimg="si2.gif" overflow="scroll" xmlns:xocs="http://www.elsevier.com/xml/xocs/dtd" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.elsevier.com/xml/ja/dtd" xmlns:ja="http://www.elsevier.com/xml/ja/dtd" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:tb="http://www.elsevier.com/xml/common/table/dtd" xmlns:sb="http://www.elsevier.com/xml/common/struct-bib/dtd" xmlns:ce="http://www.elsevier.com/xml/common/dtd" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:cals="http://www.elsevier.com/xml/common/cals/dtd"><mml:msub><mml:mi>k</mml:mi><mml:mi>t</mml:mi></mml:msub></mml:math> jet-finder
journal, September 2006