Search Menu
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information
Search Scholarly Publications

Title: FPGA-Accelerated Machine Learning Inference as a Service for Particle Physics Computing

Abstract

Large-scale particle physics experiments face challenging demands for high-throughput computing resources both now and in the future. New heterogeneous computing paradigms on dedicated hardware with increased parallelization, such as Field Programmable Gate Arrays (FPGAs), offer exciting solutions with large potential gains. The growing applications of machine learning algorithms in particle physics for simulation, reconstruction, and analysis are naturally deployed on such platforms. We demonstrate that the acceleration of machine learning inference as a web service represents a heterogeneous computing solution for particle physics experiments that potentially requires minimal modification to the current computing model. As examples, we retrain the ResNet-50 convolutional neural network to demonstrate state-of-the-art performance for top quark jet tagging at the LHC and apply a ResNet-50 model with transfer learning for neutrino event classification. Using Project Brainwave by Microsoft to accelerate the ResNet-50 image classification model, we achieve average inference times of 60 (10) ms with our experimental physics software framework using Brainwave as a cloud (edge or on-premises) service, representing an improvement by a factor of approximately 30 (175) in model inference latency over traditional CPU inference in current experimental hardware. A single FPGA service accessed by many CPUs achieves a throughput of 600–700 inferences permore » second using an image batch of one, comparable to large batch-size GPU throughput and significantly better than small batch-size GPU throughput. Deployed as an edge or cloud service for the particle physics computing model, coprocessor accelerators can have a higher duty cycle and are potentially much more cost-effective.« less

Authors:
 [1];  [2];  [3];  [1];  [3];  [1];  [4];  [1];  [4];  [1];  [5];  [6];  [1];  [4];  [6];  [2]; ORCiD logo [1];  [3];  [1];  [4] more »;  [4];  [3];  [7] « less
+ Show Author Affiliations
  1. Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
  2. Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States)
  3. Univ. of Washington, Seattle, WA (United States)
  4. Microsoft, Redmond, WA (United States)
  5. European Organization for Nuclear Research (CERN), Geneva (Switzerland); Univ. of Belgrade (Serbia)
  6. European Organization for Nuclear Research (CERN), Geneva (Switzerland)
  7. Univ. of Illinois, Chicago, IL (United States)
Publication Date:
Research Org.:
Fermi National Accelerator Laboratory (FNAL), Batavia, IL (United States)
Sponsoring Org.:
USDOE Office of Science (SC), High Energy Physics (HEP)
OSTI Identifier:
1565955
Report Number(s):
arXiv:1904.08986; FERMILAB-PUB-19-170-CD-CMS-E-ND
Journal ID: ISSN 2510-2036; oai:inspirehep.net:1730403; TRN: US2000947
Grant/Contract Number:  
AC02-07CH11359
Resource Type:
Accepted Manuscript
Journal Name:
Computing and Software for Big Science
Additional Journal Information:
Journal Volume: 3; Journal Issue: 1; Journal ID: ISSN 2510-2036
Publisher:
Springer
Country of Publication:
United States
Language:
English
Subject:
46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND TECHNOLOGY; 72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS

Citation Formats

Duarte, Javier, Harris, Philip, Hauck, Scott, Holzman, Burt, Hsu, Shih-Chieh, Jindariani, Sergo, Khan, Suffian, Kreis, Benjamin, Lee, Brian, Liu, Mia, Lončar, Vladimir, Ngadiuba, Jennifer, Pedro, Kevin, Perez, Brandon, Pierini, Maurizio, Rankin, Dylan, Tran, Nhan, Trahms, Matthew, Tsaris, Aristeidis, Versteeg, Colin, Way, Ted W., Werran, Dustin, and Wu, Zhenbin. FPGA-Accelerated Machine Learning Inference as a Service for Particle Physics Computing. United States: N. p., 2019. Web. doi:10.1007/s41781-019-0027-2.
Copy to clipboard
Duarte, Javier, Harris, Philip, Hauck, Scott, Holzman, Burt, Hsu, Shih-Chieh, Jindariani, Sergo, Khan, Suffian, Kreis, Benjamin, Lee, Brian, Liu, Mia, Lončar, Vladimir, Ngadiuba, Jennifer, Pedro, Kevin, Perez, Brandon, Pierini, Maurizio, Rankin, Dylan, Tran, Nhan, Trahms, Matthew, Tsaris, Aristeidis, Versteeg, Colin, Way, Ted W., Werran, Dustin, & Wu, Zhenbin. FPGA-Accelerated Machine Learning Inference as a Service for Particle Physics Computing. United States. https://doi.org/10.1007/s41781-019-0027-2
Copy to clipboard
Duarte, Javier, Harris, Philip, Hauck, Scott, Holzman, Burt, Hsu, Shih-Chieh, Jindariani, Sergo, Khan, Suffian, Kreis, Benjamin, Lee, Brian, Liu, Mia, Lončar, Vladimir, Ngadiuba, Jennifer, Pedro, Kevin, Perez, Brandon, Pierini, Maurizio, Rankin, Dylan, Tran, Nhan, Trahms, Matthew, Tsaris, Aristeidis, Versteeg, Colin, Way, Ted W., Werran, Dustin, and Wu, Zhenbin. Mon . "FPGA-Accelerated Machine Learning Inference as a Service for Particle Physics Computing". United States. https://doi.org/10.1007/s41781-019-0027-2. https://www.osti.gov/servlets/purl/1565955.
Copy to clipboard
@article{osti_1565955,
title = {FPGA-Accelerated Machine Learning Inference as a Service for Particle Physics Computing},
author = {Duarte, Javier and Harris, Philip and Hauck, Scott and Holzman, Burt and Hsu, Shih-Chieh and Jindariani, Sergo and Khan, Suffian and Kreis, Benjamin and Lee, Brian and Liu, Mia and Lončar, Vladimir and Ngadiuba, Jennifer and Pedro, Kevin and Perez, Brandon and Pierini, Maurizio and Rankin, Dylan and Tran, Nhan and Trahms, Matthew and Tsaris, Aristeidis and Versteeg, Colin and Way, Ted W. and Werran, Dustin and Wu, Zhenbin},
abstractNote = {Large-scale particle physics experiments face challenging demands for high-throughput computing resources both now and in the future. New heterogeneous computing paradigms on dedicated hardware with increased parallelization, such as Field Programmable Gate Arrays (FPGAs), offer exciting solutions with large potential gains. The growing applications of machine learning algorithms in particle physics for simulation, reconstruction, and analysis are naturally deployed on such platforms. We demonstrate that the acceleration of machine learning inference as a web service represents a heterogeneous computing solution for particle physics experiments that potentially requires minimal modification to the current computing model. As examples, we retrain the ResNet-50 convolutional neural network to demonstrate state-of-the-art performance for top quark jet tagging at the LHC and apply a ResNet-50 model with transfer learning for neutrino event classification. Using Project Brainwave by Microsoft to accelerate the ResNet-50 image classification model, we achieve average inference times of 60 (10) ms with our experimental physics software framework using Brainwave as a cloud (edge or on-premises) service, representing an improvement by a factor of approximately 30 (175) in model inference latency over traditional CPU inference in current experimental hardware. A single FPGA service accessed by many CPUs achieves a throughput of 600–700 inferences per second using an image batch of one, comparable to large batch-size GPU throughput and significantly better than small batch-size GPU throughput. Deployed as an edge or cloud service for the particle physics computing model, coprocessor accelerators can have a higher duty cycle and are potentially much more cost-effective.},
doi = {10.1007/s41781-019-0027-2},
journal = {Computing and Software for Big Science},
number = 1,
volume = 3,
place = {United States},
year = {Mon Oct 14 00:00:00 EDT 2019},
month = {Mon Oct 14 00:00:00 EDT 2019}
}
Copy to clipboard
Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
https://doi.org/10.1007/s41781-019-0027-2
Copyright Statement
Other availability
Search WorldCat Search WorldCat to find libraries that may hold this journal

Figures / Tables:

Fig. 1 Fig. 1: A diagram of the computing model used in the CMS software.
All figures and tables (16 total)
Save / Share:
Export Metadata
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.

Works referenced in this record:

Energy calibration and resolution of the CMS electromagnetic calorimeter in pp collisions at √s= 7 TeV
journal, September 2013

Densely Connected Convolutional Networks
conference, July 2017

  • Huang, Gao; Liu, Zhuang; Maaten, Laurens van der
  • 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • DOI: 10.1109/CVPR.2017.243

The anti- k t jet clustering algorithm
journal, April 2008

Deep-learned Top Tagging with a Lorentz Layer
journal, January 2018

Fast inference of deep neural networks in FPGAs for particle physics
journal, July 2018

Reionization and the Cosmic Dawn with the Square Kilometre Array
journal, April 2013

  • Mellema, Garrelt; Koopmans, Léon V. E.; Abdalla, Filipe A.
  • Experimental Astronomy, Vol. 36, Issue 1-2
  • DOI: 10.1007/s10686-013-9334-5

Convolutional neural networks applied to neutrino events in a liquid argon time projection chamber
journal, March 2017

CaloGAN: Simulating 3D high energy particle showers in multilayer electromagnetic calorimeters with generative adversarial networks
journal, January 2018

Observation of a new boson at a mass of 125 GeV with the CMS experiment at the LHC
journal, September 2012

DELPHES 3: a modular framework for fast simulation of a generic collider experiment
journal, February 2014

  • de Favereau, J.; Delaere, C.; Demin, P.
  • Journal of High Energy Physics, Vol. 2014, Issue 2
  • DOI: 10.1007/JHEP02(2014)057

Observation of a new particle in the search for the Standard Model Higgs boson with the ATLAS detector at the LHC
journal, September 2012

A convolutional neural network neutrino event classifier
journal, September 2016

FastJet user manual: (for version 3.0.2)
journal, March 2012

Machine learning at the energy and intensity frontiers of particle physics
journal, August 2018

Constraints on Oscillation Parameters from ν e Appearance and ν μ Disappearance in NOvA
journal, June 2017

ImageNet: A large-scale hierarchical image database
conference, June 2009

  • Deng, Jia; Dong, Wei; Socher, Richard
  • 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPR Workshops), 2009 IEEE Conference on Computer Vision and Pattern Recognition
  • DOI: 10.1109/CVPR.2009.5206848

Deep Residual Learning for Image Recognition
conference, June 2016

  • He, Kaiming; Zhang, Xiangyu; Ren, Shaoqing
  • 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • DOI: 10.1109/CVPR.2016.90

The HEP.TrkX Project: deep neural networks for HL-LHC online and offline tracking
journal, January 2017

An introduction to PYTHIA 8.2
journal, June 2015

  • Sjöstrand, Torbjörn; Ask, Stefan; Christiansen, Jesper R.
  • Computer Physics Communications, Vol. 191
  • DOI: 10.1016/j.cpc.2015.01.024

A cloud-scale acceleration architecture
conference, October 2016

  • Caulfield, Adrian M.; Chung, Eric S.; Putnam, Andrew
  • 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)
  • DOI: 10.1109/MICRO.2016.7783710

Tuning PYTHIA 8.1: the Monash 2013 tune
journal, August 2014

Machine Learning in High Energy Physics Community White Paper
journal, September 2018

Dispelling the <mml:math altimg="si1.gif" overflow="scroll" xmlns:xocs="http://www.elsevier.com/xml/xocs/dtd" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.elsevier.com/xml/ja/dtd" xmlns:ja="http://www.elsevier.com/xml/ja/dtd" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:tb="http://www.elsevier.com/xml/common/table/dtd" xmlns:sb="http://www.elsevier.com/xml/common/struct-bib/dtd" xmlns:ce="http://www.elsevier.com/xml/common/dtd" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:cals="http://www.elsevier.com/xml/common/cals/dtd"><mml:msup><mml:mi>N</mml:mi><mml:mn>3</mml:mn></mml:msup></mml:math> myth for the <mml:math altimg="si2.gif" overflow="scroll" xmlns:xocs="http://www.elsevier.com/xml/xocs/dtd" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.elsevier.com/xml/ja/dtd" xmlns:ja="http://www.elsevier.com/xml/ja/dtd" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:tb="http://www.elsevier.com/xml/common/table/dtd" xmlns:sb="http://www.elsevier.com/xml/common/struct-bib/dtd" xmlns:ce="http://www.elsevier.com/xml/common/dtd" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:cals="http://www.elsevier.com/xml/common/cals/dtd"><mml:msub><mml:mi>k</mml:mi><mml:mi>t</mml:mi></mml:msub></mml:math> jet-finder
journal, September 2006

The Future of Computing Performance
journal, July 2011

Observation of a new Boson at a mass of 125 gev with the cms Experiment at the lhc
conference, March 2015

  • Mariotti, Chiara
  • Proceedings of the MG13 Meeting on General Relativity, The Thirteenth Marcel Grossmann Meeting
  • DOI: 10.1142/9789814623995_0019

Energy calibration and resolution of the CMS electromagnetic calorimeter in pp collisions at √s = 7 TeV
text, January 2013

Deep-learned Top Tagging with a Lorentz Layer
text, January 2018

Observation of a new boson at a mass of 125 GeV with the CMS experiment at the LHC
text, January 2018

Energy calibration and resolution of the CMS electromagnetic calorimeter in pp collisions at √s = 7 TeV
text, January 2013

  • Collaboration, Cms; Chatrchyan, S.; Khachatryan, V.
  • IOP Publishing
  • DOI: 10.5167/uzh-92313

Convolutional Neural Networks Applied to Neutrino Events in a Liquid Argon Time Projection Chamber
text, January 2017

  • Weber, Michael; Ereditato, Antonio; Kreslo, Igor
  • Institute of Physics Publishing IOP
  • DOI: 10.7892/boris.114294

The anti-k_t jet clustering algorithm
text, January 2008

DELPHES 3, A modular framework for fast simulation of a generic collider experiment
text, January 2013

Densely Connected Convolutional Networks
preprint, January 2016

Convolutional Neural Networks Applied to Neutrino Events in a Liquid Argon Time Projection Chamber
text, January 2016

Machine Learning in High Energy Physics Community White Paper
preprint, January 2018

Dispelling the N^3 myth for the Kt jet-finder
text, January 2005

    Sort by:
    [ × clear filter / sort ]

    Figures / Tables found in this record:

    Fig. 1 (p. 4)figure
    Fig. 2 (p. 4)figure
    Fig. 3 (p. 6)figure
    Fig. 4 (p. 6)figure
    Table 1 (p. 6)table
    Fig. 5 (p. 8)figure
    Fig. 6 (p. 8)figure
    Fig. 7 (p. 9)figure
    Fig. 8 (p. 9)figure
    Fig. 9 (p. 9)figure
    Fig. 10 (p. 10)figure
    Fig. 11 (p. 11)figure
    Fig. 12 (p. 11)figure
    Fig. 13 (p. 12)figure
    Fig. 14 (p. 13)figure
    Table 2 (p. 13)table
      [ × clear filter / sort ]
      Figures/Tables have been extracted from DOE-funded journal article accepted manuscripts.

      Similar Records in DOE PAGES and OSTI.GOV collections: