Bringing heterogeneity to the CMS software framework

Bocci, Andrea; Dagenhart, David; Innocente, Vincenzo; Jones, Christopher; Kortelainen, Matti; Pantaleo, Felice; Rovere, Marco

doi:10.1051/epjconf/202024505009

Title: Bringing heterogeneity to the CMS software framework

Abstract

The advent of computing resources with co-processors, for example Graphics Processing Units (GPU) or Field-Programmable Gate Arrays (FPGA), for use cases like the CMS High-Level Trigger (HLT) or data processing at leadership-class supercomputers imposes challenges for the current data processing frameworks. These challenges include developing a model for algorithms to offload their computations on the co-processors as well as keeping the traditional CPU busy doing other work. The CMS data processing framework, CMSSW, implements multithreading using the Intel Threading Building Blocks (TBB) library, that utilizes tasks as concurrent units of work. In this paper we will discuss a generic mechanism to interact effectively with non-CPU resources that has been implemented in CMSSW. In addition, configuring such a heterogeneous system is challenging. In CMSSW an application is configured with a configuration file written in the Python language. The algorithm types are part of the configuration. The challenge therefore is to unify the CPU and co-processor settings while allowing their implementations to be separate. We will explain how we solved these challenges while minimizing the necessary changes to the CMSSW framework. We will also discuss on a concrete example how algorithms would offload work to NVIDIA GPUs using directly the CUDA API.

Authors:

Bocci, Andrea ^[1]; Dagenhart, David ^[2]; Innocente, Vincenzo ^[1]; Jones, Christopher ^[2]; Kortelainen, Matti ^[2]; Pantaleo, Felice ^[1]; Rovere, Marco ^[1]

European Organization for Nuclear Research (CERN), Geneva (Switzerland)
Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)

Publication Date:: Mon Nov 16 00:00:00 EST 2020

Research Org.:: Fermi National Accelerator Laboratory (FNAL), Batavia, IL (United States)

Sponsoring Org.:: USDOE Office of Science (SC), High Energy Physics (HEP)

Contributing Org.:: CMS Collaboration

OSTI Identifier:: 1617227

Report Number(s):: FERMILAB-CONF-19-545-SCD
Journal ID: ISSN 2100-014X; oai:inspirehep.net:1790694; TRN: US2106735

Grant/Contract Number:: AC02-07CH11359

Resource Type:: Accepted Manuscript

Journal Name:: EPJ Web of Conferences (Online)

Additional Journal Information:: Journal Name: EPJ Web of Conferences (Online); Journal Volume: 245; Conference: 24.International Conference on Computing in High Energy and Nuclear Physics (CHEP 2019), Adelaide (Australia), 4-8 Nov 2019; Journal ID: ISSN 2100-014X

Publisher:: EDP Sciences

Country of Publication:: United States

Language:: English

Subject:: 72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS; 97 MATHEMATICS AND COMPUTING

Citation Formats


                    Bocci, Andrea, Dagenhart, David, Innocente, Vincenzo, Jones, Christopher, Kortelainen, Matti, Pantaleo, Felice, and Rovere, Marco. Bringing heterogeneity to the CMS software framework.  United States: N. p., 2020. 
Web.  doi:10.1051/epjconf/202024505009.

Copy to clipboard


                    Bocci, Andrea, Dagenhart, David, Innocente, Vincenzo, Jones, Christopher, Kortelainen, Matti, Pantaleo, Felice, & Rovere, Marco. Bringing heterogeneity to the CMS software framework.  United States.  https://doi.org/10.1051/epjconf/202024505009

Copy to clipboard


                    Bocci, Andrea, Dagenhart, David, Innocente, Vincenzo, Jones, Christopher, Kortelainen, Matti, Pantaleo, Felice, and Rovere, Marco. Mon .  
"Bringing heterogeneity to the CMS software framework".  United States.  https://doi.org/10.1051/epjconf/202024505009.  https://www.osti.gov/servlets/purl/1617227.

Copy to clipboard


                    
@article{osti_1617227,

  title        = {Bringing heterogeneity to the CMS software framework},

  author       = {Bocci, Andrea and Dagenhart, David and Innocente, Vincenzo and Jones, Christopher and Kortelainen, Matti and Pantaleo, Felice and Rovere, Marco},

  abstractNote = {The advent of computing resources with co-processors, for example Graphics Processing Units (GPU) or Field-Programmable Gate Arrays (FPGA), for use cases like the CMS High-Level Trigger (HLT) or data processing at leadership-class supercomputers imposes challenges for the current data processing frameworks. These challenges include developing a model for algorithms to offload their computations on the co-processors as well as keeping the traditional CPU busy doing other work. The CMS data processing framework, CMSSW, implements multithreading using the Intel Threading Building Blocks (TBB) library, that utilizes tasks as concurrent units of work. In this paper we will discuss a generic mechanism to interact effectively with non-CPU resources that has been implemented in CMSSW. In addition, configuring such a heterogeneous system is challenging. In CMSSW an application is configured with a configuration file written in the Python language. The algorithm types are part of the configuration. The challenge therefore is to unify the CPU and co-processor settings while allowing their implementations to be separate. We will explain how we solved these challenges while minimizing the necessary changes to the CMSSW framework. We will also discuss on a concrete example how algorithms would offload work to NVIDIA GPUs using directly the CUDA API.},

  doi          = {10.1051/epjconf/202024505009},

  journal      = {EPJ Web of Conferences (Online)},

  number       = ,

  volume       = 245,

  place        = {United States},

  year         = {Mon Nov 16 00:00:00 EST 2020},

  month        = {Mon Nov 16 00:00:00 EST 2020}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1051/epjconf/202024505009

Other availability

Search WorldCat to find libraries that may hold this journal

Save / Share:

Export Metadata

Save to My Library

Works referenced in this record:

LHC Machine
journal, August 2008

Evans, Lyndon; Bryant, Philip
Journal of Instrumentation, Vol. 3, Issue 08
DOI: 10.1088/1748-0221/3/08/S08001

Works referencing / citing this record:

Graph Neural Networks for Particle Tracking and Reconstruction
text, January 2020

Duarte, Javier; Vlimant, Jean-Roch
arXiv
DOI: 10.48550/arxiv.2012.01249

Similar Records in DOE PAGES and OSTI.GOV collections:

Bringing heterogeneity to the CMS software framework [Slides]

Technical Report Bocci, Andrea ; Dagenhart, W. David ; Innocente, Vincenzo ; ...

Co-processors or accelerators like GPUs and FPGAs are becoming more and more popular. CMS' data processing framework (CMSSW) implements multi-threading using Intel TBB utilizing tasks as concurrent units of work. We have developed generic mechanisms within the CMSSW framework to interact effectively with non-CPU resources and configure CPU and non-CPU algorithms in a unified way. As a first step to gain experience, we have explored mechanisms for how algorithms could offload work to NVIDIA GPUs with CUDA.
https://doi.org/10.2172/1623357

Full Text Available
Bringing heterogeneity to the CMS software framework [Slides]

Technical Report Bocci, Andrea ; Dagenhart, W. David ; Innocente, Vincenzo ; ...

Co-processors or accelerators like GPUs and FPGAs are becoming more and more popular. CMS’ data processing framework (CMSSW) implements multi-threading using Intel TBB utilizing tasks as concurrent units of work. We have developed generic mechanisms within the CMSSW framework to interact effectively with non-CPU resources and configure CPU and non-CPU algorithms in a unified way. As a first step to gain experience, we have explored mechanisms for how algorithms could offload work to NVIDIA GPUs with CUDA.
https://doi.org/10.2172/1630717

Full Text Available
Storage-Intensive Supercomputing Benchmark Study

Technical Report Cohen, J ; Dossa, D ; Gokhale, M ; ...

Critical data science applications requiring frequent access to storage perform poorly on today's computing architectures. This project addresses efficient computation of data-intensive problems in national security and basic science by exploring, advancing, and applying a new form of computing called storage-intensive supercomputing (SISC). Our goal is to enable applications that simply cannot run on current systems, and, for a broad range of data-intensive problems, to deliver an order of magnitude improvement in price/performance over today's data-intensive architectures. This technical report documents much of the work done under LDRD 07-ERD-063 Storage Intensive Supercomputing during the period 05/07-09/07. The following chapters describe:more »« less
https://doi.org/10.2172/924182

Full Text Available
Investigation of Portable Event-Based Monte Carlo Transport Using the NVIDIA Thrust Library

Journal Article Bleile, Ryan C. ; Department of Computer and Information Science, University of Oregon, Eugene, OR 97403 ; Brantley, Patrick S. ; ... - Transactions of the American Nuclear Society

Power consumption considerations are driving future high performance computing platforms toward many-core computing architectures. Los Alamos National Laboratory's Trinity machine, available in 2016, will use both Intel Xeon Haswell processors and Intel Xeon Phi Knights Landing many integrated core (MIC) architecture coprocessors. Lawrence Livermore National Laboratory's Sierra machine, available in 2018, will use an IBM PowerPC architecture along with Nvidia graphics processing unit (GPU) architecture accelerators. These different advanced architectures make the computing landscape in upcoming years complex. Traditional approaches to Monte Carlo transport do not work efficiently on these new computing platforms. MIC architectures require vectorization to operate efficiently,more »« less
Celeritas R&D Report: Accelerating Geant4

Technical Report Johnson, Seth R. ; Biondo, Elliott ; Esseiva, Julien ; ...

Celeritas is a new Monte Carlo (MC) detector simulation code designed for computationally intensive applications on high-performance heterogeneous architectures. In the past two years Celeritas has advanced from prototyping a Graphics Processing Unit (GPU)-based single physics model in infinite medium to implementing a full set of electromagnetic (EM) physics processes in complex geometries. The current release of Celeritas, version 0.4, has incorporated full device-based navigation, an event loop in the presence of magnetic fields, and detector hit scoring. New functionality incorporates a scheduler to offload electromagnetic physics to the GPU within a Geant4-driven simulation, enabling straightforward integration of Celeritas intomore »« less
https://doi.org/10.2172/2281972

Full Text Available

Similar Records
Related Works
report (2)

Title: Bringing heterogeneity to the CMS software framework

Abstract

Citation Formats

LHC Machine journal, August 2008

Graph Neural Networks for Particle Tracking and Reconstruction text, January 2020

LHC Machine
journal, August 2008

Graph Neural Networks for Particle Tracking and Reconstruction
text, January 2020