A Parallelised ROOT for Future HEP Data Processing

Piparo, Danilo; Canal, Philippe; Amadio, Guilherme; Guiraud, Enrico; Naumann, Axel; Valls, Xavier; Tejedor, Enric

doi:10.1051/epjconf/201921405033

Title: A Parallelised ROOT for Future HEP Data Processing

Abstract

In the coming years, HEP data processing will need to exploit parallelism on present and future hardware resources to sustain the bandwidth requirements. As one of the cornerstones of the HEP software ecosystem, ROOT embraced an ambitious parallelisation plan which delivered compelling results. In this contribution the strategy is characterised as well as its evolution in the medium term. The units of the ROOT framework are discussed where task and data parallelism have been introduced, with runtime and scaling measurements. We will give an overview of concurrent operations in ROOT, for instance in the areas of I/O (reading and writing of data), fitting / minimization, and data analysis. This paper introduces the programming model and use cases for explicit and implicit parallelism, where the former is explicit in user code and the latter is implicitly managed by ROOT internally.

Authors:

Piparo, Danilo ^[1];

^[2]; Amadio, Guilherme ^[1]; Guiraud, Enrico ^[3]; Naumann, Axel ^[1]; Valls, Xavier ^[4]; Tejedor, Enric ^[1]

European Organization for Nuclear Research (CERN), Geneva (Switzerland)
Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
European Organization for Nuclear Research (CERN), Geneva (Switzerland); Univ. of Oldenburg, Oldenburg (Germany)
European Organization for Nuclear Research (CERN), Geneva (Switzerland); Jaume I Univ., Castelló de la Plana, Valencia (Spain)

Publication Date:: Tue Sep 17 00:00:00 EDT 2019

Research Org.:: Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)

Sponsoring Org.:: USDOE Office of Science (SC), High Energy Physics (HEP)

OSTI Identifier:: 1574961

Report Number(s):: FERMILAB-CONF-19-551-SCD
Journal ID: ISSN 2100-014X; oai:inspirehep.net:1761253; TRN: US2001155

Grant/Contract Number:: AC02-07CH11359

Resource Type:: Accepted Manuscript

Journal Name:: EPJ Web of Conferences

Additional Journal Information:: Journal Volume: 214; Conference: 23. International Conference on Computing in High Energy and Nuclear Physics, Sofia (Bulgaria), 9-13 Jul 2018; Journal ID: ISSN 2100-014X

Publisher:: EDP Sciences

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING

Citation Formats


                    Piparo, Danilo, Canal, Philippe, Amadio, Guilherme, Guiraud, Enrico, Naumann, Axel, Valls, Xavier, and Tejedor, Enric. A Parallelised ROOT for Future HEP Data Processing.  United States: N. p., 2019. 
Web.  doi:10.1051/epjconf/201921405033.

Copy to clipboard


                    Piparo, Danilo, Canal, Philippe, Amadio, Guilherme, Guiraud, Enrico, Naumann, Axel, Valls, Xavier, & Tejedor, Enric. A Parallelised ROOT for Future HEP Data Processing.  United States.  https://doi.org/10.1051/epjconf/201921405033

Copy to clipboard


                    Piparo, Danilo, Canal, Philippe, Amadio, Guilherme, Guiraud, Enrico, Naumann, Axel, Valls, Xavier, and Tejedor, Enric. Tue .  
"A Parallelised ROOT for Future HEP Data Processing".  United States.  https://doi.org/10.1051/epjconf/201921405033.  https://www.osti.gov/servlets/purl/1574961.

Copy to clipboard


                    
@article{osti_1574961,

  title        = {A Parallelised ROOT for Future HEP Data Processing},

  author       = {Piparo, Danilo and Canal, Philippe and Amadio, Guilherme and Guiraud, Enrico and Naumann, Axel and Valls, Xavier and Tejedor, Enric},

  abstractNote = {In the coming years, HEP data processing will need to exploit parallelism on present and future hardware resources to sustain the bandwidth requirements. As one of the cornerstones of the HEP software ecosystem, ROOT embraced an ambitious parallelisation plan which delivered compelling results. In this contribution the strategy is characterised as well as its evolution in the medium term. The units of the ROOT framework are discussed where task and data parallelism have been introduced, with runtime and scaling measurements. We will give an overview of concurrent operations in ROOT, for instance in the areas of I/O (reading and writing of data), fitting / minimization, and data analysis. This paper introduces the programming model and use cases for explicit and implicit parallelism, where the former is explicit in user code and the latter is implicitly managed by ROOT internally.},

  doi          = {10.1051/epjconf/201921405033},

  journal      = {EPJ Web of Conferences},

  number       = ,

  volume       = 214,

  place        = {United States},

  year         = {Tue Sep 17 00:00:00 EDT 2019},

  month        = {Tue Sep 17 00:00:00 EDT 2019}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1051/epjconf/201921405033

Other availability

Search WorldCat to find libraries that may hold this journal

Save / Share:

Export Metadata

Save to My Library

Works referenced in this record:

Apache Spark: a unified engine for big data processing
journal, October 2016

Zaharia, Matei; Franklin, Michael J.; Ghodsi, Ali
Communications of the ACM, Vol. 59, Issue 11
DOI: 10.1145/2934664

NVIDIA cuda software and gpu parallel computing architecture
conference, January 2007

Kirk, David
Proceedings of the 6th international symposium on Memory management - ISMM '07
DOI: 10.1145/1296907.1296909

Similar Records in DOE PAGES and OSTI.GOV collections:

ROOT I/O compression improvements for HEP analysis

Conference Shadura, Oksana ; Canal, Philippe ; Piparo, Danilo ; ...

We overview recent changes in the ROOT I/O system, increasing performance and enhancing it and improving its interaction with other data analysis ecosystems. Both the newly introduced compression algorithms, the much faster bulk I/O data path, and a few additional techniques have the potential to significantly to improve experiment's software performance. The need for efficient lossless data compression has grown significantly as the amount of HEP data collected, transmitted, and stored has dramatically increased during the LHC era. While compression reduces storage space and, potentially, I/O bandwidth usage, it should not be applied blindly: there are significant trade-offs between themore »« less
Full Text Available
RDataFrame: Easy Parallel ROOT Analysis at 100 Threads

Journal Article Piparo, Danilo ; Canal, Philippe ; Guiraud, Enrico ; ... - EPJ Web of Conferences (Online)

The Physics programmes of LHC Run III and HL-LHC challenge the HEP community. The volume of data to be handled is unprecedented at every step of the data processing chain: analysis is no exception. Physicists must be provided with first-class analysis tools which are easy to use, exploit bleeding edge hardware technologies and allow to seamlessly express parallelism. This document discusses the declarative analysis engine of ROOT, RDataFrame, and gives details about how it allows to profitably exploit commodity hardware as well as high-end servers and manycore accelerators thanks to the synergy with the existing parallelised ROOT components. Real-life analysesmore »« less
https://doi.org/10.1051/epjconf/201921406029

Full Text Available
Parallel processing algorithms for hydrocodes on a computer with MIMD architecture (DENELCOR's HEP)

Technical Report Hicks, D L

In real time simulation/prediction of complex systems such as water-cooled nuclear reactors, if reactor operators had fast simulator/predictors to check the consequences of their operations before implementing them, events such as the incident at Three Mile Island might be avoided. However, existing simulator/predictors such as RELAP run slower than real time on serial computers. It appears that the only way to overcome the barrier to higher computing rates is to use computers with architectures that allow concurrent computations or parallel processing. The computer architecture with the greatest degree of parallelism is labeled Multiple Instruction Stream, Multiple Data Stream (MIMD). Anmore »« less
...And Eat it Too: High Read Performance in Write-Optimized HPC I/O Middleware File Formats

Conference Klasky, Scott A ; Lofstead, J. ; Bent, John ; ...

As HPC applications run on increasingly high process counts on larger and larger machines, both the frequency of checkpoints needed for fault tolerance and the resolution and size of Data Analysis Dumps are expected to increase proportionally. In order to maintain an acceptable ratio of time spent performing useful computation work to time spent performing I/O, write bandwidth to the underlying storage system must increase proportionally to this increase in the checkpoint and computation size. Unfortunately, popular scientific self-describing file formats such as netCDF and HDF5 are designed with a focus on portability and flexibility. Extra care and careful craftingmore »« less
Using OpenMP for HEP Framework Algorithm Scheduling [Slides]

Technical Report Jones, Christopher D. ; Gartung, Patrick

The OpenMP standard is the primary mechanism used at high performance computing facilities to allow intra-process parallelization. In contrast, many HEP specific software (such as CMSSW, GaudiHive, and ROOT) make use of Intel's Threading Building Blocks (TBB) library to accomplish the same goal. In this talk we will discuss our work to compare TBB and OpenMP when used for scheduling algorithms to be run by a HEP style data processing framework (i.e. running hundreds of interdependent algorithms at most once for each event read from the detector). This includes both scheduling of different algorithms to be run concurrently as wellmore »« less
https://doi.org/10.5281/zenodo.3598796

Full Text Available

Similar Records

Title: A Parallelised ROOT for Future HEP Data Processing

Abstract

Citation Formats

Apache Spark: a unified engine for big data processing journal, October 2016

NVIDIA cuda software and gpu parallel computing architecture conference, January 2007

Apache Spark: a unified engine for big data processing
journal, October 2016

NVIDIA cuda software and gpu parallel computing architecture
conference, January 2007