A Parallelised ROOT for Future HEP Data Processing

doi:https://doi.org/10.1051/epjconf/201921405033

Title: A Parallelised ROOT for Future HEP Data Processing

Abstract

In the coming years, HEP data processing will need to exploit parallelism on present and future hardware resources to sustain the bandwidth requirements. As one of the cornerstones of the HEP software ecosystem, ROOT embraced an ambitious parallelisation plan which delivered compelling results. In this contribution the strategy is characterised as well as its evolution in the medium term. The units of the ROOT framework are discussed where task and data parallelism have been introduced, with runtime and scaling measurements. We will give an overview of concurrent operations in ROOT, for instance in the areas of I/O (reading and writing of data), fitting / minimization, and data analysis. This paper introduces the programming model and use cases for explicit and implicit parallelism, where the former is explicit in user code and the latter is implicitly managed by ROOT internally.

Authors:

Piparo, Danilo ^[1];

^[2]; Amadio, Guilherme ^[1]; Guiraud, Enrico ^[3]; Naumann, Axel ^[1]; Valls, Xavier ^[4]; Tejedor, Enric ^[1]

European Organization for Nuclear Research (CERN), Geneva (Switzerland)
Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
European Organization for Nuclear Research (CERN), Geneva (Switzerland); Univ. of Oldenburg, Oldenburg (Germany)
European Organization for Nuclear Research (CERN), Geneva (Switzerland); Jaume I Univ., Castelló de la Plana, Valencia (Spain)

Publication Date:: 2019-09-17

Research Org.:: Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)

Sponsoring Org.:: USDOE Office of Science (SC), High Energy Physics (HEP)

OSTI Identifier:: 1574961

Report Number(s):: FERMILAB-CONF-19-551-SCD
Journal ID: ISSN 2100-014X; oai:inspirehep.net:1761253; TRN: US2001155

Grant/Contract Number:: AC02-07CH11359

Resource Type:: Journal Article: Accepted Manuscript

Journal Name:: EPJ Web of Conferences

Additional Journal Information:: Journal Volume: 214; Conference: 23. International Conference on Computing in High Energy and Nuclear Physics, Sofia (Bulgaria), 9-13 Jul 2018; Journal ID: ISSN 2100-014X

Publisher:: EDP Sciences

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING

Citation Formats


                    Piparo, Danilo, Canal, Philippe, Amadio, Guilherme, Guiraud, Enrico, Naumann, Axel, Valls, Xavier, and Tejedor, Enric. A Parallelised ROOT for Future HEP Data Processing.  United States: N. p., 2019. 
        Web.  doi:10.1051/epjconf/201921405033.

Copy to clipboard


                    Piparo, Danilo, Canal, Philippe, Amadio, Guilherme, Guiraud, Enrico, Naumann, Axel, Valls, Xavier, & Tejedor, Enric. A Parallelised ROOT for Future HEP Data Processing.  United States.  https://doi.org/10.1051/epjconf/201921405033

Copy to clipboard


                    Piparo, Danilo, Canal, Philippe, Amadio, Guilherme, Guiraud, Enrico, Naumann, Axel, Valls, Xavier, and Tejedor, Enric. Tue .  
        "A Parallelised ROOT for Future HEP Data Processing".  United States.  https://doi.org/10.1051/epjconf/201921405033.  https://www.osti.gov/servlets/purl/1574961.

Copy to clipboard


                    
@article{osti_1574961,

  title        = {A Parallelised ROOT for Future HEP Data Processing},

  author       = {Piparo, Danilo and Canal, Philippe and Amadio, Guilherme and Guiraud, Enrico and Naumann, Axel and Valls, Xavier and Tejedor, Enric},

  abstractNote = {In the coming years, HEP data processing will need to exploit parallelism on present and future hardware resources to sustain the bandwidth requirements. As one of the cornerstones of the HEP software ecosystem, ROOT embraced an ambitious parallelisation plan which delivered compelling results. In this contribution the strategy is characterised as well as its evolution in the medium term. The units of the ROOT framework are discussed where task and data parallelism have been introduced, with runtime and scaling measurements. We will give an overview of concurrent operations in ROOT, for instance in the areas of I/O (reading and writing of data), fitting / minimization, and data analysis. This paper introduces the programming model and use cases for explicit and implicit parallelism, where the former is explicit in user code and the latter is implicitly managed by ROOT internally.},

  doi          = {10.1051/epjconf/201921405033},

  url          = {https://www.osti.gov/biblio/1574961},
  journal      = {EPJ Web of Conferences},

  issn         = {2100-014X},

  number       = ,

  volume       = 214,

  place        = {United States},

  year         = {2019},

  month        = {9}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1051/epjconf/201921405033

Other availability

Search WorldCat to find libraries that may hold this journal

Save / Share:

Export Metadata

Save to My Library

Works referenced in this record:

Apache Spark: a unified engine for big data processing
journal, October 2016

Zaharia, Matei; Franklin, Michael J.; Ghodsi, Ali
Communications of the ACM, Vol. 59, Issue 11
https://doi.org/10.1145/2934664

NVIDIA cuda software and gpu parallel computing architecture
conference, January 2007

Kirk, David
Proceedings of the 6th international symposium on Memory management - ISMM '07
https://doi.org/10.1145/1296907.1296909

Similar records in OSTI.GOV collections:

ROOT I/O compression improvements for HEP analysis

Conference Shadura, Oksana ; Canal, Philippe ; Piparo, Danilo ; ...

We overview recent changes in the ROOT I/O system, increasing performance and enhancing it and improving its interaction with other data analysis ecosystems. Both the newly introduced compression algorithms, the much faster bulk I/O data path, and a few additional techniques have the potential to significantly to improve experiment's software performance. The need for efficient lossless data compression has grown significantly as the amount of HEP data collected, transmitted, and stored has dramatically increased during the LHC era. While compression reduces storage space and, potentially, I/O bandwidth usage, it should not be applied blindly: there are significant trade-offs between themore »« less
Full Text Available
RDataFrame: Easy Parallel ROOT Analysis at 100 Threads

Journal Article Piparo, Danilo ; Canal, Philippe ; Guiraud, Enrico ; ... - EPJ Web of Conferences

The Physics programmes of LHC Run III and HL-LHC challenge the HEP community. The volume of data to be handled is unprecedented at every step of the data processing chain: analysis is no exception. Physicists must be provided with first-class analysis tools which are easy to use, exploit bleeding edge hardware technologies and allow to seamlessly express parallelism. This document discusses the declarative analysis engine of ROOT, RDataFrame, and gives details about how it allows to profitably exploit commodity hardware as well as high-end servers and manycore accelerators thanks to the synergy with the existing parallelised ROOT components. Real-life analysesmore »« less
https://doi.org/10.1051/epjconf/201921406029

Full Text Available
Parallel processing algorithms for hydrocodes on a computer with MIMD architecture (DENELCOR's HEP)

Technical Report Hicks, D L

In real time simulation/prediction of complex systems such as water-cooled nuclear reactors, if reactor operators had fast simulator/predictors to check the consequences of their operations before implementing them, events such as the incident at Three Mile Island might be avoided. However, existing simulator/predictors such as RELAP run slower than real time on serial computers. It appears that the only way to overcome the barrier to higher computing rates is to use computers with architectures that allow concurrent computations or parallel processing. The computer architecture with the greatest degree of parallelism is labeled Multiple Instruction Stream, Multiple Data Stream (MIMD). Anmore »« less
Expressing Parallelism with ROOT

Journal Article Piparo, D. ; Tejedor, E. ; Guiraud, E. ; ... - Journal of Physics. Conference Series

The need for processing the ever-increasing amount of data generated by the LHC experiments in a more efficient way has motivated ROOT to further develop its support for parallelism. Such support is being tackled both for shared-memory and distributed-memory environments. The incarnations of the aforementioned parallelism are multi-threading, multi-processing and cluster-wide executions. In the area of multi-threading, we discuss the new implicit parallelism and related interfaces, as well as the new building blocks to safely operate with ROOT objects in a multi-threaded environment. Regarding multi-processing, we review the new MultiProc framework, comparing it with similar tools (e.g. multiprocessing module inmore »« less
https://doi.org/10.1088/1742-6596/898/7/072022

Full Text Available
Using the force to write parallel HEP programs

Technical Report Jordan, H F

The basic idea behind using the force method of parallel programming is to write a program (a force) which can be executed by any number of processes, including one process. The fork into parallel processes is done before the force is ever executed and the number of processes can be chosen on the basis of hardware efficiency only. Experience indicates that it is not hard to write programs which are independent of the number of executing processes on a shared-memory MIMD computer. Several such applications programs have been written for the HEP computer in HEP Fortran 77, which is standardmore »« less
https://doi.org/10.2172/86860

Full Text Available

Similar Records

Title: A Parallelised ROOT for Future HEP Data Processing

Abstract

Citation Formats

Apache Spark: a unified engine for big data processing journal, October 2016

NVIDIA cuda software and gpu parallel computing architecture conference, January 2007

Apache Spark: a unified engine for big data processing
journal, October 2016

NVIDIA cuda software and gpu parallel computing architecture
conference, January 2007