Novel functional and distributed approaches to data analysis available in ROOT

Journal Article · Thu Oct 18 00:00:00 EDT 2018 · Journal of Physics. Conference Series

DOI:https://doi.org/10.1088/1742-6596/1085/4/042008· OSTI ID:1523435

Amadio, G. ^[1]; Blomer, J. ^[1]; Canal, P.  ^[2]; Ganis, G. ^[1]; Guiraud, E. ^[3]; Mato Vila, P. ^[1]; Moneta, L. ^[1]; Piparo, D. ^[1]; Tejedor, E. ^[1]; Valls Pla, X. ^[4]

---

1. European Organization for Nuclear Research (CERN), Geneva (Switzerland)
2. Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
3. European Organization for Nuclear Research (CERN), Geneva (Switzerland); Univ. of Oldenburg, Oldenburg (Germany)
4. European Organization for Nuclear Research (CERN), Geneva (Switzerland); Univ. Jaume I, Castellon (Spain)

The bright future of particle physics at the Energy and Intensity frontiers poses exciting challenges to the scientific software community. The traditional strategies for processing and analyzing data are evolving in order to (i) offer higher-level programming model approaches and (ii) exploit parallelism to cope with the ever increasing complexity and size of the datasets. This contribution describes how the ROOT framework, a cornerstone of software stacks dedicated to particle physics, is preparing to provide adequate solutions for the analysis of large amount of scientific data on parallel architectures. The functional approach to parallel data analysis provided with the ROOT TDataFrame interface is then characterized. The design choices behind this new interface are described also comparing with other widely adopted tools such as Pandas and Apache Spark. The programming model is illustrated highlighting the reduction of boilerplate code, composability of the actions and data transformations as well as the capabilities of dealing with different data sources such as ROOT, JSON, CSV or databases. Details are given about how the functional approach allows transparent implicit parallelization of the chain of operations specified by the user. The progress done in the field of distributed analysis is examined. In particular, the power of the integration of ROOT with Apache Spark via the PyROOT interface is shown. In addition, the building blocks for the expression of parallelism in ROOT are briefly characterized together with the structural changes applied in the building and testing infrastructure which were necessary to put them in production.

• Research Organization: Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
• Sponsoring Organization: USDOE Office of Science (SC), High Energy Physics (HEP)
• Grant/Contract Number: AC02-07CH11359
• OSTI ID: 1523435
• Report Number(s): FERMILAB-CONF-18-750-CD; 1699879
• Journal Information: Journal of Physics. Conference Series, Vol. 1085, Issue 4; Conference: 18th International Workshop on Advanced Computing and Analysis Techniques in Physics Research, Seattle, WA (United States), 21-25 Aug 2017; ISSN 1742-6588
• Publisher: IOP Publishing
• Country of Publication: United States
• Language: English

Related Subjects

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS