Coffea -- Columnar Object Framework For Effective Analysis
Abstract
The coffea framework provides a new approach to High-Energy Physics analysis, via columnar operations, that improves time-to-insight, scalability, portability, and reproducibility of analysis. It is implemented with the Python programming language, the scientific python package ecosystem, and commodity big data technologies. To achieve this suite of improvements across many use cases, coffea takes a factorized approach, separating the analysis implementation and data delivery scheme. All analysis operations are implemented using the NumPy or awkward-array packages which are wrapped to yield user code whose purpose is quickly intuited. Various data delivery schemes are wrapped into a common front-end which accepts user inputs and code, and returns user defined outputs. We will discuss our experience in implementing analysis of CMS data using the coffea framework along with a discussion of the user experience and future directions.
- Publication Date:
- Research Org.:
- Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC), High Energy Physics (HEP)
- OSTI Identifier:
- 1706163
- Report Number(s):
- arXiv:2008.12712; FERMILAB-CONF-20-494-CMS-SCD
oai:inspirehep.net:1813893
- DOE Contract Number:
- AC02-07CH11359
- Resource Type:
- Conference
- Resource Relation:
- Conference: 24th International Conference on Computing in High Energy and Nuclear Physics, Adelaide, Australia, 11/04-11/08/2019
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS
Citation Formats
Smith, Nicholas, Gray, Lindsey, Cremonesi, Matteo, Jayatilaka, Bo, Gutsche, Oliver, Hall, Allison, Pedro, Kevin, Acosta, Maria, Melo, Andrew, Belforte, Stefano, and Pivarski, Jim. Coffea -- Columnar Object Framework For Effective Analysis. United States: N. p., 2020.
Web.
Smith, Nicholas, Gray, Lindsey, Cremonesi, Matteo, Jayatilaka, Bo, Gutsche, Oliver, Hall, Allison, Pedro, Kevin, Acosta, Maria, Melo, Andrew, Belforte, Stefano, & Pivarski, Jim. Coffea -- Columnar Object Framework For Effective Analysis. United States.
Smith, Nicholas, Gray, Lindsey, Cremonesi, Matteo, Jayatilaka, Bo, Gutsche, Oliver, Hall, Allison, Pedro, Kevin, Acosta, Maria, Melo, Andrew, Belforte, Stefano, and Pivarski, Jim. Fri .
"Coffea -- Columnar Object Framework For Effective Analysis". United States. https://www.osti.gov/servlets/purl/1706163.
@article{osti_1706163,
title = {Coffea -- Columnar Object Framework For Effective Analysis},
author = {Smith, Nicholas and Gray, Lindsey and Cremonesi, Matteo and Jayatilaka, Bo and Gutsche, Oliver and Hall, Allison and Pedro, Kevin and Acosta, Maria and Melo, Andrew and Belforte, Stefano and Pivarski, Jim},
abstractNote = {The coffea framework provides a new approach to High-Energy Physics analysis, via columnar operations, that improves time-to-insight, scalability, portability, and reproducibility of analysis. It is implemented with the Python programming language, the scientific python package ecosystem, and commodity big data technologies. To achieve this suite of improvements across many use cases, coffea takes a factorized approach, separating the analysis implementation and data delivery scheme. All analysis operations are implemented using the NumPy or awkward-array packages which are wrapped to yield user code whose purpose is quickly intuited. Various data delivery schemes are wrapped into a common front-end which accepts user inputs and code, and returns user defined outputs. We will discuss our experience in implementing analysis of CMS data using the coffea framework along with a discussion of the user experience and future directions.},
doi = {},
url = {https://www.osti.gov/biblio/1706163},
journal = {},
number = ,
volume = ,
place = {United States},
year = {2020},
month = {8}
}