Constraints on Future Analysis Metadata Systems in High Energy Physics

Journal Article · 26 July 2022 · Computing and Software for Big Science

DOI: https://doi.org/10.1007/s41781-022-00086-2 · OSTI ID:1873673

Khoo, T. J. ^[1]; Reinsvold Hall, A.  ^[2]; Skidmore, N. ^[3]; Alderweireldt, S. ^[4]; Anders, J. ^[5]; Burr, C. ^[6]; Buttinger, W. ^[7]; David, P. ^[8]; Gouskos, L. ^[6]; Gray, L. ^[9]; Hageböck, S. ^[6]; Krasznahorkay, A. ^[6]; Laycock, P. ^[10]; Lister, A. ^[11]; Marshall, Z. ^[12]; Meyer, A. B. ^[13]; Novak, T. ^[13]; Rappoccio, S. ^[14]; Ritter, M. ^[15]; Rodrigues, E. ^[16] ; Rumsevicius, J. ^[6]; Sexton-Kennedy, L. ^[9]; Smith, N. ^[9]; Stewart, G. A. ^[6]; Wertz, S. ^[8]

---

1. Humboldt University of Berlin (Germany)
2. US Naval Academy, Annapolis, MD (United States)
3. University of Manchester (United Kingdom)
4. University of Edinburgh, Scotland (United Kingdom)
5. University of Bern (Switzerland)
6. European Organization for Nuclear Research (CERN), Geneva (Switzerland)
7. Rutherford Appleton Laboratory, Didcot (United Kingdom)
8. Universite Catholique de Louvain, Louvain-la-Neuve (Belgium)
9. Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
10. Brookhaven National Lab. (BNL), Upton, NY (United States)
11. University of British Columbia, Vancouver, BC (Canada)
12. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
13. Deutsches Elektronen-Synchrotron DESY, Hamburg (Germany)
14. State University of New York at Buffalo, NY (United States)
15. Ludwig Maximilian University of Munich, Munich (Germany)
16. University of Liverpool (United Kingdom)

In high energy physics (HEP), analysis metadata comes in many forms—from theoretical cross-sections, to calibration corrections, to details about file processing. Correctly applying metadata is a crucial and often time-consuming step in an analysis, but designing analysis metadata systems has historically received little direct attention. Among other considerations, an ideal metadata tool should be easy to use by new analysers, should scale to large data volumes and diverse processing paradigms, and should enable future analysis reinterpretation. This document, which is the product of community discussions organised by the HEP Software Foundation, categorises types of metadata by scope and format and gives examples of current metadata solutions. Important design considerations for metadata systems, including sociological factors, analysis preservation efforts, and technical factors, are discussed. A list of best practices and technical requirements for future analysis metadata systems is presented. These best practices could guide the development of a future cross-experimental effort for analysis metadata tools.

• Research Organization: Brookhaven National Laboratory (BNL), Upton, NY (United States); Fermi National Accelerator Laboratory (FNAL), Batavia, IL (United States); Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)
• Sponsoring Organization: European Research Council (ERC); National Science Foundation (NSF); USDOE Office of Science (SC), High Energy Physics (HEP)
• Grant/Contract Number: AC02-07CH11359; SC0012704
• OSTI ID: 1873673
• Report Number(s): BNL-223654-2022-JAAM; FERMILAB-PUB-22-345-OCIO-SCD; arXiv:2203.00463; oai:inspirehep.net:2040531
• Journal Information: Computing and Software for Big Science, Journal Name: Computing and Software for Big Science Journal Issue: 1 Vol. 6; ISSN 2510-2036
• Publisher: SpringerCopyright Statement
• Country of Publication: United States
• Language: English