Data management for high energy physics experiments: Preliminary proposals
Conference
·
OSTI ID:6641106
Currently HEP experimental data are reduced as they become available. We propose instead a ''demand driven'' approach to data analysis. Full analysis will be performed only as needed, in response to user queries which specify the subset of events for which reduced data is needed. To support this approach we propose to partition the datasets on the cross product of several trigger inputs, instead of storing the data in chronological order. Queries will be automatically decomposed into a set of requests against several partitions. Indexing, physically clustering the data on the logical partitions, and caching of partitions will be employed for efficiency.
- Research Organization:
- Lawrence Berkeley Lab., CA (USA)
- DOE Contract Number:
- AC03-76SF00098
- OSTI ID:
- 6641106
- Report Number(s):
- LBL-22883; CONF-870249-4; ON: DE87006940
- Country of Publication:
- United States
- Language:
- English
Similar Records
ServiceX A Distributed, Caching, Columnar Data Delivery Service
The ATree: A data structure to support very large scientific databases
Distributed caching for processing raw arrays
Conference
·
Tue Dec 31 23:00:00 EST 2019
· EPJ Web Conf.
·
OSTI ID:1842729
The ATree: A data structure to support very large scientific databases
Technical Report
·
Tue Feb 28 23:00:00 EST 1995
·
OSTI ID:638241
Distributed caching for processing raw arrays
Conference
·
Mon Jul 09 00:00:00 EDT 2018
·
OSTI ID:1580975
Related Subjects
645100* -- High Energy Physics-- Particle Interactions & Properties-Experimental
72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS
99 GENERAL AND MISCELLANEOUS
990220 -- Computers
Computerized Models
& Computer Programs-- (1987-1989)
DATA ANALYSIS
DATA BASE MANAGEMENT
HIGH ENERGY PHYSICS
MANAGEMENT
PHYSICS
72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS
99 GENERAL AND MISCELLANEOUS
990220 -- Computers
Computerized Models
& Computer Programs-- (1987-1989)
DATA ANALYSIS
DATA BASE MANAGEMENT
HIGH ENERGY PHYSICS
MANAGEMENT
PHYSICS