Bitmap Indices for Fast End-User Physics Analysis in ROOT

Journal Article · Tue Jul 26 04:00:00 EDT 2005 · Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment

OSTI ID:891814

Stockinger, Kurt; Wu, Kesheng; Brun, Rene; Canal, Philippe

Most physics analysis jobs involve multiple selection steps on the input data. These selection steps are called ''cuts'' or ''queries''. A common strategy to implement these queries is to read all input data from files and then process the queries in memory. In many applications the number of variables used to define these queries is a relative small portion of the overall data set therefore reading all variables into memory takes unnecessarily long time. In this paper we describe an integration effort that can significantly reduce this unnecessary reading by using an efficient compressed bitmap index technology. The primary advantage of this index is that it can process arbitrary combinations of queries very efficiently, while most other indexing technologies suffer from the ''curse of dimensionality'' as the number of queries increases. By integrating this index technology with the ROOT analysis framework, the end-users can benefit from the added efficiency without having to modify their analysis programs. Our performance results show that for multi-dimensional queries, bitmap indices outperform the traditional analysis method up to a factor of 10.

• Research Organization: Ernest Orlando Lawrence Berkeley NationalLaboratory, Berkeley, CA (US)
• Sponsoring Organization: USDOE Director. Office of Science. Office of AdvancedScientific Computing Research
• DOE Contract Number: AC02-05CH11231
• OSTI ID: 891814
• Report Number(s): LBNL--58426; BnR: KJ0101030
• Journal Information: Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, Journal Name: Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment Journal Issue: 1 Vol. 559
• Country of Publication: United States
• Language: English

Related Subjects

99 GENERAL AND MISCELLANEOUS
EFFICIENCY
PERFORMANCE
PHYSICS