Efficient Joins with Compressed Bitmap Indexes

Conference · Wed Aug 19 04:00:00 EDT 2009

OSTI ID:982896

Madduri, Kamesh; Wu, Kesheng

We present a new class of adaptive algorithms that use compressed bitmap indexes to speed up evaluation of the range join query in relational databases. We determine the best strategy to process a join query based on a fast sub-linear time computation of the join selectivity (the ratio of the number of tuples in the result to the total number of possible tuples). In addition, we use compressed bitmaps to represent the join output compactly: the space requirement for storing the tuples representing the join of two relations is asymptotically bounded by min(h; n . cb), where h is the number of tuple pairs in the result relation, n is the number of tuples in the smaller of the two relations, and cb is the cardinality of the larger column being joined. We present a theoretical analysis of our algorithms, as well as experimental results on large-scale synthetic and real data sets. Our implementations are efficient, and consistently outperform well-known approaches for a range of join selectivity factors. For instance, our count-only algorithm is up to three orders of magnitude faster than the sort-merge approach, and our best bitmap index-based algorithm is 1.2x-80x faster than the sort-merge algorithm, for various query instances. We achieve these speedups by exploiting several inherent performance advantages of compressed bitmap indexes for join processing: an implicit partitioning of the attributes, space-efficiency, and tolerance of high-cardinality relations.

• Research Organization: Ernest Orlando Lawrence Berkeley National Laboratory, Berkeley, CA (US)
• Sponsoring Organization: Computational Research Division
• DOE Contract Number: AC02-05CH11231
• OSTI ID: 982896
• Report Number(s): LBNL-3115E
• Country of Publication: United States
• Language: English

Related Subjects

97 MATHEMATICS AND COMPUTING
ALGORITHMS
Compressed Bitmap Index
Range Join
EVALUATION
KNOWLEDGE MANAGEMENT
PERFORMANCE
PROCESSING
TOLERANCE
VELOCITY