Using Bitmap Indexing Technology for Combined Numerical and TextQueries
In this paper, we describe a strategy of using compressedbitmap indices to speed up queries on both numerical data and textdocuments. By using an efficient compression algorithm, these compressedbitmap indices are compact even for indices with millions of distinctterms. Moreover, bitmap indices can be used very efficiently to answerBoolean queries over text documents involving multiple query terms.Existing inverted indices for text searches are usually inefficient forcorpora with a very large number of terms as well as for queriesinvolving a large number of hits. We demonstrate that our compressedbitmap index technology overcomes both of those short-comings. In aperformance comparison against a commonly used database system, ourindices answer queries 30 times faster on average. To provide full SQLsupport, we integrated our indexing software, called FastBit, withMonetDB. The integrated system MonetDB/FastBit provides not onlyefficient searches on a single table as FastBit does, but also answersjoin queries efficiently. Furthermore, MonetDB/FastBit also provides avery efficient retrieval mechanism of result records.
- Research Organization:
- Ernest Orlando Lawrence Berkeley NationalLaboratory, Berkeley, CA (US)
- Sponsoring Organization:
- USDOE Director, Office of Science. Office of AdvancedScientific Computing Research. Mathematical, Information, andComputational Sciences Division
- DOE Contract Number:
- AC02-05CH11231
- OSTI ID:
- 918636
- Report Number(s):
- LBNL--61768; BnR: 400470000
- Country of Publication:
- United States
- Language:
- English
Similar Records
A compute-Efficient Bitmap Compression Index for Database Applications
An efficient compression scheme for bitmap indices