| | |
Summary: Diamond: A Storage Architecture for Early Discard in Interactive Search
Larry Huston, Rahul Sukthankar,· Rajiv Wickremesinghe, M. Satyanarayanan,·
Gregory R. Ganger,· Erik Riedel, Anastassia Ailamaki·
Intel Research Pittsburgh, ·Carnegie Mellon University, Duke University, Seagate Research
Abstract
This paper explores the concept of early discard for in-
teractive search of unindexed data. Processing data in-
side storage devices using downloaded searchlet code
enables Diamond to perform efficient, application-
specific filtering of large data collections. Early dis-
card helps users who are looking for "needles in a
haystack" by eliminating the bulk of the irrelevant items
as early as possible. A searchlet consists of a set of
application-generated filters that Diamond uses to deter-
mine whether an object may be of interest to the user.
The system optimizes the evaluation order of the filters
based on run-time measurements of each filter's selec-
tivity and computational cost. Diamond can also dy-
namically partition computation between the storage de-
vices and the host computer to adjust for changes in
|
