| | |
Summary: A Sampling Technique for Efficiently Estimating Measures of
Query Retrieval Performance Using Incomplete Judgments
Javed A. Aslam jaa@ccs.neu.edu
College of Computer and Information Science, Northeastern University, Boston, MA 02115 USA
Virgiliu Pavlu vip@ccs.neu.edu
College of Computer and Information Science, Northeastern University, Boston, MA 02115 USA
Emine Yilmaz emine@ccs.neu.edu
College of Computer and Information Science, Northeastern University, Boston, MA 02115 USA
Abstract
We consider the problem of evaluating the
performance of query retrieval systems, and
we propose a sampling technique for effi-
ciently estimating standard measures of re-
trieval performance using incomplete judg-
ments. Unlike existing techniques which (1)
rely on effectively complete, and thus pro-
hibitively expensive, relevance judgment sets,
(2) produce biased estimates of standard per-
formance measures, or (3) produce estimates
of non-standard measures thought to be cor-
|
