Summary: Where to Stop Reading a Ranked List?
Threshold Optimization using Truncated Score Distributions
University of Amsterdam, The Netherlands
Microsoft Research Cambridge, United Kingdom
Ranked retrieval has a particular disadvantage in comparison with
traditional Boolean retrieval: there is no clear cut-off point where
to stop consulting results. This is a serious problem in some setups.
We investigate and further develop methods to select the rank cut-
off value which optimizes a given effectiveness measure. Assuming
no other input than a system's output for a query--document scores
and their distribution--the task is essentially a score-distribution-
al threshold optimization problem. The recent trend in modeling
score distributions is to use a normal-exponential mixture: normal