Home

About

Advanced Search

Browse by Discipline

Scientific Societies

E-print Alerts

Add E-prints

E-print Network
FAQHELPSITE MAPCONTACT US


  Advanced Search  

 
A Framework for Determining Necessary Query Set Sizes to Evaluate Web
 

Summary: A Framework for Determining Necessary
Query Set Sizes to Evaluate Web
Search Effectiveness
Eric C. Jensen, Steven M. Beitzel, Ophir Frieder
Information Retrieval Laboratory
Illinois Institute of Technology
Chicago, IL 60616
{ej,steve,ophir}@ir.iit.edu
Abdur Chowdhury
Search & Navigation Group
America Online, Inc.
Dulles, VA 20166
cabdur@aol.com
Methodology
1. Randomly sample a distinct set of queries Q with size n from a query log.
2. For each query in Q, manually evaluate the union of the top X retrieved results from each of the engines.
3. Calculate each engines score for each query using the metric of interest, e.g. average precision (AvgP), reciprocal
rank of the best page (MRR), etc.
4. For B iterations:
a. Randomly sample, with repetition, a set of queries Q* with size m from the original set Q.

  

Source: Argamon, Shlomo - Department of Computer Science, Illinois Institute of Technology

 

Collections: Computer Technologies and Information Sciences