Home

About

Advanced Search

Browse by Discipline

Scientific Societies

E-print Alerts

Add E-prints

E-print Network
FAQHELPSITE MAPCONTACT US


  Advanced Search  

 
Using Titles and Category Names from Editor-Driven Taxonomies for Automatic Evaluation
 

Summary: Using Titles and Category Names from Editor-Driven
Taxonomies for Automatic Evaluation
Steven M. Beitzel, Eric C. Jensen, Abdur Chowdhury, David Grossman
Information Retrieval Laboratory
Illinois Institute of Technology
Chicago, IL 60616
{steve,ej,abdur,dagr}@ir.iit.edu
ABSTRACT
Evaluation of IR systems has always been difficult because of the
need for manually assessed relevance judgments. The advent of
large editor-driven taxonomies on the web opens the door to a new
evaluation approach. We use the ODP (Open Directory Project)
taxonomy to find sets of pseudo-relevant documents via one of
two assumptions: 1) taxonomy entries are relevant to a given query
if their editor-entered titles exactly match the query, or 2) all
entries in a leaf-level taxonomy category are relevant to a given
query if the category title exactly matches the query. We compare
and contrast these two methodologies by evaluating six web search
engines on a sample from an America Online log of ten million
web queries, using MRR measures for the first method and

  

Source: Argamon, Shlomo - Department of Computer Science, Illinois Institute of Technology
Chowdhury, Abdur - Information Retrievel Laboratory, Illinois Institute of Technology

 

Collections: Computer Technologies and Information Sciences