| | |
Summary: The TREC Files: The (Ground) Truth Is Out There
Savvas A. Chatzichristofis Konstantinos Zagoris Avi Arampatzis
Department of Electrical and Computer Engineering
Democritus University of Thrace, Xanthi 67100, Greece
{schatzic,kzagoris,avi}@ee.duth.gr
ABSTRACT
Traditional tools for information retrieval (IR) evaluation, such as
TREC's trec_eval, have outdated command-line interfaces with
many unused features, or `switches', accumulated over the years.
They are usually seen as cumbersome applications by new IR re-
searchers, steepening the learning curve. We introduce a platform-
independent application for IR evaluation with a graphical easy-to-
use interface: the TREC_Files Evaluator. The application supports
most of the standard measures used for evaluation in TREC, CLEF,
and elsewhere, such as MAP, P10, P20, and bpref, as well as the Av-
eraged Normalized Modified Retrieval Rank (ANMRR) proposed
by MPEG for image retrieval evaluation. Additional features in-
clude a batch mode and statistical significance testing of the results
against a pre-selected baseline.
Categories and Subject Descriptors: D.0 [Software]: General
|
