Summary: Measurebased Performance Evaluation
Arne Andersson a , Paul Davidsson b , Johan Lind'en c
a Computing Science Dept., Uppsala University, Box 311, S--751 05 Uppsala,
Sweden. Email: firstname.lastname@example.org
b Dept. of Computer Science, University of Karlskrona/Ronneby, S--372 25
Ronneby, Sweden. Email: email@example.com
c Dept. of Computer Science, Lund University, Box 118, S--221 00 Lund, Sweden.
The concept of measure functions for generalization performance is suggested. This
concept provides an alternative way of selecting and evaluating learned classifiers,
and it allows us to define the learning problem as a computational problem.
Key words: Classifier performance evaluation, crossvalidation, generalization.
In this work, we suggest a new approach to evaluation of classification perfor
mance. Today, most methods for evaluating the quality of a learned classifier
are based on some kind of crossvalidation (Kohavi, 1995). However, we argue
that it is possible to make evaluations that take into account other important
aspects of the classifier than just classification accuracy on a few instances.
It has been known for a long time, see for example the ``no free lunch'' theorems