Summary: Measure­based Performance Evaluation
Arne Andersson a , Paul Davidsson b , Johan Lind'en c
a Computing Science Dept., Uppsala University, Box 311, S--751 05 Uppsala,
Sweden. E­mail: arnea@csd.uu.se
b Dept. of Computer Science, University of Karlskrona/Ronneby, S--372 25
Ronneby, Sweden. E­mail: pdv@ipd.hk­r.se
c Dept. of Computer Science, Lund University, Box 118, S--221 00 Lund, Sweden.
E­mail: johan@cs.lth.se
Abstract
The concept of measure functions for generalization performance is suggested. This
concept provides an alternative way of selecting and evaluating learned classifiers,
and it allows us to define the learning problem as a computational problem.
Key words: Classifier performance evaluation, cross­validation, generalization.
1 Introduction
In this work, we suggest a new approach to evaluation of classification perfor­
mance. Today, most methods for evaluating the quality of a learned classifier
are based on some kind of cross­validation (Kohavi, 1995). However, we argue
that it is possible to make evaluations that take into account other important
aspects of the classifier than just classification accuracy on a few instances.
It has been known for a long time, see for example the ``no free lunch'' theorems

Source: Andersson, Arne - Department of Information Technology, Uppsala Universitet

Collections: Computer Technologies and Information Sciences