Summary: PREPRINT. Proc. of the ECML'98 Workshop on
UpgradingLearning3 the MetaLevel: Model Selection and
DataTransformation, pages 5465, 1998.
Model selection using measure functions
Arne Andersson 1 , Paul Davidsson 2 , and Johan Lind’ en 1
1 Dept. of Computer Science, Lund University, Box 118, S--221 00 Lund, Sweden.
2 Dept. of Computer Science, University of Karlskrona/Ronneby, S--372 25 Ronneby, Sweden.
firstname.lastname@example.org, email@example.com, firstname.lastname@example.org
The concept of measure functions for generalization performance is suggested. This
concept provides an alternative way of selecting and evaluating learned models (classi
fiers). In addition, it makes it possible to state a learning problem as a computational
problem. The the known prior (meta)knowledge about the problem domain is captured
in a measure function that, to each possible combination of a training set and a classifier,
assigns a value describing how good the classifier is. The computational problem is then
to find a classifier maximizing the measure function. We argue that measure functions are
of great value for practical applications. Besides of being a tool for model selection, they:
(i) force us to make explicit the relevant prior knowledge about the learning problem at
hand, (ii) provide a deeper understanding of existing algorithms, and (iii) help us in the
construction of problemspecific algorithms. We illustrate the last point by suggesting a