Summary: PREPRINT. Proc. of the ECML'98 Workshop on Upgrading Learningto the Meta-Level: Model Selection and Data Transformation, pages 54-65, 1998.
Model selection using measure functions
, Paul Davidsson2
, and Johan LindŽen1
1 Dept. of Computer Science, Lund University, Box 118, S221 00 Lund, Sweden.
2 Dept. of Computer Science, University of Karlskrona/Ronneby, S372 25 Ronneby, Sweden.
firstname.lastname@example.org, email@example.com, firstname.lastname@example.org
The concept of measure functions for generalization performance is suggested. This
concept provides an alternative way of selecting and evaluating learned models (classi-
fiers). In addition, it makes it possible to state a learning problem as a computational
problem. The the known prior (meta-)knowledge about the problem domain is captured
in a measure function that, to each possible combination of a training set and a classifier,
assigns a value describing how good the classifier is. The computational problem is then
to find a classifier maximizing the measure function. We argue that measure functions are
of great value for practical applications. Besides of being a tool for model selection, they:
(i) force us to make explicit the relevant prior knowledge about the learning problem at
hand, (ii) provide a deeper understanding of existing algorithms, and (iii) help us in the
construction of problem-specific algorithms. We illustrate the last point by suggesting a