Advanced Search

Browse by Discipline

Scientific Societies

E-print Alerts

Add E-prints

E-print Network

  Advanced Search  

Ordering and Finding the Best of K > 2 Supervised Learning Algorithms

Summary: Ordering and Finding the Best of K > 2
Supervised Learning Algorithms
Olcay Taner Yildiz and Ethem Alpaydin, Senior Member, IEEE
Abstract--Given a data set and a number of supervised learning algorithms, we would like to find the algorithm with the smallest
expected error. Existing pairwise tests allow a comparison of two algorithms only; range tests and ANOVA check whether multiple
algorithms have the same expected error and cannot be used for finding the smallest. We propose a methodology, the MultiTest
algorithm, whereby we order supervised learning algorithms taking into account 1) the result of pairwise statistical tests on expected
error (what the data tells us), and 2) our prior preferences, e.g., due to complexity. We define the problem in graph-theoretic terms and
propose an algorithm to find the "best" learning algorithm in terms of these two criteria, or in the more general case, order learning
algorithms in terms of their "goodness." Simulation results using five classification algorithms on 30 data sets indicate the utility of the
method. Our proposed method can be generalized to regression and other loss functions by using a suitable pairwise test.
Index Terms--Machine learning, classifier design and evaluation, experimental design.
IN machine learning literature, there exist several super-
vised learning algorithms, and for any application, we
need to find the algorithm that generalizes the best, i.e., the
one with the smallest probability of misclassifying an
instance unseen during training. To check for a statistically
significant difference, the algorithms are trained and tested


Source: Alpaydın, Ethem - Department of Computer Engineering, Bogaziçi University


Collections: Computer Technologies and Information Sciences