Advanced Search

Browse by Discipline

Scientific Societies

E-print Alerts

Add E-prints

E-print Network

  Advanced Search  

LETTER Communicated by David Wolpert No Free Lunch for Early Stopping

Summary: LETTER Communicated by David Wolpert
No Free Lunch for Early Stopping
Zehra Cataltepe
Yaser S. Abu-Mostafa
Bell Laboratories, Lucent Technologies, 600 Mountain Ave., Rm. 2C265, Murray Hill,
NJ 07974, U.S.A.
Malik Magdon-Ismail
Learning Systems Group, California Institute of Technology, MC 13693, Pasadena,
CA 91125, U.S.A.
We show that with a uniform prior on models having the same training
error, early stopping at some fixed training error above the training error
minimum results in an increase in the expected generalization error.
1 Introduction
Early stopping of training is one of the methods that aim to prevent over-
training due to too powerful a model class, noisy training examples, or a
small training set. We study early stopping at a predetermined training er-
ror level. If there is no prior information other than the training examples,
all models with the same training error should be equally likely to be cho-
sen as the early stopping solution. When this is the case, we show that for
general linear models, early stopping at any training error level above the


Source: Abu-Mostafa, Yaser S. - Department of Mechanical Engineering & Computer Science Department, California Institute of Technology


Collections: Computer Technologies and Information Sciences