Summary: No Free Lunch for Early Stopping
Zehra Cataltepe
Bell Laboratories, Lucent Technologies
600 Mountain Ave, Rm 2C­265, Murray Hill, NJ 07974, U.S.A.
zehra@lucent.com
Yaser S. Abu­Mostafa
Malik Magdon­Ismail
Learning Systems Group, California Institute of Technology
MC 136­93, Pasadena, CA 91125, U.S.A
fyaser, magdong@cs.caltech.edu
Abstract
We show that, with a uniform prior on models having the same
training error, early stopping at some fixed training error above the
training error minimum results in an increase in the expected general­
ization error.
1 Introduction
Early stopping of training is one of the methods that aim to prevent over­
training due to too powerful a model class, noisy training examples or a
small training set. We study early stopping at a predetermined training er­
ror level. If there is no prior information, other than the training examples,

Source: Abu-Mostafa, Yaser S. - Department of Mechanical Engineering & Computer Science Department, California Institute of Technology
Magdon-Ismail, Malik - Department of Computer Science, Rensselaer Polytechnic Institute

Collections: Computer Technologies and Information Sciences