 
Summary: Progressive mixture rules are deviation suboptimal
JeanYves Audibert
Willow Project  Certis Lab
ParisTech, Ecole des Ponts
77455 MarnelaVall´ee, France
audibert@certis.enpc.fr
Abstract
We consider the learning task consisting in predicting as well as the best function
in a finite reference set G up to the smallest possible additive term. If R(g) denotes
the generalization error of a prediction function g, under reasonable assumptions
on the loss function (typically satisfied by the least square loss when the output is
bounded), it is known that the progressive mixture rule ^g satisfies
ER(^g) mingG R(g) + Cst log G
n , (1)
where n denotes the size of the training set, and E denotes the expectation w.r.t.
the training set distribution.This work shows that, surprisingly, for appropriate
reference sets G, the deviation convergence rate of the progressive mixture rule is
no better than Cst /
n: it fails to achieve the expected Cst /n. We also provide
