Summary: Infinite Ensemble Learning with
Support Vector Machines
Hsuan­Tien Lin and Ling Li
Learning Systems Group, California Institute of Technology, USA
htlin@caltech.edu, ling@caltech.edu
Abstract. Ensemble learning algorithms such as boosting can achieve
better performance by averaging over the predictions of base hypothe­
ses. However, existing algorithms are limited to combining only a finite
number of hypotheses, and the generated ensemble is usually sparse. It
is not clear whether we should construct an ensemble classifier with a
larger or even infinite number of hypotheses. In addition, constructing an
infinite ensemble itself is a challenging task. In this paper, we formulate
an infinite ensemble learning framework based on SVM. The framework
can output an infinite and nonsparse ensemble, and can be used to con­
struct new kernels for SVM as well as to interpret some existing ones.
We demonstrate the framework with a concrete application, the stump
kernel, which embodies infinitely many decision stumps. The stump ker­
nel is simple, yet powerful. Experimental results show that SVM with
the stump kernel is usually superior than boosting, even with noisy data.
1 Introduction

Source: Abu-Mostafa, Yaser S. - Department of Mechanical Engineering & Computer Science Department, California Institute of Technology

Collections: Computer Technologies and Information Sciences