Advanced Search

Browse by Discipline

Scientific Societies

E-print Alerts

Add E-prints

E-print Network

  Advanced Search  

Infinite Ensemble Learning with Support Vector Machines

Summary: Infinite Ensemble Learning with
Support Vector Machines
Hsuan-Tien Lin and Ling Li
Learning Systems Group, California Institute of Technology, USA
htlin@caltech.edu, ling@caltech.edu
Abstract. Ensemble learning algorithms such as boosting can achieve
better performance by averaging over the predictions of base hypothe-
ses. However, existing algorithms are limited to combining only a finite
number of hypotheses, and the generated ensemble is usually sparse. It
is not clear whether we should construct an ensemble classifier with a
larger or even infinite number of hypotheses. In addition, constructing an
infinite ensemble itself is a challenging task. In this paper, we formulate
an infinite ensemble learning framework based on SVM. The framework
can output an infinite and nonsparse ensemble, and can be used to con-
struct new kernels for SVM as well as to interpret some existing ones.
We demonstrate the framework with a concrete application, the stump
kernel, which embodies infinitely many decision stumps. The stump ker-
nel is simple, yet powerful. Experimental results show that SVM with
the stump kernel is usually superior than boosting, even with noisy data.
1 Introduction


Source: Abu-Mostafa, Yaser S. - Department of Mechanical Engineering & Computer Science Department, California Institute of Technology


Collections: Computer Technologies and Information Sciences