Advanced Search

Browse by Discipline

Scientific Societies

E-print Alerts

Add E-prints

E-print Network

  Advanced Search  

Novel Distance-Based SVM Kernels for Infinite Ensemble Learning

Summary: Novel Distance-Based SVM Kernels for
Infinite Ensemble Learning
Hsuan-Tien Lin and Ling Li
htlin@caltech.edu, ling@caltech.edu
Learning Systems Group, California Institute of Technology, USA
Abstract-- Ensemble learning algorithms such as boosting can
achieve better performance by averaging over the predictions of
base hypotheses. However, most existing algorithms are limited to
combining only a finite number of hypotheses, and the generated
ensemble is usually sparse. It has recently been shown that the
support vector machine (SVM) with a carefully crafted kernel
can be used to construct a nonsparse ensemble of infinitely
many hypotheses. Such infinite ensembles may surpass finite
and/or sparse ensembles in learning performance and robustness.
In this paper, we derive two novel kernels, the stump kernel
and the perceptron kernel, for infinite ensemble learning. The
stump kernel embodies an infinite number of decision stumps,
and measures the similarity between examples by the 1-norm
distance. The perceptron kernel embodies perceptrons, and works
with the 2-norm distance. Experimental results show that SVM


Source: Abu-Mostafa, Yaser S. - Department of Mechanical Engineering & Computer Science Department, California Institute of Technology


Collections: Computer Technologies and Information Sciences