Summary: Large­Margin Thresholded Ensembles for
Ordinal Regression: Theory and Practice
Hsuan­Tien Lin and Ling Li
Learning Systems Group, California Institute of Technology, USA
htlin@caltech.edu, ling@caltech.edu
Abstract. We propose a thresholded ensemble model for ordinal regres­
sion problems. The model consists of a weighted ensemble of confidence
functions and an ordered vector of thresholds. We derive novel large­
margin bounds of common error functions, such as the classification error
and the absolute error. In addition to some existing algorithms, we also
study two novel boosting approaches for constructing thresholded ensem­
bles. Both our approaches not only are simpler than existing algorithms,
but also have a stronger connection to the large­margin bounds. In addi­
tion, they have comparable performance to SVM­based algorithms, but
enjoy the benefit of faster training. Experimental results on benchmark
datasets demonstrate the usefulness of our boosting approaches.
1 Introduction
Ordinal regression resides between multiclass classification and metric regression
in the area of supervised learning. They have many applications in social science
and information retrieval to match human preferences. In an ordinal regression

Source: Abu-Mostafa, Yaser S. - Department of Mechanical Engineering & Computer Science Department, California Institute of Technology

Collections: Computer Technologies and Information Sciences