 
Summary: Estimating the Posterior Probabilities
Using the KNearest Neighbor Rule
Amir F. Atiya
Dept Computer Engineering
Cairo University
Giza, Egypt
amir@alumni.caltech.edu
July 19, 2004
Abstract
In many pattern classification problems an estimate of the posterior prob
abilities (rather than only a classification) is required. This is usually the case
when some confidence measure in the classification is needed. In this paper
we propose a new posterior probability estimator. The proposed estimator
considers the Knearest neighbors. It attaches a weight to each neighbor that
contributes in an additive fashion to the posterior probability estimate. The
weights corresponding to the Knearestneighbors (which add to 1) are esti
mated from the data using a maximum likelihood approach. Simulation studies
confirm the effectiveness of the proposed estimator.
1 Introduction
The posterior probability a key variable in any pattern classification problem. For
