| | |
Summary: IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 45, NO. 4, APRIL 1997 1051
Conditional Distribution Learning
with Neural Networks and Its
Application to Channel Equalization
T¨ulay Adali, Member, IEEE, Xiao Liu, and M. Kemal S¨onmez
Abstract-- We present a conditional distribution learning
formulation for real-time signal processing with neural
networks based on a recent extension of maximum likelihood
theory--partial likelihood (PL) estimation--which allows for
i) dependent observations and ii) sequential processing. For
a general neural network conditional distribution model, we
establish a fundamental information-theoretic connection, the
equivalence of maximum PL estimation, and accumulated
relative entropy (ARE) minimization, and obtain large sample
properties of PL for the general case of dependent observations.
As an example, the binary case with the sigmoidal perceptron as
the probability model is presented. It is shown that the single
and multilayer perceptron (MLP) models satisfy conditions for
the equivalence of the two cost functions: ARE and negative
log partial likelihood. The practical issue of their gradient
|
