 
Summary: Mathematics and Engineering
Communications Laboratory
Technical Report
A Formula for the KullbackLeibler Divergence
Rate Between Discrete Markov Sources
Z. Rached, F. Alajaji, and L. L. Campbell
November 2003
1
A Formula for the KullbackLeibler
Divergence Rate Between Discrete
Markov Sources
Ziad Rached Fady Alajaji L. Lorne Campbell
Abstract
In this work, we provide a computable expression for the KullbackLeibler
divergence rate, lim n!1 1
n D(p (n) kq
(n) ), between two timeinvariant nitealpha
bet Markov sources of arbitrary order and arbitrary initial distributions de
scribed by the probability distributions p (n) and q (n) , respectively. We illus
