 
Summary: A Lower Bound on the Expected Length of 11 Codes
Noga Alon
Alon Orlitsky
February 22, 2002
Abstract
We show that the minimum expected length of a 11 encoding of a discrete random variable
X is at least1
H(X)log(H(X)+1)log e and that this bound is asymptotically achievable.
1 Introduction
Let X be a random variable distributed over a countable support set X. A (binary, 11)
encoding of X is an injection : X {0,1}
, the set of finite binary strings. The expected
number of bits uses to encode X is
l()
def
=
xX
Pr(x)(x)
where Pr(x) is the probability that X = x and (x) is the length of (x).
A string x1, . . . ,xm is a prefix of a string y1, . . . ,yn if m n and xi = yi for i = 1, . . . ,m.
