Summary: The Third Claude E. Shannon
April 29, 2005
Are There Turbo-Codes on Mars?
Robert J. McEliece
California Institute of Technology
"The fundamental problem of communication is that of
reproducing at one point either exactly or approximately a
message selected at another point."
To solve this problem, Shannon created a branch of applied
mathematics which is today called
Relative entropy H(X|Y) measures our
uncertainty about X after Y is observed
H(X) = -
p(x) log p(x)