| | |
Summary: Entropy Gradient: A Technique for Estimating
The Entropy of Finite Time Series
ADEL S. ATAWY AHMED A. BELAL
a_atawy@acm.org aabelal@yahoo.com
Department of Computer Science
Faculty of Engineering, University of Alexandria
Alexandria
EGYPT
Abstract: - Several techniques are used to estimate the entropy of a finite sequence of symbols taken from a finite
alphabet. Here we will present a new technique based on using varying levels of approximations for the entropy.
Fitting the different approximations into one of a set of functions will result in an estimate to the absolute entropy.
Applications of the entropy gradient technique include estimating the real memory span of finite memory machines
and investigating the degree of compositeness of numbers.
Keywords: - entropy, estimation, gradient, prefix tree, suffix tree, finite memory machines, number properties.
1 Introduction and Preliminaries
The Information Theoretic version of entropy was
first proposed in its modern form by C. E. Shannon
[13] in a statistical form and by Kolmogorov in an
algorithmic theoretic view [8].
The Shannon Entropy of a data sequence is a highly
|
