 
Summary: IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. IT32, NO. 4, JULY 1986 513
The Complexity of Information Extraction
YASER S. ABUMOSTAFA
This paper is dedicated to the memory of Herbert J. Ryser.
AbstractHow difficult are decision problems based on natural data,
such as pattern recognition? To answer this question, decision problems
are characterized by introducing four measures defined on a Boolean
function f of N variables: the implementation cost C(f), the randomness
R(f), the deterministic entropy H(f), and the complexity K(f). The
highlights and main results are roughly as follows. 1) C(f) = R(f) =
H( f ) = K( f ), all measured in bits. 2) Decision problems based on natural
data are partially random (in the Kolmogorov sense) and have low entropy
with respect to their dimensionality, and the relations between the four
measures translate to lower and upper bounds on the cost of solving these
problems. 3) Allowing small errors in the implementation of f saves a lot
in the low entropy case but saves nothing in the highentropy case. If f is
partially structured, the implementation cost is reduced substantially.
I. INTRODUCTION
T HE ACCESSIBILITY of available information is the
central issue in decisionmaking based on natural
