Home

About

Advanced Search

Browse by Discipline

Scientific Societies

E-print Alerts

Add E-prints

E-print Network
FAQHELPSITE MAPCONTACT US


  Advanced Search  

 
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. IT-32, NO. 4, JULY 1986 513 The Complexity of Information Extraction
 

Summary: IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. IT-32, NO. 4, JULY 1986 513
The Complexity of Information Extraction
YASER S. ABU-MOSTAFA
This paper is dedicated to the memory of Herbert J. Ryser.
Abstract-How difficult are decision problems based on natural data,
such as pattern recognition? To answer this question, decision problems
are characterized by introducing four measures defined on a Boolean
function f of N variables: the implementation cost C(f), the randomness
R(f), the deterministic entropy H(f), and the complexity K(f). The
highlights and main results are roughly as follows. 1) C(f) = R(f) =
H( f ) = K( f ), all measured in bits. 2) Decision problems based on natural
data are partially random (in the Kolmogorov sense) and have low entropy
with respect to their dimensionality, and the relations between the four
measures translate to lower and upper bounds on the cost of solving these
problems. 3) Allowing small errors in the implementation of f saves a lot
in the low entropy case but saves nothing in the high-entropy case. If f is
partially structured, the implementation cost is reduced substantially.
I. INTRODUCTION
T HE ACCESSIBILITY of available information is the
central issue in decision-making based on natural

  

Source: Abu-Mostafa, Yaser S. - Department of Mechanical Engineering & Computer Science Department, California Institute of Technology

 

Collections: Computer Technologies and Information Sciences