 
Summary: Precise capacity analysis in binary networks with
multiple coding level inputs
Yali Amit
and Yibi Huang
October 13, 2009
Abstract
We compute retrieval probabilities as a function of pattern age for networks
with binary neurons and synapses updated with the simple Hebbian learning model
studied in (Amit & Fusi 1994). The analysis depends on choosing a neural thresh
old that enables patterns to stabilize in the neural dynamics. In contrast to most
earlier work where selective neurons for each pattern are drawn independently
with fixed probability f, here we analyze the situation where f is drawn from
some distribution on a range of coding levels. In order to set a workable threshold
in this setting it is necessary to introduce a simple inhibition in the neural dynam
ics whose magnitude depends on the total activity of the network. Proper choice
of the threshold depends on the value of the covariances between the synapses
for which we provide an explicit formula. Retrieval probabilities depend on the
distribution of the fields induced by a learned pattern. We show that the field
induced by the first learned pattern evolves as a Markov chain during subsequent
learning epochs, leading to a recursive formula for the distribution. Alternatively
