Willshaw model: Associative memory with sparse coding and low firing rates

The Willshaw model of associative memory, implemented in a fully connected network with stochastic asynchronous dynamics, is studied. In addition to Willshaw’s learning rule, the network contains uniform synaptic inhibition, of relative strength K, and negative neural threshold -θ,θ>0. The P stored memories are sparsely coded. The total number of on bits in each memory is Nf, where f is much smaller than 1 but much larger than lnN/N. Mean-field theory of the system is solved in the limit where C==exp(-$f^2$P) is finite. Memory states are stable (at zero temperature), as long as C>$h_0$==K-1+θ and $h_0$>0. When C$h_0$ or $h_0$P retrieval phases, highly correlated with the memory states, exist. These phases are only partially frozen at low temperature, so that the full memories can be retrieved from them by averaging over the dynamic fluctuations of the neural activity. In particular, when $h_0$P for which stable retrieval phases exist, scales as $f^{\mathrm{-}3}$/‖lnf‖ for f≫1/lnN, and as $f^{\mathrm{-}2}$ln(Nf/‖lnf‖) for f≪1/lnN. Numerical simulations of the model with N=1000 and f=0.04 are presented. We also discuss the possible realization of the model in a biologically plausible architecture, where the inhibition is provided by special inhibitory neurons.