Capacity of neural networks with discrete synaptic couplings
- 21 June 1990
- journal article
- Published by IOP Publishing in Journal of Physics A: General Physics
- Vol. 23 (12) , 2613-2630
- https://doi.org/10.1088/0305-4470/23/12/036
Abstract
The authors study the optimal storage capacity of neural networks with discrete local constraints on the synaptic couplings Jij. Models with such constraints include those with binary couplings Jij=+or-1 or Jij=0, 1, quantised couplings with larger synaptic range, e.g. Jij=+or-1/L, +or-2/L, . . ., +or-1 and, in the limit, continuous couplings confined to the hypercube mod Jij mod <or=1 ('box confinement'). They find that the optimal storage capacity alpha ( kappa ) is best determined by the vanishing of a suitably defined 'entropy' as calculated in the replica symmetric approximation. They also extend their results to cases with biased memories and make contact with sparse coding models.Keywords
This publication has 14 references indexed in Scilit:
- Willshaw model: Associative memory with sparse coding and low firing ratesPhysical Review A, 1990
- Perceptron learning with sign-constrained weightsJournal of Physics A: General Physics, 1989
- Three unfinished works on the optimal storage capacity of networksJournal of Physics A: General Physics, 1989
- Storage capacity of memory networks with binary couplingsJournal de Physique, 1989
- Optimal storage properties of neural network modelsJournal of Physics A: General Physics, 1988
- The space of interactions in neural network modelsJournal of Physics A: General Physics, 1988
- Learning algorithms with optimal stability in neural networksJournal of Physics A: General Physics, 1987
- Learning of correlated patterns in spin-glass networks by local learning rulesPhysical Review Letters, 1987
- Neural networks and physical systems with emergent collective computational abilities.Proceedings of the National Academy of Sciences, 1982
- Stability of the Sherrington-Kirkpatrick solution of a spin glass modelJournal of Physics A: General Physics, 1978