Pattern retrieval and learning in nets of asynchronous binary threshold elements
- 1 March 1989
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems
- Vol. 36 (3) , 353-364
- https://doi.org/10.1109/31.17581
Abstract
A training algorithm for network of asynchronous learning-threshold elements is presented and analyzed. The algorithm is based on the Hebbian hypothesis, and it allows adaptation of the learning-network parameters to changing pattern environments. In particular, the network's properties can be quantified in environments where pattern occurrence is random, with nonequal, nonstationary probability distributions. The state reassessment probabilities of neurons during information retrieval can also be nonstationary and not equal for all neurons. The trained network is a content-addressable memory. The authors evaluate its stabilization properties with respect to a given set of patterns, using the theory of Markov processes. The results are applicable for the determination of efficient coding for information that has to be stored, and for prediction of actual pattern-retrieval capabilities of the trained network. The authors include the popular sum-of-outer-products assignment as an analyzable specific case of their training procedure, and allow the steady-state analysis of a large group of sigmoidal learning curves.Keywords
This publication has 12 references indexed in Scilit:
- A massively parallel architecture for a self-organizing neural pattern recognition machinePublished by Elsevier ,2005
- Convergence in neural memoriesIEEE Transactions on Information Theory, 1989
- The capacity of the Hopfield associative memoryIEEE Transactions on Information Theory, 1987
- Stochastic Dynamics of Neural NetworksIEEE Transactions on Systems, Man, and Cybernetics, 1986
- Storing Infinite Numbers of Patterns in a Spin-Glass Model of Neural NetworksPhysical Review Letters, 1985
- Spin-glass models of neural networksPhysical Review A, 1985
- Information capacity of the Hopfield modelIEEE Transactions on Information Theory, 1985
- Neural networks and physical systems with emergent collective computational abilities.Proceedings of the National Academy of Sciences, 1982
- Analytic study of the memory storage capacity of a neural networkMathematical Biosciences, 1978
- Learning Patterns and Pattern Sequences by Self-Organizing Nets of Threshold ElementsIEEE Transactions on Computers, 1972