Stochastic competitive learning
- 1 January 1991
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks
- Vol. 2 (5) , 522-529
- https://doi.org/10.1109/72.134289
Abstract
Competitive learning systems are examined as stochastic dynamical systems. This includes continuous and discrete formulations of unsupervised, supervised, and differential competitive learning systems. These systems estimate an unknown probability density function from random pattern samples and behave as adaptive vector quantizers. Synaptic vectors, in feedforward competitive neural networks, quantize the pattern space and converge to pattern class centroids or local probability maxima. A stochastic Lyapunov argument shows that competitive synaptic vectors converge to centroids exponentially quickly and reduces competitive learning to stochastic gradient descent. Convergence does not depend on a specific dynamical model of how neuronal activations change. These results extend to competitive estimation of local covariances and higher order statistics.Keywords
This publication has 18 references indexed in Scilit:
- Differential competitive learning for centroid estimation and phoneme recognitionIEEE Transactions on Neural Networks, 1991
- Unsupervised learning in noiseIEEE Transactions on Neural Networks, 1990
- Hidden patterns in combined and adaptive knowledge networksInternational Journal of Approximate Reasoning, 1988
- Counterpropagation networksApplied Optics, 1987
- Adaptive bidirectional associative memoriesApplied Optics, 1987
- A drive-reinforcement model of single neuron function: An alternative to the Hebbian neuronal modelAIP Conference Proceedings, 1986
- Differential Hebbian learningAIP Conference Proceedings, 1986
- On learning and energy-entropy dependence in recurrent and nonrecurrent signed networksJournal of Statistical Physics, 1969
- Quantizing for minimum distortionIEEE Transactions on Information Theory, 1960
- On a Pooling Problem from the Statistical Decision ViewpointEconometrica, 1953