Complexity Optimized Data Clustering by Competitive Neural Networks
- 1 January 1993
- journal article
- Published by MIT Press in Neural Computation
- Vol. 5 (1) , 75-88
- https://doi.org/10.1162/neco.1993.5.1.75
Abstract
Data clustering is a complex optimization problem with applications ranging from vision and speech processing to data transmission and data storage in technical as well as in biological systems. We discuss a clustering strategy that explicitly reflects the tradeoff between simplicity and precision of a data representation. The resulting clustering algorithm jointly optimizes distortion errors and complexity costs. A maximum entropy estimation of the clustering cost function yields an optimal number of clusters, their positions, and their cluster probabilities. Our approach establishes a unifying framework for different clustering methods like K-means clustering, fuzzy clustering, entropy constrained vector quantization, or topological feature maps and competitive neural networks.Keywords
This publication has 16 references indexed in Scilit:
- Self-organizing maps: ordering, convergence properties and energy functionsBiological Cybernetics, 1992
- Current-mode subthreshold MOS circuits for analog VLSI neural systemsIEEE Transactions on Neural Networks, 1991
- A study of vector quantization for noisy channelsIEEE Transactions on Information Theory, 1990
- Minimum class entropy: A maximum information approach to layered networksNeural Networks, 1989
- Entropy-constrained vector quantizationIEEE Transactions on Acoustics, Speech, and Signal Processing, 1989
- An analogue approach to the travelling salesman problem using an elastic net methodNature, 1987
- Optimization by Simulated AnnealingScience, 1983
- A Convergence Theorem for the Fuzzy ISODATA Clustering AlgorithmsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,1980
- Asymptotically efficient quantizingIEEE Transactions on Information Theory, 1968
- Information Theory and Statistical MechanicsPhysical Review B, 1957