Learning in linear neural networks: a survey
- 1 July 1995
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks
- Vol. 6 (4) , 837-858
- https://doi.org/10.1109/72.392248
Abstract
Networks of linear units are the simplest kind of networks, where the basic questions related to learning, generalization, and self-organization can sometimes be answered analytically. We survey most of the known results on linear networks, including: 1) backpropagation learning and the structure of the error function landscape, 2) the temporal evolution of generalization, and 3) unsupervised learning algorithms and their properties. The connections to classical statistical ideas, such as principal component analysis (PCA), are emphasized as well as several simple but challenging open questions. A few new results are also spread across the paper, including an analysis of the effect of noise on backpropagation networks and a unified view of all unsupervised algorithms.Keywords
This publication has 36 references indexed in Scilit:
- Anti-Hebbian learning in topologically constrained linear networks: a tutorialIEEE Transactions on Neural Networks, 1993
- Convergence analysis of local feature extraction algorithmsNeural Networks, 1992
- Generalization in a linear perceptron in the presence of noiseJournal of Physics A: General Physics, 1992
- Convergence of learning algorithms with constant learning ratesIEEE Transactions on Neural Networks, 1991
- Learning from examples in large neural networksPhysical Review Letters, 1990
- Analysis of Linsker's Simulations of Hebbian RulesNeural Computation, 1990
- NEURAL NETWORKS, PRINCIPAL COMPONENTS, AND SUBSPACESInternational Journal of Neural Systems, 1989
- On stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrixJournal of Mathematical Analysis and Applications, 1985
- Simplified neuron model as a principal component analyzerJournal of Mathematical Biology, 1982
- The Use of Control Systems Analysis in the Neurophysiology of Eye MovementsAnnual Review of Neuroscience, 1981