Learning and convergence analysis of neural-type structured networks
- 1 January 1992
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks
- Vol. 3 (1) , 39-50
- https://doi.org/10.1109/72.105416
Abstract
A class of feedforward neural networks, structured networks, has recently been introduced as a method for solving matrix algebra problems in an inherently parallel formulation. A convergence analysis for the training of structured networks is presented. Since the learning techniques used in structured networks are also employed in the training of neural networks, the issue of convergence is discussed not only from a numerical algebra perspective but also as a means of deriving insight into connectionist learning. Bounds on the learning rate are developed under which exponential convergence of the weights to their correct values is proved for a class of matrix algebra problems that includes linear equation solving, matrix inversion, and Lyapunov equation solving. For a special class of problems, the orthogonalized back-propagation algorithm, an optimal recursive update law for minimizing a least-squares cost functional, is introduced. It guarantees exact convergence in one epoch. Several learning issues are investigated.Keywords
This publication has 11 references indexed in Scilit:
- A neural-type parallel algorithm for fast matrix inversionPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Cumulant-based parameter estimation using structured networksIEEE Transactions on Neural Networks, 1991
- Artificial neural networks, back propagation, and the Kelley-Bryson gradient procedureJournal of Guidance, Control, and Dynamics, 1990
- Identification and control of dynamical systems using neural networksIEEE Transactions on Neural Networks, 1990
- Matrix computations and equation solving using structured networks and trainingPublished by Institute of Electrical and Electronics Engineers (IEEE) ,1990
- Structured trainable networks for matrix algebraPublished by Institute of Electrical and Electronics Engineers (IEEE) ,1990
- On the approximate realization of continuous mappings by neural networksNeural Networks, 1989
- Multilayer feedforward networks are universal approximatorsNeural Networks, 1989
- Increased rates of convergence through learning rate adaptationNeural Networks, 1988
- Adaptive Systems with Reduced ModelsPublished by Springer Nature ,1983