A class of unconstrained minimization methods for neural network training
- 1 January 1994
- journal article
- research article
- Published by Taylor & Francis in Optimization Methods and Software
- Vol. 4 (2) , 135-150
- https://doi.org/10.1080/10556789408805583
Abstract
In this paper the problem of neural network training is formulated as the unconstrained minimization of a sum of differentiate error terms on the output space. For problems of this form we consider solution algorithms of the backpropagation-type, where the gradient evaluation is split into different steps, and we state sufficient convergence conditions that exploit the special structure of the objective function. Then we define a globally convergent algorithm that uses the knowledge of the overall error function for the computation of the learning rates. Potential advantages and possible shortcomings of this approach, in comparison with alternative approaches are discussed.Keywords
This publication has 11 references indexed in Scilit:
- A class of unconstrained minimization methods for neural network trainingOptimization Methods and Software, 1994
- Mathematical Programming in Neural NetworksINFORMS Journal on Computing, 1993
- Theory of algorithms for unconstrained optimizationActa Numerica, 1992
- Uncertainty modelling and structured singular-value computation applied to an electromechanical systemIEE Proceedings D Control Theory and Applications, 1992
- Truncated Newton method for sparse unconstrained optimization using automatic differentiationJournal of Optimization Theory and Applications, 1989
- Accelerating the convergence of the back-propagation methodBiological Cybernetics, 1988
- Global convergence and stabilization of unconstrained minimization methods without derivativesJournal of Optimization Theory and Applications, 1988
- A Nonmonotone Line Search Technique for Newton’s MethodSIAM Journal on Numerical Analysis, 1986
- Parallel Distributed ProcessingPublished by MIT Press ,1986
- Stopping criteria for linesearch methods without derivativesMathematical Programming, 1984