A new class of quasi-newtonian methods for optimal learning in mlp-networks
- 26 March 2003
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks
- Vol. 14 (2) , 263-273
- https://doi.org/10.1109/tnn.2003.809425
Abstract
In this paper, we present a new class of quasi-Newton methods for an effective learning in large multilayer perceptron (MLP)-networks. The algorithms introduced in this work, named LQN, utilize an iterative scheme of a generalized BFGS-type method, involving a suitable family of matrix algebras L. The main advantages of these innovative methods are based upon the fact that they have an O(nlogn) complexity per step and that they require O(n) memory allocations. Numerical experiences, performed on a set of standard benchmarks of MLP-networks, show the competitivity of the LQN methods, especially for large values of n.Keywords
This publication has 28 references indexed in Scilit:
- Numerical OptimizationPublished by Springer Nature ,1999
- Optimal trigonometric preconditioners for nonsymmetric Toeplitz systemsLinear Algebra and its Applications, 1998
- Optimal and Superoptimal Circulant PreconditionersSIAM Journal on Matrix Analysis and Applications, 1992
- Terminal attractor learning algorithms for back propagation neural networksPublished by Institute of Electrical and Electronics Engineers (IEEE) ,1991
- Accelerating the convergence of the back-propagation methodBiological Cybernetics, 1988
- Conversion of FFT’s to Fast Hartley TransformsSIAM Journal on Scientific and Statistical Computing, 1986
- On computing the discrete Hartley transformIEEE Transactions on Acoustics, Speech, and Signal Processing, 1985
- Fast algorithms for the discrete W transform and for the discrete Fourier transformIEEE Transactions on Acoustics, Speech, and Signal Processing, 1984
- Updating Quasi-Newton Matrices with Limited StorageMathematics of Computation, 1980
- The Convergence of a Class of Double-rank Minimization AlgorithmsIMA Journal of Applied Mathematics, 1970