Efficient algorithms for function approximation with piecewise linear sigmoidal networks
- 1 January 1998
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks
- Vol. 9 (6) , 1129-1141
- https://doi.org/10.1109/72.728357
Abstract
This paper presents a computationally efficient algorithm for function approximation with piecewise linear sigmoidal nodes. A one hidden layer network is constructed one node at a time using the well-known method of fitting the residual. The task of fitting an individual node is accomplished using a new algorithm that searches for the best fit by solving a sequence of quadratic programming problems. This approach offers significant advantages over derivative-based search algorithms (e.g., backpropagation and its extensions). Unique characteristics of this algorithm include: finite step convergence, a simple stopping criterion, solutions that are independent of initial conditions, good scaling properties and a robust numerical implementation. Empirical results are included to illustrate these characteristicsKeywords
This publication has 28 references indexed in Scilit:
- Comparison of adaptive methods for function estimation from samplesIEEE Transactions on Neural Networks, 1996
- A note on error bounds for approximation in inner product spacesCircuits, Systems, and Signal Processing, 1996
- The Complexity of Approximating a Nonlinear ProgramPublished by World Scientific Pub Co Pte Ltd ,1993
- Hinging hyperplanes for regression, classification, and function approximationIEEE Transactions on Information Theory, 1993
- A Simple Lemma on Greedy Approximation in Hilbert Space and Convergence Rates for Projection Pursuit Regression and Neural Network TrainingThe Annals of Statistics, 1992
- First- and Second-Order Methods for Learning: Between Steepest Descent and Newton's MethodNeural Computation, 1992
- A Proposal for More Powerful Learning AlgorithmsNeural Computation, 1989
- Algorithms in Combinatorial GeometryPublished by Springer Nature ,1987
- The polynomial solvability of convex quadratic programmingUSSR Computational Mathematics and Mathematical Physics, 1980
- Computationally Related ProblemsSIAM Journal on Computing, 1974