A learning algorithm for multi-layer perceptrons with hard-limiting threshold units
- 17 December 2002
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 219-228
- https://doi.org/10.1109/nnsp.1994.366045
Abstract
We propose a novel learning algorithm to train networks with multilayer linear-threshold or hard-limiting units. The learning scheme is based on the standard backpropagation, but with "pseudo-gradient" descent, which uses the gradient of a sigmoid function as a heuristic hint in place of that of the hard-limiting function. A justification that the pseudo-gradient always points in the right down hill direction in error surface for networks with one hidden layer is provided. The advantages of such networks are that their internal representations in the hidden layers are clearly interpretable, and well-defined classification rules can be easily obtained, that calculations for classifications after training are very simple, and that they are easily implementable in hardware. Comparative experimental results on several benchmark problems using both the conventional backpropagation networks and our learning scheme for multilayer perceptrons are presented and analyzed.Keywords
This publication has 5 references indexed in Scilit:
- Discrete recurrent neural networks for grammatical inferenceIEEE Transactions on Neural Networks, 1994
- Learning Finite State Machines With Self-Clustering Recurrent NetworksNeural Computation, 1993
- Analysis of hidden units in a layered network trained to classify sonar targetsNeural Networks, 1988
- Parallel Distributed ProcessingPublished by MIT Press ,1986
- THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMSAnnals of Eugenics, 1936