A Neural Network Construction Algorithm which Maximizes the Likelihood Function

Publisher Website
Google Scholar
Abstract
A new method for constructing a feedforward neural network is proposed. The method starts with a single hidden unit and more units are added to the hidden layer one at a time until a network that completely recognizes all its input patterns is constructed. The novel idea about this method is that the network is trained to maximize a certain likelihood function and not to minimize the more widely used mean squared error function. We show that when a new hidden unit is added to the network, this likelihood function is guaranteed to increase and this increase ensures the finite termination of the method. We also provide a wide range of numerical results. The method was tested on the n -bit parity problems and the spiral problem. It was able to construct networks having less than n hidden units that solve the n -bit parity problems for n = 4, 5, 6, 7 and 8. The method was also tested on some real-world data and the networks it constructed were shown to be able to predict patterns not in the training set with more than 95% accuracy.

This publication has 0 references indexed in Scilit: