Classification trees with neural network feature extraction
- 1 November 1992
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks
- Vol. 3 (6) , 923-933
- https://doi.org/10.1109/72.165594
Abstract
The ideal use of small multilayer nets at the decision nodes of a binary classification tree to extract nonlinear features is proposed. The nets are trained and the tree is grown using a gradient-type learning algorithm in the multiclass case. The method improves on standard classification tree design methods in that it generally produces trees with lower error rates and fewer nodes. It also reduces the problems associated with training large unstructured nets and transfers the problem of selecting the size of the net to the simpler problem of finding a tree of the right size. An efficient tree pruning algorithm is proposed for this purpose. Trees constructed with the method and the CART method are compared on a waveform recognition problem and a handwritten character recognition problem. The approach demonstrates significant decrease in error rate and tree size. It also yields comparable error rates and shorter training times than a large multilayer net trained with backpropagation on the same problems.<>Keywords
This publication has 20 references indexed in Scilit:
- Learning and convergence analysis of neural-type structured networksIEEE Transactions on Neural Networks, 1992
- A tree-structured adaptive network for function approximation in high-dimensional spacesIEEE Transactions on Neural Networks, 1991
- An iterative growing and pruning algorithm for classification tree designPublished by Institute of Electrical and Electronics Engineers (IEEE) ,1991
- Convergence of learning algorithms with constant learning ratesIEEE Transactions on Neural Networks, 1991
- Analysis of gradient descent learning algorithms for multilayer feedforward neural networksIEEE Transactions on Circuits and Systems, 1991
- Automated design of linear tree classifiersPattern Recognition, 1990
- A performance comparison of trained multilayer perceptrons and trained classification treesProceedings of the IEEE, 1990
- Entropy nets: from decision trees to neural networksProceedings of the IEEE, 1990
- An adaptive least squares algorithm for the efficient training of artificial neural networksIEEE Transactions on Circuits and Systems, 1989
- An Analysis of Variance for Categorical DataJournal of the American Statistical Association, 1971