Feature Selection in MLPs and SVMs Based on Maximum Output Information

12 July 2004

journal article
research article
Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks

Vol. 15 (4) , 937-948
https://doi.org/10.1109/tnn.2004.828772

Abstract

This paper presents feature selection algorithms for multilayer perceptrons (MLPs) and multiclass support vector machines (SVMs), using mutual information between class labels and classifier outputs, as an objective function. This objective function involves inexpensive computation of information measures only on discrete variables; provides immunity to prior class probabilities; and brackets the probability of error of the classifier. The maximum output information (MOI) algorithms employ this function for feature subset selection by greedy elimination and directed search. The output of the MOI algorithms is a feature subset of user-defined size and an associated trained classifier (MLP/SVM). These algorithms compare favorably with a number of other methods in terms of performance on various artificial and real-world data sets.

Keywords

This publication has 13 references indexed in Scilit:

The ANNIGMA-wrapper approach to fast feature selection for neural nets
IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2002
Information Theoretic Feature Crediting in Multiclass Support Vector Machines
Published by Society for Industrial & Applied Mathematics (SIAM) ,2001
Neural-network feature selector
IEEE Transactions on Neural Networks, 1997
Feature selection for classification
Intelligent Data Analysis, 1997
Using mutual information for selecting features in supervised neural net learning
IEEE Transactions on Neural Networks, 1994
Bias in information-based measures in decision tree induction
Machine Learning, 1994
Irrelevant Features and the Subset Selection Problem
Published by Elsevier ,1994
Connectionist learning procedures
Artificial Intelligence, 1989
Self-organization in a perceptual network
Computer, 1988
Transmission of Information: A Statistical Theory of Communications
American Journal of Physics, 1961