Some theory for Fisher's linear discriminant function, `naive Bayes', and some alternatives when there are many more variables than observations
Top Cited Papers
Open Access
- 1 December 2004
- journal article
- Published by Bernoulli Society for Mathematical Statistics and Probability in Bernoulli
- Vol. 10 (6) , 989-1010
- https://doi.org/10.3150/bj/1106314847
Abstract
We show that the 'naive Bayes' classifier which assumes independent covariates greatly outperforms the Fisher linear discriminant rule under broad conditions when the number of variables grows faster than the number of observations, in the classical problem of discriminating between two normal populations. We also introduce a class of rules spanning the range between independence and arbitrary dependence. These rules are shown to achieve Bayes consistency for the Gaussian 'coloured noise' model and to adapt to a spectrum of convergence rates, which we conjecture to be minimax.Keywords
This publication has 7 references indexed in Scilit:
- Persistence in high-dimensional linear predictor selection and the virtue of overparametrizationBernoulli, 2004
- Comparison of Discrimination Methods for the Classification of Tumors Using Gene Expression DataJournal of the American Statistical Association, 2002
- Naive (Bayes) at forty: The independence assumption in information retrievalPublished by Springer Nature ,1998
- On the Optimality of the Simple Bayesian Classifier under Zero-One LossMachine Learning, 1997
- Lectures on Operator Theory and Its ApplicationsPublished by American Mathematical Society (AMS) ,1995
- Constructive ApproximationPublished by Springer Nature ,1993
- Discriminant Analysis and Statistical Pattern RecognitionWiley Series in Probability and Statistics, 1992