A Relationship between Linear Discriminant Analysis and the Generalized Minimum Squared Error Solution
- 1 January 2005
- journal article
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Matrix Analysis and Applications
- Vol. 27 (2) , 474-492
- https://doi.org/10.1137/040607599
Abstract
In this paper, a relationship between linear discriminant analysis (LDA) and the generalized minimum squared error (MSE) solution is presented. The generalized MSE solution is shown to be equivalent to applying a certain classification rule in the space defined by LDA. The relationship between the MSE solution and Fisher discriminant analysis is extended to multiclass problems and also to undersampled problems for which the classical LDA is not applicable due to singularity of the scatter matrices. In addition, an efficient algorithm for LDA is proposed exploiting its relationship with the MSE procedure. Extensive experiments verify the theoretical results.Keywords
This publication has 15 references indexed in Scilit:
- What's wrong with Fisher criterion?Pattern Recognition, 2002
- A direct LDA algorithm for high-dimensional data — with application to face recognitionPublished by Elsevier ,2001
- A new LDA-based face recognition system which can solve the small sample size problemPattern Recognition, 2000
- Penalized Discriminant AnalysisThe Annals of Statistics, 1995
- Flexible Discriminant Analysis by Optimal ScoringJournal of the American Statistical Association, 1994
- Regularized Discriminant AnalysisJournal of the American Statistical Association, 1989
- CANONICAL VARIATE ANALYSIS—A GENERAL MODEL FORMULATIONAustralian Journal of Statistics, 1984
- The use of an adaptive threshold element to design a linear optimal pattern classifierIEEE Transactions on Information Theory, 1966
- An Algorithm for Linear Inequalities and its ApplicationsIEEE Transactions on Electronic Computers, 1965
- THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMSAnnals of Eugenics, 1936