A generalized Dantzig selector with shrinkage tuning
- 20 May 2009
- journal article
- research article
- Published by Oxford University Press (OUP) in Biometrika
- Vol. 96 (2) , 323-337
- https://doi.org/10.1093/biomet/asp013
Abstract
The Dantzig selector performs variable selection and model fitting in linear regression. It uses an L1 penalty to shrink the regression coefficients towards zero, in a similar fashion to the lasso. While both the lasso and Dantzig selector potentially do a good job of selecting the correct variables, they tend to overshrink the final coefficients. This results in an unfortunate trade-off. One can either select a high shrinkage tuning parameter that produces an accurate model but poor coefficient estimates or a low shrinkage parameter that produces more accurate coefficients but includes many irrelevant variables. We extend the Dantzig selector to fit generalized linear models while eliminating overshrinkage of the coefficient estimates, and develop a computationally efficient algorithm, similar in nature to least angle regression, to compute the entire path of coefficient estimates. A simulation study illustrates the advantages of our approach relative to others. We apply the methodology to two datasets.Keywords
This publication has 12 references indexed in Scilit:
- Variable Inclusion and Shrinkage AlgorithmsJournal of the American Statistical Association, 2008
- DASSO: Connections Between the Dantzig Selector and LassoJournal of the Royal Statistical Society Series B: Statistical Methodology, 2008
- Discussion: The Dantzig selector: Statistical estimation when p is much larger than nThe Annals of Statistics, 2007
- Relaxed LassoComputational Statistics & Data Analysis, 2007
- L1-Regularization Path Algorithm for Generalized Linear ModelsJournal of the Royal Statistical Society Series B: Statistical Methodology, 2007
- The Adaptive Lasso and Its Oracle PropertiesJournal of the American Statistical Association, 2006
- Regularization and Variable Selection Via the Elastic NetJournal of the Royal Statistical Society Series B: Statistical Methodology, 2005
- Least angle regressionThe Annals of Statistics, 2004
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle PropertiesJournal of the American Statistical Association, 2001
- Atomic Decomposition by Basis PursuitSIAM Journal on Scientific Computing, 1998