Forward stagewise regression and the monotone lasso
Top Cited Papers
Open Access
- 1 January 2007
- journal article
- Published by Institute of Mathematical Statistics in Electronic Journal of Statistics
- Vol. 1 (none) , 1-29
- https://doi.org/10.1214/07-ejs004
Abstract
We consider the least angle regression and forward stagewise algorithms for solving penalized least squares regression problems. In Efron, Hastie, Johnstone & Tibshirani (2004) it is proved that the least angle regression algorithm, with a small modification, solves the lasso regression problem. Here we give an analogous result for incremental forward stagewise regression, showing that it solves a version of the lasso problem that enforces monotonicity. One consequence of this is as follows: while lasso makes optimal progress in terms of reducing the residual sum-of-squares per unit increase in L1-norm of the coefficient β, forward stage-wise is optimal per unit L1 arc-length traveled along the coefficient path. We also study a condition under which the coefficient paths of the lasso are monotone, and hence the different algorithms coincide. Finally, we compare the lasso and forward stagewise procedures in a simulation study involving a large number of correlated predictors.Keywords
All Related Versions
This publication has 0 references indexed in Scilit: