Boosting Algorithms: Regularization, Prediction and Model Fitting
Top Cited Papers
Open Access
- 1 November 2007
- journal article
- research article
- Published by Institute of Mathematical Statistics in Statistical Science
- Vol. 22 (4) , 477-505
- https://doi.org/10.1214/07-sts242
Abstract
We present a statistical perspective on boosting. Special emphasis is given to estimating potentially complex parametric or nonparametric models, including generalized linear and additive models as well as regression models for survival analysis. Concepts of degrees of freedom and corresponding Akaike or Bayesian information criteria, particularly useful for regularization and variable selection in high-dimensional covariate spaces, are discussed as well. The practical aspects of boosting procedures for fitting statistical models are illustrated by means of the dedicated open-source software package mboost. This package implements functions which can be used for model fitting, prediction and variable selection. It is flexible, allowing for the implementation of new boosting algorithms optimizing user-specified loss functions.Keywords
All Related Versions
This publication has 70 references indexed in Scilit:
- On boosting kernel regressionJournal of Statistical Planning and Inference, 2008
- Generalized Smooth Monotonic Regression in Additive ModelingJournal of Computational and Graphical Statistics, 2007
- The Adaptive Lasso and Its Oracle PropertiesJournal of the American Statistical Association, 2006
- Unbiased Recursive Partitioning: A Conditional Inference FrameworkJournal of Computational and Graphical Statistics, 2006
- Boosting for high-dimensional linear modelsThe Annals of Statistics, 2006
- Model Selection and the Principle of Minimum Description LengthJournal of the American Statistical Association, 2001
- A new approach to variable selection in least squares problemsIMA Journal of Numerical Analysis, 2000
- A Decision-Theoretic Generalization of On-Line Learning and an Application to BoostingJournal of Computer and System Sciences, 1997
- Better Subset Regression Using the Nonnegative GarroteTechnometrics, 1995
- Matching pursuits with time-frequency dictionariesIEEE Transactions on Signal Processing, 1993