Towards good practice in large-scale learning for image classification
- 1 June 2012
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 3482-3489
- https://doi.org/10.1109/cvpr.2012.6248090
Abstract
We propose a benchmark of several objective functions for large-scale image classification: we compare the one-vs-rest, multiclass, ranking and weighted average ranking SVMs. Using stochastic gradient descent optimization, we can scale the learning to millions of images and thousands of classes. Our experimental evaluation shows that ranking based algorithms do not outperform a one-vs-rest strategy and that the gap between the different algorithms reduces in case of high-dimensional data. We also show that for one-vs-rest, learning through cross-validation the optimal degree of imbalance between the positive and the negative samples can have a significant impact. Furthermore, early stopping can be used as an effective regularization strategy when training with stochastic gradient algorithms. Following these "good practices", we were able to improve the state-of-the-art on a large subset of 10K classes and 9M of images of lmageNet from 16.7% accuracy to 19.1%.Keywords
This publication has 24 references indexed in Scilit:
- Large-scale image classification: Fast feature extraction and SVM trainingPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2011
- The devil is in the details: an evaluation of recent feature encoding methodsPublished by British Machine Vision Association and Society for Pattern Recognition ,2011
- Large scale image annotation: learning to rank with joint word-image embeddingsMachine Learning, 2010
- Structured Learning and Prediction in Computer VisionFoundations and Trends® in Computer Graphics and Vision, 2010
- Fisher Kernels on Visual Vocabularies for Image CategorizationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2007
- Training linear SVMs in linear timePublished by Association for Computing Machinery (ACM) ,2006
- Convexity, Classification, and Risk BoundsJournal of the American Statistical Association, 2006
- Distinctive Image Features from Scale-Invariant KeypointsInternational Journal of Computer Vision, 2004
- Optimizing search engines using clickthrough dataPublished by Association for Computing Machinery (ACM) ,2002
- Efficient BackPropPublished by Springer Nature ,1998