Convergence of Approximate and Incremental Subgradient Methods for Convex Optimization
Top Cited Papers
- 1 January 2004
- journal article
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Optimization
- Vol. 14 (3) , 807-840
- https://doi.org/10.1137/s1052623400376366
Abstract
We present a unified convergence framework for approximate subgradient methods that covers various stepsize rules (including both diminishing and nonvanishing stepsizes), convergence in objective values, and convergence to a neighborhood of the optimal set. We discuss ways of ensuring the boundedness of the iterates and give efficiency estimates. Our results are extended to incremental subgradient methods for minimizing a sum of convex functions, which have recently been shown to be promising for various large-scale problems, including those arising from Lagrangian relaxation.Keywords
This publication has 25 references indexed in Scilit:
- A variable target value method for nondifferentiable optimizationOperations Research Letters, 2000
- Convergence of a simple subgradient level methodMathematical Programming, 1999
- Conditional subgradient optimization — Theory and applicationsEuropean Journal of Operational Research, 1996
- A class of unconstrained minimization methods for neural network trainingOptimization Methods and Software, 1994
- Convergence properties of backpropagation for neural nets via theory of stochastic gradient methods. Part 1Optimization Methods and Software, 1994
- Serial and parallel backpropagation convergence via nonmonotone perturbed minimizationOptimization Methods and Software, 1994
- Analysis of an approximate gradient projection method with applications to the backpropagation algorithmOptimization Methods and Software, 1994
- An improved subgradient method for constrained nondifferentiable optimizationOperations Research Letters, 1993
- On the Convergence of the LMS Algorithm with Adaptive Learning Rate for Linear Feedforward NetworksNeural Computation, 1991
- A note on the convergence of subgradient optimization methodsMathematische Operationsforschung und Statistik. Series Optimization, 1983