A self-correcting variable-metric algorithm framework for nonsmooth optimization
Abstract
Abstract An algorithm framework is proposed for minimizing nonsmooth functions. The framework is variable metric in that, in each iteration, a step is computed using a symmetric positive-definite matrix whose value is updated as in a quasi-Newton scheme. However, unlike previously proposed variable-metric algorithms for minimizing nonsmooth functions, the framework exploits self-correcting properties made possible through Broyden–Fletcher–Goldfarb–Shanno-type updating. In so doing, the framework does not overly restrict the manner in which the step computation matrices are updated, yet the scheme is controlled well enough that global convergence guarantees can be established. The results of numerical experiments for a few algorithms are presented to demonstrate the self-correcting behaviours that are guaranteed by the framework.
- Authors:
-
- Department of Industrial and Systems Engineering, Lehigh University, Bethlehem, PA, USA
- Department of Applied Mathematics and Statistics, Johns Hopkins University, Baltimore, MD, USA
- Publication Date:
- Sponsoring Org.:
- USDOE
- OSTI Identifier:
- 1498577
- Grant/Contract Number:
- SC0010615; DMS-1016291; DMS-1319356; IIS-1704458
- Resource Type:
- Published Article
- Journal Name:
- IMA Journal of Numerical Analysis
- Additional Journal Information:
- Journal Name: IMA Journal of Numerical Analysis Journal Volume: 40 Journal Issue: 2; Journal ID: ISSN 0272-4979
- Publisher:
- Oxford University Press
- Country of Publication:
- United Kingdom
- Language:
- English
Citation Formats
Curtis, Frank E., Robinson, Daniel P., and Zhou, Baoyu. A self-correcting variable-metric algorithm framework for nonsmooth optimization. United Kingdom: N. p., 2019.
Web. doi:10.1093/imanum/drz008.
Curtis, Frank E., Robinson, Daniel P., & Zhou, Baoyu. A self-correcting variable-metric algorithm framework for nonsmooth optimization. United Kingdom. https://doi.org/10.1093/imanum/drz008
Curtis, Frank E., Robinson, Daniel P., and Zhou, Baoyu. Thu .
"A self-correcting variable-metric algorithm framework for nonsmooth optimization". United Kingdom. https://doi.org/10.1093/imanum/drz008.
@article{osti_1498577,
title = {A self-correcting variable-metric algorithm framework for nonsmooth optimization},
author = {Curtis, Frank E. and Robinson, Daniel P. and Zhou, Baoyu},
abstractNote = {Abstract An algorithm framework is proposed for minimizing nonsmooth functions. The framework is variable metric in that, in each iteration, a step is computed using a symmetric positive-definite matrix whose value is updated as in a quasi-Newton scheme. However, unlike previously proposed variable-metric algorithms for minimizing nonsmooth functions, the framework exploits self-correcting properties made possible through Broyden–Fletcher–Goldfarb–Shanno-type updating. In so doing, the framework does not overly restrict the manner in which the step computation matrices are updated, yet the scheme is controlled well enough that global convergence guarantees can be established. The results of numerical experiments for a few algorithms are presented to demonstrate the self-correcting behaviours that are guaranteed by the framework.},
doi = {10.1093/imanum/drz008},
journal = {IMA Journal of Numerical Analysis},
number = 2,
volume = 40,
place = {United Kingdom},
year = {2019},
month = {3}
}
https://doi.org/10.1093/imanum/drz008
Works referenced in this record:
Proximity control in bundle methods for convex nondifferentiable minimization
journal, January 1990
- Kiwiel, Krzysztof C.
- Mathematical Programming, Vol. 46, Issue 1-3
New variants of bundle methods
journal, July 1995
- Lemaréchal, Claude; Nemirovskii, Arkadii; Nesterov, Yurii
- Mathematical Programming, Vol. 69, Issue 1-3
Algorithms for nonlinear constraints that use lagrangian functions
journal, December 1978
- Powell, M. J. D.
- Mathematical Programming, Vol. 14, Issue 1
A Version of the Bundle Idea for Minimizing a Nonsmooth Function: Conceptual Idea, Convergence Analysis, Numerical Results
journal, February 1992
- Schramm, Helga; Zowe, Jochem
- SIAM Journal on Optimization, Vol. 2, Issue 1
The Speed of Shor's R-algorithm
journal, February 2008
- Burke, J. V.; Lewis, A. S.; Overton, M. L.
- IMA Journal of Numerical Analysis, Vol. 28, Issue 4
A family of variable-metric methods derived by variational means
journal, January 1970
- Goldfarb, Donald
- Mathematics of Computation, Vol. 24, Issue 109
Optimization of lipschitz continuous functions
journal, December 1977
- Goldstein, A. A.
- Mathematical Programming, Vol. 13, Issue 1
A characterization of superlinear convergence and its application to quasi-Newton methods
journal, May 1974
- Dennis, J. E.; Mor{é, Jorge J.
- Mathematics of Computation, Vol. 28, Issue 126
A Redistributed Proximal Bundle Method for Nonconvex Optimization
journal, January 2010
- Hare, Warren; Sagastizábal, Claudia
- SIAM Journal on Optimization, Vol. 20, Issue 5
A family of variable metric proximal methods
journal, January 1995
- Bonnans, J. F.; Gilbert, J. Ch.; Lemaréchal, C.
- Mathematical Programming, Vol. 68, Issue 1-3
Global Convergence of a Cass of Quasi-Newton Methods on Convex Problems
journal, October 1987
- Byrd, Richard H.; Nocedal, Jorge; Yuan, Ya-Xiang
- SIAM Journal on Numerical Analysis, Vol. 24, Issue 5
�ber die globale Konvergenz von Variable-Metrik-Verfahren mit nicht-exakter Schrittweitenbestimmung
journal, September 1978
- Werner, Jochen
- Numerische Mathematik, Vol. 31, Issue 3
A Robust Gradient Sampling Algorithm for Nonsmooth, Nonconvex Optimization
journal, January 2005
- Burke, James V.; Lewis, Adrian S.; Overton, Michael L.
- SIAM Journal on Optimization, Vol. 15, Issue 3
Restricted Step and Levenberg–Marquardt Techniques in Proximal Bundle Methods for Nonconvex Nondifferentiable Optimization
journal, February 1996
- Kiwiel, Krzysztof C.
- SIAM Journal on Optimization, Vol. 6, Issue 1
An adaptive gradient sampling algorithm for non-smooth optimization
journal, September 2012
- Curtis, Frank E.; Que, Xiaocun
- Optimization Methods and Software, Vol. 28, Issue 6
Globally Convergent Variable Metric Method for Nonconvex Nondifferentiable Unconstrained Minimization
journal, November 2001
- Vlček, J.; Lukšan, L.
- Journal of Optimization Theory and Applications, Vol. 111, Issue 2
A bundle-Newton method for nonsmooth unconstrained minimization
journal, January 1998
- Lukšan, Ladislav; Vlček, Jan
- Mathematical Programming, Vol. 83, Issue 1-3
Efficiency of Proximal Bundle Methods
journal, March 2000
- Kiwiel, K. C.
- Journal of Optimization Theory and Applications, Vol. 104, Issue 3
Variable Metric Method for Minimization
journal, February 1991
- Davidon, William C.
- SIAM Journal on Optimization, Vol. 1, Issue 1
New limited memory bundle method for large-scale nonsmooth optimization
journal, December 2004
- Haarala, M.; Miettinen †, K.; Mäkelä ‡, M. M.
- Optimization Methods and Software, Vol. 19, Issue 6
Quasi-Newton Bundle-Type Methods for Nondifferentiable Convex Optimization
journal, May 1998
- Mifflin, Robert; Sun, Defeng; Qi, Liqun
- SIAM Journal on Optimization, Vol. 8, Issue 2
Globally convergent limited memory bundle method for large-scale nonsmooth optimization
journal, April 2006
- Haarala, Napsu; Miettinen, Kaisa; Mäkelä, Marko M.
- Mathematical Programming, Vol. 109, Issue 1
Local and superlinear convergence of a class of variable metric methods
journal, September 1979
- Ritter, K.
- Computing, Vol. 23, Issue 3
Linear convergence of epsilon-subgradient descent methods for a class of convex functions
journal, September 1999
- Robinson, Stephen M.
- Mathematical Programming, Vol. 86, Issue 1
Convergence of the Gradient Sampling Algorithm for Nonsmooth Nonconvex Optimization
journal, January 2007
- Kiwiel, Krzysztof C.
- SIAM Journal on Optimization, Vol. 18, Issue 2
A Tool for the Analysis of Quasi-Newton Methods with Application to Unconstrained Minimization
journal, June 1989
- Byrd, Richard H.; Nocedal, Jorge
- SIAM Journal on Numerical Analysis, Vol. 26, Issue 3
A quasi-Newton algorithm for nonconvex, nonsmooth optimization with global convergence guarantees
journal, May 2015
- Curtis, Frank E.; Que, Xiaocun
- Mathematical Programming Computation, Vol. 7, Issue 4
An Algorithm for Constrained Optimization with Semismooth Functions
journal, May 1977
- Mifflin, Robert
- Mathematics of Operations Research, Vol. 2, Issue 2
Rate of Convergence of the Bundle Method
journal, March 2017
- Du, Yu; Ruszczyński, Andrzej
- Journal of Optimization Theory and Applications, Vol. 173, Issue 3
Two-Point Step Size Gradient Methods
journal, January 1988
- Barzilai, Jonathan; Borwein, Jonathan M.
- IMA Journal of Numerical Analysis, Vol. 8, Issue 1
Conditioning of quasi-Newton methods for function minimization
journal, September 1970
- Shanno, D. F.
- Mathematics of Computation, Vol. 24, Issue 111
A Linearization Algorithm for Nonsmooth Minimization
journal, May 1985
- Kiwiel, Krzysztof Czesław
- Mathematics of Operations Research, Vol. 10, Issue 2
Variable metric methods of minimisation
journal, February 1969
- Pearson, J. D.
- The Computer Journal, Vol. 12, Issue 2
A Method for Solving Certain Quadratic Programming Problems Arising in Nonsmooth Optimization
journal, January 1986
- Kiwiel, Krzysztof C.
- IMA Journal of Numerical Analysis, Vol. 6, Issue 2
Nonsmooth optimization via quasi-Newton methods
journal, February 2012
- Lewis, Adrian S.; Overton, Michael L.
- Mathematical Programming, Vol. 141, Issue 1-2
The Convergence of a Class of Double-rank Minimization Algorithms 1. General Considerations
journal, January 1970
- Broyden, C. G.
- IMA Journal of Applied Mathematics, Vol. 6, Issue 1
A new approach to variable metric algorithms
journal, March 1970
- Fletcher, R.
- The Computer Journal, Vol. 13, Issue 3
A Trust Region Spectral Bundle Method for Nonconvex Eigenvalue Optimization
journal, January 2008
- Apkarian, P.; Noll, D.; Prot, O.
- SIAM Journal on Optimization, Vol. 19, Issue 1
A -algorithm for convex minimization
journal, July 2005
- Mifflin, Robert; Sagastizábal, Claudia
- Mathematical Programming, Vol. 104, Issue 2-3