A self-correcting variable-metric algorithm framework for nonsmooth optimization

Curtis, Frank E.; Robinson, Daniel P.; Zhou, Baoyu

doi:10.1093/imanum/drz008

Title: A self-correcting variable-metric algorithm framework for nonsmooth optimization

Journal Article · Thu Mar 07 00:00:00 EST 2019 · IMA Journal of Numerical Analysis

DOI:https://doi.org/10.1093/imanum/drz008· OSTI ID:1802139

Curtis, Frank E. ^[1]; Robinson, Daniel P. ^[2]; Zhou, Baoyu ^[1]

Department of Industrial and Systems Engineering, Lehigh University, Bethlehem, PA, USA
Department of Applied Mathematics and Statistics, Johns Hopkins University, Baltimore, MD, USA

Abstract An algorithm framework is proposed for minimizing nonsmooth functions. The framework is variable metric in that, in each iteration, a step is computed using a symmetric positive-definite matrix whose value is updated as in a quasi-Newton scheme. However, unlike previously proposed variable-metric algorithms for minimizing nonsmooth functions, the framework exploits self-correcting properties made possible through Broyden–Fletcher–Goldfarb–Shanno-type updating. In so doing, the framework does not overly restrict the manner in which the step computation matrices are updated, yet the scheme is controlled well enough that global convergence guarantees can be established. The results of numerical experiments for a few algorithms are presented to demonstrate the self-correcting behaviours that are guaranteed by the framework.

Cite

Export

Save

Research Organization:: Lehigh Univ., Bethlehem, PA (United States)

Sponsoring Organization:: USDOE Office of Science (SC)

DOE Contract Number:: SC0010615

OSTI ID:: 1802139

Journal Information:: IMA Journal of Numerical Analysis, Vol. 40, Issue 2; ISSN 0272-4979

Publisher:: Oxford University Press/Institute of Mathematics and its Applications

Country of Publication:: United States

Language:: English

References (39)

A Trust Region Spectral Bundle Method for Nonconvex Eigenvalue Optimization Apkarian, P.; Noll, D.; Prot, O. SIAM Journal on Optimization, Vol. 19, Issue 1 https://doi.org/10.1137/060665191	journal	January 2008
Two-Point Step Size Gradient Methods Barzilai, Jonathan; Borwein, Jonathan M. IMA Journal of Numerical Analysis, Vol. 8, Issue 1 https://doi.org/10.1093/imanum/8.1.141	journal	January 1988
A family of variable metric proximal methods Bonnans, J. F.; Gilbert, J. Ch.; Lemaréchal, C. Mathematical Programming, Vol. 68, Issue 1-3 https://doi.org/10.1007/BF01585756	journal	January 1995
The Convergence of a Class of Double-rank Minimization Algorithms 1. General Considerations Broyden, C. G. IMA Journal of Applied Mathematics, Vol. 6, Issue 1 https://doi.org/10.1093/imamat/6.1.76	journal	January 1970
A Robust Gradient Sampling Algorithm for Nonsmooth, Nonconvex Optimization Burke, James V.; Lewis, Adrian S.; Overton, Michael L. SIAM Journal on Optimization, Vol. 15, Issue 3 https://doi.org/10.1137/030601296	journal	January 2005
The Speed of Shor's R-algorithm Burke, J. V.; Lewis, A. S.; Overton, M. L. IMA Journal of Numerical Analysis, Vol. 28, Issue 4 https://doi.org/10.1093/imanum/drn008	journal	February 2008
A Tool for the Analysis of Quasi-Newton Methods with Application to Unconstrained Minimization Byrd, Richard H.; Nocedal, Jorge SIAM Journal on Numerical Analysis, Vol. 26, Issue 3 https://doi.org/10.1137/0726042	journal	June 1989
Global Convergence of a Cass of Quasi-Newton Methods on Convex Problems Byrd, Richard H.; Nocedal, Jorge; Yuan, Ya-Xiang SIAM Journal on Numerical Analysis, Vol. 24, Issue 5 https://doi.org/10.1137/0724077	journal	October 1987
An adaptive gradient sampling algorithm for non-smooth optimization Curtis, Frank E.; Que, Xiaocun Optimization Methods and Software, Vol. 28, Issue 6 https://doi.org/10.1080/10556788.2012.714781	journal	September 2012
A quasi-Newton algorithm for nonconvex, nonsmooth optimization with global convergence guarantees Curtis, Frank E.; Que, Xiaocun Mathematical Programming Computation, Vol. 7, Issue 4 https://doi.org/10.1007/s12532-015-0086-2	journal	May 2015
Variable Metric Method for Minimization Davidon, William C. SIAM Journal on Optimization, Vol. 1, Issue 1 https://doi.org/10.1137/0801001	journal	February 1991
A characterization of superlinear convergence and its application to quasi-Newton methods Dennis, J. E.; Mor{é, Jorge J. Mathematics of Computation, Vol. 28, Issue 126 https://doi.org/10.1090/S0025-5718-1974-0343581-1	journal	May 1974
Rate of Convergence of the Bundle Method Du, Yu; Ruszczyński, Andrzej Journal of Optimization Theory and Applications, Vol. 173, Issue 3 https://doi.org/10.1007/s10957-017-1108-1	journal	March 2017
A new approach to variable metric algorithms Fletcher, R. The Computer Journal, Vol. 13, Issue 3 https://doi.org/10.1093/comjnl/13.3.317	journal	March 1970
A family of variable-metric methods derived by variational means Goldfarb, Donald Mathematics of Computation, Vol. 24, Issue 109 https://doi.org/10.1090/S0025-5718-1970-0258249-6	journal	January 1970
Optimization of lipschitz continuous functions Goldstein, A. A. Mathematical Programming, Vol. 13, Issue 1 https://doi.org/10.1007/BF01584320	journal	December 1977
New limited memory bundle method for large-scale nonsmooth optimization Haarala, M.; Miettinen †, K.; Mäkelä ‡, M. M. Optimization Methods and Software, Vol. 19, Issue 6 https://doi.org/10.1080/10556780410001689225	journal	December 2004
Globally convergent limited memory bundle method for large-scale nonsmooth optimization Haarala, Napsu; Miettinen, Kaisa; Mäkelä, Marko M. Mathematical Programming, Vol. 109, Issue 1 https://doi.org/10.1007/s10107-006-0728-2	journal	April 2006
A Redistributed Proximal Bundle Method for Nonconvex Optimization Hare, Warren; Sagastizábal, Claudia SIAM Journal on Optimization, Vol. 20, Issue 5 https://doi.org/10.1137/090754595	journal	January 2010
A Linearization Algorithm for Nonsmooth Minimization Kiwiel, Krzysztof Czesław Mathematics of Operations Research, Vol. 10, Issue 2 https://doi.org/10.1287/moor.10.2.185	journal	May 1985
A Method for Solving Certain Quadratic Programming Problems Arising in Nonsmooth Optimization Kiwiel, Krzysztof C. IMA Journal of Numerical Analysis, Vol. 6, Issue 2 https://doi.org/10.1093/imanum/6.2.137	journal	January 1986
Proximity control in bundle methods for convex nondifferentiable minimization Kiwiel, Krzysztof C. Mathematical Programming, Vol. 46, Issue 1-3 https://doi.org/10.1007/BF01585731	journal	January 1990
Restricted Step and Levenberg–Marquardt Techniques in Proximal Bundle Methods for Nonconvex Nondifferentiable Optimization Kiwiel, Krzysztof C. SIAM Journal on Optimization, Vol. 6, Issue 1 https://doi.org/10.1137/0806013	journal	February 1996
Efficiency of Proximal Bundle Methods Kiwiel, K. C. Journal of Optimization Theory and Applications, Vol. 104, Issue 3 https://doi.org/10.1023/A:1004689609425	journal	March 2000
Convergence of the Gradient Sampling Algorithm for Nonsmooth Nonconvex Optimization Kiwiel, Krzysztof C. SIAM Journal on Optimization, Vol. 18, Issue 2 https://doi.org/10.1137/050639673	journal	January 2007
New variants of bundle methods Lemaréchal, Claude; Nemirovskii, Arkadii; Nesterov, Yurii Mathematical Programming, Vol. 69, Issue 1-3 https://doi.org/10.1007/BF01585555	journal	July 1995
Nonsmooth optimization via quasi-Newton methods Lewis, Adrian S.; Overton, Michael L. Mathematical Programming, Vol. 141, Issue 1-2 https://doi.org/10.1007/s10107-012-0514-2	journal	February 2012
A bundle-Newton method for nonsmooth unconstrained minimization Lukšan, Ladislav; Vlček, Jan Mathematical Programming, Vol. 83, Issue 1-3 https://doi.org/10.1007/BF02680566	journal	January 1998
An Algorithm for Constrained Optimization with Semismooth Functions Mifflin, Robert Mathematics of Operations Research, Vol. 2, Issue 2 https://doi.org/10.1287/moor.2.2.191	journal	May 1977
A -algorithm for convex minimization Mifflin, Robert; Sagastizábal, Claudia Mathematical Programming, Vol. 104, Issue 2-3 https://doi.org/10.1007/s10107-005-0630-3	journal	July 2005
Quasi-Newton Bundle-Type Methods for Nondifferentiable Convex Optimization Mifflin, Robert; Sun, Defeng; Qi, Liqun SIAM Journal on Optimization, Vol. 8, Issue 2 https://doi.org/10.1137/S1052623496303329	journal	May 1998
Variable metric methods of minimisation Pearson, J. D. The Computer Journal, Vol. 12, Issue 2 https://doi.org/10.1093/comjnl/12.2.171	journal	February 1969
Algorithms for nonlinear constraints that use lagrangian functions Powell, M. J. D. Mathematical Programming, Vol. 14, Issue 1 https://doi.org/10.1007/BF01588967	journal	December 1978
Local and superlinear convergence of a class of variable metric methods Ritter, K. Computing, Vol. 23, Issue 3 https://doi.org/10.1007/BF02252133	journal	September 1979
Linear convergence of epsilon-subgradient descent methods for a class of convex functions Robinson, Stephen M. Mathematical Programming, Vol. 86, Issue 1 https://doi.org/10.1007/s101070050078	journal	September 1999
A Version of the Bundle Idea for Minimizing a Nonsmooth Function: Conceptual Idea, Convergence Analysis, Numerical Results Schramm, Helga; Zowe, Jochem SIAM Journal on Optimization, Vol. 2, Issue 1 https://doi.org/10.1137/0802008	journal	February 1992
Conditioning of quasi-Newton methods for function minimization Shanno, D. F. Mathematics of Computation, Vol. 24, Issue 111 https://doi.org/10.1090/S0025-5718-1970-0274029-X	journal	September 1970
Globally Convergent Variable Metric Method for Nonconvex Nondifferentiable Unconstrained Minimization Vlček, J.; Lukšan, L. Journal of Optimization Theory and Applications, Vol. 111, Issue 2 https://doi.org/10.1023/A:1011990503369	journal	November 2001
�ber die globale Konvergenz von Variable-Metrik-Verfahren mit nicht-exakter Schrittweitenbestimmung Werner, Jochen Numerische Mathematik, Vol. 31, Issue 3 https://doi.org/10.1007/BF01397884	journal	September 1978

Similar Records

A self-correcting variable-metric algorithm framework for nonsmooth optimization

Journal Article · Thu Mar 07 00:00:00 EST 2019 · IMA Journal of Numerical Analysis · OSTI ID:1802139

Curtis, Frank E.; Robinson, Daniel P.; Zhou, Baoyu

Automatic history matching with variable-metric methods

Journal Article · Mon Aug 01 00:00:00 EDT 1988 · SPE (Society of Petroleum Engineers) Reservoir Engineering; (USA) · OSTI ID:1802139

Yang, P H; Watson, A T

Pre-conditioned BFGS-based uncertainty quantification in elastic full-waveform inversion

Journal Article · Tue Sep 21 00:00:00 EDT 2021 · Geophysical Journal International · OSTI ID:1802139

Liu, Qiancheng; Beller, Stephen; Lei, Wenjie; +2 more

Related Subjects

Mathematics

Title: A self-correcting variable-metric algorithm framework for nonsmooth optimization

Citation Formats

References (39)

Similar Records

Related Subjects