skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: A self-correcting variable-metric algorithm framework for nonsmooth optimization

Abstract

Abstract An algorithm framework is proposed for minimizing nonsmooth functions. The framework is variable metric in that, in each iteration, a step is computed using a symmetric positive-definite matrix whose value is updated as in a quasi-Newton scheme. However, unlike previously proposed variable-metric algorithms for minimizing nonsmooth functions, the framework exploits self-correcting properties made possible through Broyden–Fletcher–Goldfarb–Shanno-type updating. In so doing, the framework does not overly restrict the manner in which the step computation matrices are updated, yet the scheme is controlled well enough that global convergence guarantees can be established. The results of numerical experiments for a few algorithms are presented to demonstrate the self-correcting behaviours that are guaranteed by the framework.

Authors:
 [1];  [2];  [1]
  1. Department of Industrial and Systems Engineering, Lehigh University, Bethlehem, PA, USA
  2. Department of Applied Mathematics and Statistics, Johns Hopkins University, Baltimore, MD, USA
Publication Date:
Sponsoring Org.:
USDOE
OSTI Identifier:
1498577
Grant/Contract Number:  
SC0010615; DMS-1016291; DMS-1319356; IIS-1704458
Resource Type:
Published Article
Journal Name:
IMA Journal of Numerical Analysis
Additional Journal Information:
Journal Name: IMA Journal of Numerical Analysis Journal Volume: 40 Journal Issue: 2; Journal ID: ISSN 0272-4979
Publisher:
Oxford University Press
Country of Publication:
United Kingdom
Language:
English

Citation Formats

Curtis, Frank E., Robinson, Daniel P., and Zhou, Baoyu. A self-correcting variable-metric algorithm framework for nonsmooth optimization. United Kingdom: N. p., 2019. Web. doi:10.1093/imanum/drz008.
Curtis, Frank E., Robinson, Daniel P., & Zhou, Baoyu. A self-correcting variable-metric algorithm framework for nonsmooth optimization. United Kingdom. doi:10.1093/imanum/drz008.
Curtis, Frank E., Robinson, Daniel P., and Zhou, Baoyu. Thu . "A self-correcting variable-metric algorithm framework for nonsmooth optimization". United Kingdom. doi:10.1093/imanum/drz008.
@article{osti_1498577,
title = {A self-correcting variable-metric algorithm framework for nonsmooth optimization},
author = {Curtis, Frank E. and Robinson, Daniel P. and Zhou, Baoyu},
abstractNote = {Abstract An algorithm framework is proposed for minimizing nonsmooth functions. The framework is variable metric in that, in each iteration, a step is computed using a symmetric positive-definite matrix whose value is updated as in a quasi-Newton scheme. However, unlike previously proposed variable-metric algorithms for minimizing nonsmooth functions, the framework exploits self-correcting properties made possible through Broyden–Fletcher–Goldfarb–Shanno-type updating. In so doing, the framework does not overly restrict the manner in which the step computation matrices are updated, yet the scheme is controlled well enough that global convergence guarantees can be established. The results of numerical experiments for a few algorithms are presented to demonstrate the self-correcting behaviours that are guaranteed by the framework.},
doi = {10.1093/imanum/drz008},
journal = {IMA Journal of Numerical Analysis},
number = 2,
volume = 40,
place = {United Kingdom},
year = {2019},
month = {3}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
DOI: 10.1093/imanum/drz008

Save / Share:

Works referenced in this record:

Proximity control in bundle methods for convex nondifferentiable minimization
journal, January 1990

  • Kiwiel, Krzysztof C.
  • Mathematical Programming, Vol. 46, Issue 1-3
  • DOI: 10.1007/BF01585731

New variants of bundle methods
journal, July 1995

  • Lemaréchal, Claude; Nemirovskii, Arkadii; Nesterov, Yurii
  • Mathematical Programming, Vol. 69, Issue 1-3
  • DOI: 10.1007/BF01585555

Algorithms for nonlinear constraints that use lagrangian functions
journal, December 1978


A Version of the Bundle Idea for Minimizing a Nonsmooth Function: Conceptual Idea, Convergence Analysis, Numerical Results
journal, February 1992

  • Schramm, Helga; Zowe, Jochem
  • SIAM Journal on Optimization, Vol. 2, Issue 1
  • DOI: 10.1137/0802008

The Speed of Shor's R-algorithm
journal, February 2008

  • Burke, J. V.; Lewis, A. S.; Overton, M. L.
  • IMA Journal of Numerical Analysis, Vol. 28, Issue 4
  • DOI: 10.1093/imanum/drn008

A family of variable-metric methods derived by variational means
journal, January 1970


Optimization of lipschitz continuous functions
journal, December 1977


A characterization of superlinear convergence and its application to quasi-Newton methods
journal, May 1974


A Redistributed Proximal Bundle Method for Nonconvex Optimization
journal, January 2010

  • Hare, Warren; Sagastizábal, Claudia
  • SIAM Journal on Optimization, Vol. 20, Issue 5
  • DOI: 10.1137/090754595

A family of variable metric proximal methods
journal, January 1995

  • Bonnans, J. F.; Gilbert, J. Ch.; Lemaréchal, C.
  • Mathematical Programming, Vol. 68, Issue 1-3
  • DOI: 10.1007/BF01585756

Global Convergence of a Cass of Quasi-Newton Methods on Convex Problems
journal, October 1987

  • Byrd, Richard H.; Nocedal, Jorge; Yuan, Ya-Xiang
  • SIAM Journal on Numerical Analysis, Vol. 24, Issue 5
  • DOI: 10.1137/0724077

�ber die globale Konvergenz von Variable-Metrik-Verfahren mit nicht-exakter Schrittweitenbestimmung
journal, September 1978


A Robust Gradient Sampling Algorithm for Nonsmooth, Nonconvex Optimization
journal, January 2005

  • Burke, James V.; Lewis, Adrian S.; Overton, Michael L.
  • SIAM Journal on Optimization, Vol. 15, Issue 3
  • DOI: 10.1137/030601296

Restricted Step and Levenberg–Marquardt Techniques in Proximal Bundle Methods for Nonconvex Nondifferentiable Optimization
journal, February 1996

  • Kiwiel, Krzysztof C.
  • SIAM Journal on Optimization, Vol. 6, Issue 1
  • DOI: 10.1137/0806013

An adaptive gradient sampling algorithm for non-smooth optimization
journal, September 2012


Globally Convergent Variable Metric Method for Nonconvex Nondifferentiable Unconstrained Minimization
journal, November 2001

  • Vlček, J.; Lukšan, L.
  • Journal of Optimization Theory and Applications, Vol. 111, Issue 2
  • DOI: 10.1023/A:1011990503369

A bundle-Newton method for nonsmooth unconstrained minimization
journal, January 1998

  • Lukšan, Ladislav; Vlček, Jan
  • Mathematical Programming, Vol. 83, Issue 1-3
  • DOI: 10.1007/BF02680566

Efficiency of Proximal Bundle Methods
journal, March 2000


Variable Metric Method for Minimization
journal, February 1991

  • Davidon, William C.
  • SIAM Journal on Optimization, Vol. 1, Issue 1
  • DOI: 10.1137/0801001

New limited memory bundle method for large-scale nonsmooth optimization
journal, December 2004


Quasi-Newton Bundle-Type Methods for Nondifferentiable Convex Optimization
journal, May 1998


Globally convergent limited memory bundle method for large-scale nonsmooth optimization
journal, April 2006

  • Haarala, Napsu; Miettinen, Kaisa; Mäkelä, Marko M.
  • Mathematical Programming, Vol. 109, Issue 1
  • DOI: 10.1007/s10107-006-0728-2

Local and superlinear convergence of a class of variable metric methods
journal, September 1979


Linear convergence of epsilon-subgradient descent methods for a class of convex functions
journal, September 1999


Convergence of the Gradient Sampling Algorithm for Nonsmooth Nonconvex Optimization
journal, January 2007

  • Kiwiel, Krzysztof C.
  • SIAM Journal on Optimization, Vol. 18, Issue 2
  • DOI: 10.1137/050639673

A Tool for the Analysis of Quasi-Newton Methods with Application to Unconstrained Minimization
journal, June 1989

  • Byrd, Richard H.; Nocedal, Jorge
  • SIAM Journal on Numerical Analysis, Vol. 26, Issue 3
  • DOI: 10.1137/0726042

A quasi-Newton algorithm for nonconvex, nonsmooth optimization with global convergence guarantees
journal, May 2015


An Algorithm for Constrained Optimization with Semismooth Functions
journal, May 1977


Rate of Convergence of the Bundle Method
journal, March 2017

  • Du, Yu; Ruszczyński, Andrzej
  • Journal of Optimization Theory and Applications, Vol. 173, Issue 3
  • DOI: 10.1007/s10957-017-1108-1

Two-Point Step Size Gradient Methods
journal, January 1988

  • Barzilai, Jonathan; Borwein, Jonathan M.
  • IMA Journal of Numerical Analysis, Vol. 8, Issue 1
  • DOI: 10.1093/imanum/8.1.141

Conditioning of quasi-Newton methods for function minimization
journal, September 1970


A Linearization Algorithm for Nonsmooth Minimization
journal, May 1985

  • Kiwiel, Krzysztof Czesław
  • Mathematics of Operations Research, Vol. 10, Issue 2
  • DOI: 10.1287/moor.10.2.185

Variable metric methods of minimisation
journal, February 1969


A Method for Solving Certain Quadratic Programming Problems Arising in Nonsmooth Optimization
journal, January 1986


Nonsmooth optimization via quasi-Newton methods
journal, February 2012


The Convergence of a Class of Double-rank Minimization Algorithms 1. General Considerations
journal, January 1970


A new approach to variable metric algorithms
journal, March 1970


A Trust Region Spectral Bundle Method for Nonconvex Eigenvalue Optimization
journal, January 2008

  • Apkarian, P.; Noll, D.; Prot, O.
  • SIAM Journal on Optimization, Vol. 19, Issue 1
  • DOI: 10.1137/060665191

A -algorithm for convex minimization
journal, July 2005