A log-barrier Newton-CG method for bound constrained optimization with complexity guarantees

O’Neill, Michael; Wright, Stephen J.

doi:10.1093/imanum/drz074

A log-barrier Newton-CG method for bound constrained optimization with complexity guarantees

Journal Article · Sat Apr 18 00:00:00 EDT 2020 · IMA Journal of Numerical Analysis

DOI:https://doi.org/10.1093/imanum/drz074· OSTI ID:1615402

O’Neill, Michael ^[1]; Wright, Stephen J. ^[1]

Department of Computer Sciences, University of Wisconsin-Madison, 1210 W. Dayton Street, Madison, WI 53706, USA

Abstract

We describe an algorithm based on a logarithmic barrier function, Newton’s method and linear conjugate gradients that seeks an approximate minimizer of a smooth function over the non-negative orthant. We develop a bound on the complexity of the approach, stated in terms of the required accuracy and the cost of a single gradient evaluation of the objective function and/or a matrix-vector multiplication involving the Hessian of the objective. The approach can be implemented without explicit calculation or storage of the Hessian.

View Journal Article

Sponsoring Organization:: USDOE

OSTI ID:: 1615402

Journal Information:: IMA Journal of Numerical Analysis, Journal Name: IMA Journal of Numerical Analysis Journal Issue: 1 Vol. 41; ISSN 0272-4979

Publisher:: Oxford University PressCopyright Statement

Country of Publication:: United Kingdom

Language:: English

References (23)

Cubic regularization of Newton method and its global performance Nesterov, Yurii; Polyak, B. T. Mathematical Programming, Vol. 108, Issue 1 https://doi.org/10.1007/s10107-006-0706-8	journal	April 2006
Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical results Cartis, Coralia; Gould, Nicholas I. M.; Toint, Philippe L. Mathematical Programming, Vol. 127, Issue 2 https://doi.org/10.1007/s10107-009-0286-5	journal	May 2009
Complexity analysis of interior point algorithms for non-Lipschitz and nonconvex minimization Bian, Wei; Chen, Xiaojun; Ye, Yinyu Mathematical Programming, Vol. 149, Issue 1-2 https://doi.org/10.1007/s10107-014-0753-5	journal	February 2014
A trust region algorithm with a worst-case iteration complexity of $$\mathcal{O}(\epsilon ^{-3/2})$$ O ( ϵ - 3 / 2 ) for nonconvex optimization Curtis, Frank E.; Robinson, Daniel P.; Samadi, Mohammadreza Mathematical Programming, Vol. 162, Issue 1-2 https://doi.org/10.1007/s10107-016-1026-2	journal	May 2016
Optimality condition and complexity analysis for linearly-constrained optimization without differentiability on the boundary Haeser, Gabriel; Liu, Hongcheng; Ye, Yinyu Mathematical Programming, Vol. 178, Issue 1-2 https://doi.org/10.1007/s10107-018-1290-4	journal	May 2018
A Newton-CG algorithm with complexity guarantees for smooth unconstrained optimization Royer, Clément W.; O’Neill, Michael; Wright, Stephen J. Mathematical Programming, Vol. 180, Issue 1-2 https://doi.org/10.1007/s10107-019-01362-7	journal	January 2019
A Newton-like method with mixed factorizations and cubic regularization for unconstrained minimization Birgin, E. G.; Martínez, J. M. Computational Optimization and Applications, Vol. 73, Issue 3 https://doi.org/10.1007/s10589-019-00089-7	journal	March 2019
Cubic-regularization counterpart of a variable-norm trust-region method for unconstrained minimization Martínez, J. M.; Raydan, M. Journal of Global Optimization, Vol. 68, Issue 2 https://doi.org/10.1007/s10898-016-0475-8	journal	October 2016
Complexity bounds for second-order optimality in unconstrained optimization Cartis, C.; Gould, N. I. M.; Toint, Ph. L. Journal of Complexity, Vol. 28, Issue 1 https://doi.org/10.1016/j.jco.2011.06.001	journal	February 2012
Optimality of orders one to three and beyond: Characterization and evaluation complexity in constrained nonconvex optimization Cartis, C.; Gould, N. I. M.; Toint, Ph. L. Journal of Complexity, Vol. 53 https://doi.org/10.1016/j.jco.2018.11.001	journal	August 2019
An adaptive cubic regularization algorithm for nonconvex optimization with convex constraints and its function-evaluation complexity Cartis, C.; Gould, N. I. M.; Toint, P. L. IMA Journal of Numerical Analysis, Vol. 32, Issue 4 https://doi.org/10.1093/imanum/drr035	journal	January 2012
Erratum “A second-order sequential optimality condition associated to the convergence of optimization algorithms” Andreani, Roberto; Haeser, Gabriel; Ramos, Alberto IMA Journal of Numerical Analysis, Vol. 37, Issue 4 https://doi.org/10.1093/imanum/drx011	journal	May 2017
An inexact regularized Newton framework with a worst-case iteration complexity of $ {\mathscr O}(\varepsilon^{-3/2}) $ for nonconvex optimization Curtis, Frank E.; Robinson, Daniel P.; Samadi, Mohammadreza IMA Journal of Numerical Analysis, Vol. 39, Issue 3 https://doi.org/10.1093/imanum/dry022	journal	May 2018
Lower Bound Theory of Nonzero Entries in Solutions of $\ell_2$-$\ell_p$ Minimization Chen, Xiaojun; Xu, Fengmin; Ye, Yinyu SIAM Journal on Scientific Computing, Vol. 32, Issue 5 https://doi.org/10.1137/090761471	journal	January 2010
On the Evaluation Complexity of Constrained Nonlinear Least-Squares and General Constrained Nonlinear Optimization Using Second-Order Methods Cartis, Coralia; Gould, Nicholas I. M.; Toint, Philippe L. SIAM Journal on Numerical Analysis, Vol. 53, Issue 2 https://doi.org/10.1137/130915546	journal	January 2015
Evaluation Complexity for Nonlinear Constrained Optimization Using Unscaled KKT Conditions and High-Order Models Birgin, E. G.; Gardenghi, J. L.; Martínez, J. M. SIAM Journal on Optimization, Vol. 26, Issue 2 https://doi.org/10.1137/15M1031631	journal	January 2016
The Use of Quadratic Regularization with a Cubic Descent Condition for Unconstrained Optimization Birgin, E. G.; Martínez, J. M. SIAM Journal on Optimization, Vol. 27, Issue 2 https://doi.org/10.1137/16M110280X	journal	January 2017
Universal Regularization Methods: Varying the Power, the Smoothness and the Accuracy Cartis, Coralia; Gould, Nick I.; Toint, Philippe L. SIAM Journal on Optimization, Vol. 29, Issue 1 https://doi.org/10.1137/16M1106316	journal	January 2019
Complexity Analysis of a Trust Funnel Algorithm for Equality Constrained Optimization Curtis, Frank E.; Robinson, Daniel P.; Samadi, Mohammadreza SIAM Journal on Optimization, Vol. 28, Issue 2 https://doi.org/10.1137/16M1108650	journal	January 2018
Accelerated Methods for NonConvex Optimization Carmon, Yair; Duchi, John C.; Hinder, Oliver SIAM Journal on Optimization, Vol. 28, Issue 2 https://doi.org/10.1137/17M1114296	journal	January 2018
On Regularization and Active-set Methods with Complexity for Constrained Optimization Birgin, E. G.; Martínez, J. M. SIAM Journal on Optimization, Vol. 28, Issue 2 https://doi.org/10.1137/17M1127107	journal	January 2018
Complexity Analysis of Second-Order Line-Search Algorithms for Smooth Nonconvex Optimization Royer, Clément W.; Wright, Stephen J. SIAM Journal on Optimization, Vol. 28, Issue 2 https://doi.org/10.1137/17M1134329	journal	January 2018
Finding approximate local minima faster than gradient descent Agarwal, Naman; Allen-Zhu, Zeyuan; Bullins, Brian Proceedings of the 49th Annual ACM SIGACT Symposium on Theory of Computing - STOC 2017 https://doi.org/10.1145/3055399.3055464	conference	January 2017

Similar Records

Inexact Newton-CG algorithms with complexity guarantees

Journal Article · Mon Aug 22 00:00:00 EDT 2022 · IMA Journal of Numerical Analysis · OSTI ID:2424937

On the convergence of the Newton/log-barrier method.

Journal Article · Wed Feb 28 23:00:00 EST 2001 · Math. Program. · OSTI ID:938229

Exact and inexact subsampled Newton methods for optimization

Journal Article · Tue Apr 03 00:00:00 EDT 2018 · IMA Journal of Numerical Analysis · OSTI ID:1610078

A log-barrier Newton-CG method for bound constrained optimization with complexity guarantees

Citation Formats

References (23)

Similar Records

Related Subjects