Search Menu
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information
Search Scholarly Publications

Title: An inexact regularized Newton framework with a worst-case iteration complexity of $$ {\mathscr O}(\varepsilon^{-3/2}) $$ for nonconvex optimization

Journal Article · · IMA Journal of Numerical Analysis
 [1];  [2];  [1]
  1. Department of Industrial and Systems Engineering, Lehigh University
  2. Department of Applied Mathematics and Statistics, Johns Hopkins University
+ Show Author Affiliations

Abstract An algorithm for solving smooth nonconvex optimization problems is proposed that, in the worst-case, takes $${\mathscr O}(\varepsilon ^{-3/2})$$ iterations to drive the norm of the gradient of the objective function below a prescribed positive real number $$\varepsilon $$ and can take $${\mathscr O}(\varepsilon ^{-3})$$ iterations to drive the leftmost eigenvalue of the Hessian of the objective above $$-\varepsilon $$. The proposed algorithm is a general framework that covers a wide range of techniques including quadratically and cubically regularized Newton methods, such as the Adaptive Regularization using Cubics (arc) method and the recently proposed Trust-Region Algorithm with Contractions and Expansions (trace). The generality of our method is achieved through the introduction of generic conditions that each trial step is required to satisfy, which in particular allows for inexact regularized Newton steps to be used. These conditions center around a new subproblem that can be approximately solved to obtain trial steps that satisfy the conditions. A new instance of the framework, distinct from arc and trace, is described that may be viewed as a hybrid between quadratically and cubically regularized Newton methods. Numerical results demonstrate that our hybrid algorithm outperforms a cubically regularized Newton method.

Sponsoring Organization:
USDOE
Grant/Contract Number:
SC0010615
OSTI ID:
1436375
Journal Information:
IMA Journal of Numerical Analysis, Journal Name: IMA Journal of Numerical Analysis Journal Issue: 3 Vol. 39; ISSN 0272-4979
Publisher:
Oxford University PressCopyright Statement
Country of Publication:
United Kingdom
Language:
English

References (14)

Introductory Lectures on Convex Optimization book January 2004
Cubic regularization of Newton method and its global performance journal April 2006
Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical results journal May 2009
Adaptive cubic regularisation methods for unconstrained optimization. Part II: worst-case function- and derivative-evaluation complexity journal January 2010
A trust region algorithm with a worst-case iteration complexity of $$\mathcal{O}(\epsilon ^{-3/2})$$ O ( ϵ - 3 / 2 ) for nonconvex optimization journal May 2016
Worst-case evaluation complexity for unconstrained nonlinear optimization using high-order regularized models journal August 2016
Benchmarking optimization software with performance profiles journal January 2002
Updating the regularization parameter in the adaptive cubic regularization algorithm journal December 2011
On solving trust-region and other regularised subproblems in optimization journal February 2010
ARC q : a new adaptive regularization by cubics journal May 2017
On the Complexity of Steepest Descent, Newton's and Regularized Newton's Methods for Nonconvex Unconstrained Optimization Problems journal January 2010
Trust Region Methods book January 2000
The Use of Quadratic Regularization with a Cubic Descent Condition for Unconstrained Optimization journal January 2017
GALAHAD, a library of thread-safe Fortran 90 packages for large-scale nonlinear optimization journal December 2003

Similar Records

Related Subjects