Large-Scale Quasi-Newton Trust-Region Methods With Low-Dimensional Linear Equality Constraints

Brust, Johannes J.; Marcia, Roummel F.; Petra, Cosmin G.

doi:10.1007/s10589-019-00127-4

Title: Large-Scale Quasi-Newton Trust-Region Methods With Low-Dimensional Linear Equality Constraints

Abstract

We propose two limited-memory BFGS (L-BFGS) trust-region methods for large-scale optimization with linear equality constraints. Here, the methods are intended for problems where the number of equality constraints is small. By exploiting the structure of the quasi-Newton compact representation, both proposed methods solve the trust-region subproblems nearly exactly, even for large problems. We derive theoretical global convergence results of the proposed algorithms, and compare their numerical effectiveness and performance on a variety of large-scale problems.

Authors:

^[1]; Marcia, Roummel F. ^[2]; Petra, Cosmin G. ^[3]

Argonne National Lab. (ANL), Argonne, IL (United States)
Univ. of California, Merced, CA (United States)
Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)

Publication Date:: Thu Sep 05 00:00:00 EDT 2019

Research Org.:: Argonne National Laboratory (ANL), Argonne, IL (United States); Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States)

Sponsoring Org.:: USDOE Laboratory Directed Research and Development (LDRD) Program; National Science Foundation (NSF); USDOE National Nuclear Security Administration (NNSA)

OSTI Identifier:: 1596681

Alternate Identifier(s):: OSTI ID: 1575872

Report Number(s):: LLNL-JRNL-755231
Journal ID: ISSN 0926-6003; 154974

Grant/Contract Number:: AC02-06CH11357; AC52-07NA27344

Resource Type:: Accepted Manuscript

Journal Name:: Computational Optimization and Applications

Additional Journal Information:: Journal Volume: 74; Journal Issue: 3; Journal ID: ISSN 0926-6003

Publisher:: Springer

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING; L-BFGS; Compact Representation; Eigendecomposition; Linear Equality Constraints; Quasi-Newton; Shape-Changing Norm; Trust-Region Algorithm; Linear equality constraints; Trust-region algorithm; Compact representation; Shape-changing norm

Citation Formats


                    Brust, Johannes J., Marcia, Roummel F., and Petra, Cosmin G. Large-Scale Quasi-Newton Trust-Region Methods With Low-Dimensional Linear Equality Constraints.  United States: N. p., 2019. 
Web.  doi:10.1007/s10589-019-00127-4.

Copy to clipboard


                    Brust, Johannes J., Marcia, Roummel F., & Petra, Cosmin G. Large-Scale Quasi-Newton Trust-Region Methods With Low-Dimensional Linear Equality Constraints.  United States.  https://doi.org/10.1007/s10589-019-00127-4

Copy to clipboard


                    Brust, Johannes J., Marcia, Roummel F., and Petra, Cosmin G. Thu .  
"Large-Scale Quasi-Newton Trust-Region Methods With Low-Dimensional Linear Equality Constraints".  United States.  https://doi.org/10.1007/s10589-019-00127-4.  https://www.osti.gov/servlets/purl/1596681.

Copy to clipboard


                    
@article{osti_1596681,

  title        = {Large-Scale Quasi-Newton Trust-Region Methods With Low-Dimensional Linear Equality Constraints},

  author       = {Brust, Johannes J. and Marcia, Roummel F. and Petra, Cosmin G.},

  abstractNote = {We propose two limited-memory BFGS (L-BFGS) trust-region methods for large-scale optimization with linear equality constraints. Here, the methods are intended for problems where the number of equality constraints is small. By exploiting the structure of the quasi-Newton compact representation, both proposed methods solve the trust-region subproblems nearly exactly, even for large problems. We derive theoretical global convergence results of the proposed algorithms, and compare their numerical effectiveness and performance on a variety of large-scale problems.},

  doi          = {10.1007/s10589-019-00127-4},

  journal      = {Computational Optimization and Applications},

  number       = 3,

  volume       = 74,

  place        = {United States},

  year         = {Thu Sep 05 00:00:00 EDT 2019},

  month        = {Thu Sep 05 00:00:00 EDT 2019}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1007/s10589-019-00127-4

Other availability

Search WorldCat to find libraries that may hold this journal

Citation Metrics:

Cited by: 5 works

Citation information provided by
Web of Science

Save / Share:

Export Metadata

Save to My Library

Works referenced in this record:

Computing a Trust Region Step
journal, September 1983

Moré, Jorge J.; Sorensen, D. C.
SIAM Journal on Scientific and Statistical Computing, Vol. 4, Issue 3
DOI: 10.1137/0904038

A trust region method based on interior point techniques for nonlinear programming
journal, November 2000

Byrd, Richard H.; Gilbert, Jean Charles; Nocedal, Jorge
Mathematical Programming, Vol. 89, Issue 1
DOI: 10.1007/PL00011391

A trust region algorithm for equality constrained optimization
journal, November 1990

Powell, M. J. D.; Yuan, Y.
Mathematical Programming, Vol. 49, Issue 1-3
DOI: 10.1007/BF01588787

Compact representation of the full Broyden class of quasi-Newton updates: Compact Representation of the Full Broyden Class of Quasi-Newton Updates
journal, May 2018

DeGuchy, Omar; Erway, Jennifer B.; Marcia, Roummel F.
Numerical Linear Algebra with Applications, Vol. 25, Issue 5
DOI: 10.1002/nla.2186

On the Implementation of an Algorithm for Large-Scale Equality Constrained Optimization
journal, August 1998

Lalee, Marucha; Nocedal, Jorge; Plantenga, Todd
SIAM Journal on Optimization, Vol. 8, Issue 3
DOI: 10.1137/S1052623493262993

On efficiently combining limited-memory and trust-region techniques
journal, June 2016

Burdakov, Oleg; Gong, Lujin; Zikrin, Spartak
Mathematical Programming Computation, Vol. 9, Issue 1
DOI: 10.1007/s12532-016-0109-7

A Trust Region Algorithm for Equality Constrained Minimization: Convergence Properties and Implementation
journal, June 1985

Vardi, Avi
SIAM Journal on Numerical Analysis, Vol. 22, Issue 3
DOI: 10.1137/0722035

On solving L-SR1 trust-region subproblems
journal, September 2016

Brust, Johannes; Erway, Jennifer B.; Marcia, Roummel F.
Computational Optimization and Applications, Vol. 66, Issue 2
DOI: 10.1007/s10589-016-9868-3

The Conjugate Gradient Method and Trust Regions in Large Scale Optimization
journal, June 1983

Steihaug, Trond
SIAM Journal on Numerical Analysis, Vol. 20, Issue 3
DOI: 10.1137/0720042

A dense initialization for limited-memory quasi-Newton methods
journal, May 2019

Brust, Johannes; Burdakov, Oleg; Erway, Jennifer B.
Computational Optimization and Applications, Vol. 74, Issue 1
DOI: 10.1007/s10589-019-00112-x

Benchmarking optimization software with performance profiles
journal, January 2002

Dolan, Elizabeth D.; Moré, Jorge J.
Mathematical Programming, Vol. 91, Issue 2
DOI: 10.1007/s101070100263

Algorithm 943: MSS: MATLAB Software for L-BFGS Trust-Region Subproblems for Large-Scale Optimization
journal, June 2014

Erway, Jennifer B.; Marcia, Roummel F.
ACM Transactions on Mathematical Software, Vol. 40, Issue 4
DOI: 10.1145/2616588

Updating the Inverse of a Matrix
journal, June 1989

Hager, William W.
SIAM Review, Vol. 31, Issue 2, p. 221-239
DOI: 10.1137/1031049

On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming
journal, April 2005

Wächter, Andreas; Biegler, Lorenz T.
Mathematical Programming, Vol. 106, Issue 1
DOI: 10.1007/s10107-004-0559-y

An Interior Point Algorithm for Large-Scale Nonlinear Programming
journal, January 1999

Byrd, Richard H.; Hribar, Mary E.; Nocedal, Jorge
SIAM Journal on Optimization, Vol. 9, Issue 4
DOI: 10.1137/S1052623497325107

Compact representation of the full Broyden class of quasi-Newton updates
preprint, January 2017

DeGuchy, Omar; Erway, Jennifer B.; Marcia, Roummel F.
arXiv
DOI: 10.48550/arxiv.1705.08306

Benchmarking Optimization Software with Performance Profiles
text, January 2001

Dolan, Elizabeth D.; Moré, Jorge J.
arXiv
DOI: 10.48550/arxiv.cs/0102001

Representations of quasi-Newton matrices and their use in limited memory methods
journal, January 1994

Byrd, Richard H.; Nocedal, Jorge; Schnabel, Robert B.
Mathematical Programming, Vol. 63, Issue 1-3
DOI: 10.1007/bf01582063

The Conjugate Gradient Method and Trust Regions in Large Scale Optimization
journal, June 1983

Steihaug, Trond
SIAM Journal on Numerical Analysis, Vol. 20, Issue 3
DOI: 10.1137/0720042

Similar Records in DOE PAGES and OSTI.GOV collections:

A memory-distributed quasi-Newton solver for nonlinear programming problems with a small number of general constraints

Journal Article Petra, Cosmin G. - Journal of Parallel and Distributed Computing

We approach the problem of parallelizing state-of-the-art nonlinear programming optimization algorithms.Specifically, we focus on parallelizing quasi-Newton interior-point methods that use limited-memory secant Hessian approximations. Such interior-point methods are known to have better convergence properties and to be more effective on large-scale problems than gradient-based and derivative-free optimization algorithms. We target nonlinear and potentially nonconvex optimization problems with an arbitrary number of bound constraints and a small number of general equality and inequality constraints on the optimization variables. These problems occur for example in the form of optimal control, optimal design, and inverse problems governed by ordinary or partial differential equations,more »« less
Cited by 11
https://doi.org/10.1016/j.jpdc.2018.10.009

Full Text Available
Convergence properties of trust region methods for linear and convex constraints

Technical Report Burke, J V ; More, J J ; Toraldo, G

We develop a convergence theory for convex and linearly constrained trust region methods which only requires that the step between iterates produce a sufficient reduction in the trust region subproblem. Global convergence is established for a general convex minimization problem while the local analysis is for linearly constrained problems. The main local result establishes that if the sequence converges to a nondegenerate stationary point then the active constraints at the solution are identified in a finite number of iterations. As a consequence of the identification properties, we develop rate of convergence results by assuming that the step is a truncatedmore »« less
Quasi-Newton methods for large scale nonlinear equations and constrained optimization: Progress report, July 1, 1987-June 30, 1988

Technical Report Dennis, Jr, J E ; Tapia, R A

This grant is for research in Quasi-Newton Methods for Large-Scale Nonlinear Equations and Constrained Optimization. The research proposed is divided into major areas: large-scale nonlinear equations and constrained optimization. In nonlinear equations the investigators shall perform extensive theoretical and numerical investigations of trust region methods which have been formulated using arbitrary norms in both the objective function and the specification of the trust region of the model problem. It is expected that the theoretical studies will lead to an effective algorithm that uses linear programming techniques to solve the intermediate model problems. In constrained optimization they shall continue the researchmore »« less
ALESQP: An Augmented Lagrangian Equality-Constrained SQP Method for Optimization with General Constraints

Journal Article Antil, Harbir ; Kouri, Drew Philip ; Ridzal, Denis - SIAM Journal on Optimization

Here we present a new algorithm for infinite-dimensional optimization with general constraints, called ALESQP. In short, ALESQP is an augmented Lagrangian method that penalizes inequality constraints and solves equality-constrained nonlinear optimization subproblems at every iteration. The subproblems are solved using a matrix-free trust-region sequential quadratic programming (SQP) method that takes advantage of iterative, i.e., inexact linear solvers, and is suitable for large-scale applications. A key feature of ALESQP is a constraint decomposition strategy that allows it to exploit problem-specific variable scalings and inner products. We analyze convergence of ALESQP under different assumptions. We show that strong accumulation points are stationary.more »« less
https://doi.org/10.1137/20m1378399
A Newton barrier method for minimizing a sum of norms subject to linear equality constraints, and efficient method to handle the L{sub 1} penalty function

Conference Andersen, K

In this talk we will present a success story of using barrier methods for a large scale non-smooth non-linear convex programming problem. The problem of concern is minimizing a sum of Euclidean norms subject to linear equality constraints (MSN). We will present the duality theory for this problem. This theory is important for the development of an efficient barrier method. We will describe such an algorithm based on the Newton barrier method proposed by K.D. Andersen 1993 for minimizing a sum of norms MSN. The linear equality constraints are handled using an exact Ll penalty function, which is smoothed inmore »« less

Similar Records

Title: Large-Scale Quasi-Newton Trust-Region Methods With Low-Dimensional Linear Equality Constraints

Abstract

Citation Formats

Computing a Trust Region Step journal, September 1983

A trust region method based on interior point techniques for nonlinear programming journal, November 2000

A trust region algorithm for equality constrained optimization journal, November 1990

Compact representation of the full Broyden class of quasi-Newton updates: Compact Representation of the Full Broyden Class of Quasi-Newton Updates journal, May 2018

On the Implementation of an Algorithm for Large-Scale Equality Constrained Optimization journal, August 1998

On efficiently combining limited-memory and trust-region techniques journal, June 2016

A Trust Region Algorithm for Equality Constrained Minimization: Convergence Properties and Implementation journal, June 1985

On solving L-SR1 trust-region subproblems journal, September 2016

The Conjugate Gradient Method and Trust Regions in Large Scale Optimization journal, June 1983

A dense initialization for limited-memory quasi-Newton methods journal, May 2019

Benchmarking optimization software with performance profiles journal, January 2002

Algorithm 943: MSS: MATLAB Software for L-BFGS Trust-Region Subproblems for Large-Scale Optimization journal, June 2014

Updating the Inverse of a Matrix journal, June 1989

On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming journal, April 2005

An Interior Point Algorithm for Large-Scale Nonlinear Programming journal, January 1999

Compact representation of the full Broyden class of quasi-Newton updates preprint, January 2017

Benchmarking Optimization Software with Performance Profiles text, January 2001

Representations of quasi-Newton matrices and their use in limited memory methods journal, January 1994

The Conjugate Gradient Method and Trust Regions in Large Scale Optimization journal, June 1983

Computing a Trust Region Step
journal, September 1983

A trust region method based on interior point techniques for nonlinear programming
journal, November 2000

A trust region algorithm for equality constrained optimization
journal, November 1990

Compact representation of the full Broyden class of quasi-Newton updates: Compact Representation of the Full Broyden Class of Quasi-Newton Updates
journal, May 2018

On the Implementation of an Algorithm for Large-Scale Equality Constrained Optimization
journal, August 1998

On efficiently combining limited-memory and trust-region techniques
journal, June 2016

A Trust Region Algorithm for Equality Constrained Minimization: Convergence Properties and Implementation
journal, June 1985

On solving L-SR1 trust-region subproblems
journal, September 2016

The Conjugate Gradient Method and Trust Regions in Large Scale Optimization
journal, June 1983

A dense initialization for limited-memory quasi-Newton methods
journal, May 2019

Benchmarking optimization software with performance profiles
journal, January 2002

Algorithm 943: MSS: MATLAB Software for L-BFGS Trust-Region Subproblems for Large-Scale Optimization
journal, June 2014

Updating the Inverse of a Matrix
journal, June 1989

On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming
journal, April 2005

An Interior Point Algorithm for Large-Scale Nonlinear Programming
journal, January 1999

Compact representation of the full Broyden class of quasi-Newton updates
preprint, January 2017

Benchmarking Optimization Software with Performance Profiles
text, January 2001

Representations of quasi-Newton matrices and their use in limited memory methods
journal, January 1994

The Conjugate Gradient Method and Trust Regions in Large Scale Optimization
journal, June 1983