ALESQP: An Augmented Lagrangian Equality-Constrained SQP Method for Optimization with General Constraints

Antil, Harbir; Kouri, Drew Philip; Ridzal, Denis

doi:10.1137/20m1378399

Title: ALESQP: An Augmented Lagrangian Equality-Constrained SQP Method for Optimization with General Constraints

Journal Article · 20 November 2023 · SIAM Journal on Optimization

DOI: https://doi.org/10.1137/20m1378399 · OSTI ID:2311683

^[1];

^[2]; Ridzal, Denis ^[2]

George Mason University, Fairfax, VA (United States)
Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States)

Here we present a new algorithm for infinite-dimensional optimization with general constraints, called ALESQP. In short, ALESQP is an augmented Lagrangian method that penalizes inequality constraints and solves equality-constrained nonlinear optimization subproblems at every iteration. The subproblems are solved using a matrix-free trust-region sequential quadratic programming (SQP) method that takes advantage of iterative, i.e., inexact linear solvers, and is suitable for large-scale applications. A key feature of ALESQP is a constraint decomposition strategy that allows it to exploit problem-specific variable scalings and inner products. We analyze convergence of ALESQP under different assumptions. We show that strong accumulation points are stationary. Consequently, in finite dimensions ALESQP converges to a stationary point. In infinite dimensions we establish that weak accumulation points are feasible in many practical situations. Under additional assumptions we show that weak accumulation points are stationary. We present several infinite-dimensional examples where ALESQP shows remarkable discretization-independent performance in all of its iterative components, requiring a modest number of iterations to meet constraint tolerances at the level of machine precision. Also, we demonstrate a fully matrix-free solution of an infinite-dimensional problem with nonlinear inequality constraints.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA); National Science Foundation (NSF); Air Force Office of Scientific Research (AFOSR); USDOE Laboratory Directed Research and Development (LDRD) Program

Grant/Contract Number:: NA0003525

OSTI ID:: 2311683

Report Number(s):: SAND--2023-12921J

Journal Information:: SIAM Journal on Optimization, Journal Name: SIAM Journal on Optimization Journal Issue: 1 Vol. 33; ISSN 1052-6234

Publisher:: Society for Industrial and Applied Mathematics (SIAM)Copyright Statement

Country of Publication:: United States

Language:: English

References (38)

Projections on Convex Sets in Hilbert Space and Spectral Theory: Part I. Projections on Convex Sets: Part II. Spectral Theory Zarantonello, Eduardo H. Contributions to Nonlinear Functional Analysis https://doi.org/10.1016/B978-0-12-775850-3.50013-3	book	January 1971
On Augmented Lagrangian Methods with General Lower-Level Constraints Andreani, R.; Birgin, E. G.; Martínez, J. M. SIAM Journal on Optimization, Vol. 18, Issue 4 https://doi.org/10.1137/060654797	journal	January 2008
Affine Invariant Convergence Analysis for Inexact Augmented Lagrangian-SQP Methods Volkwein, S.; Weiser, M. SIAM Journal on Control and Optimization, Vol. 41, Issue 3 https://doi.org/10.1137/S0363012900383089	journal	January 2002
A Matrix-Free Trust-Region SQP Method for Equality Constrained Optimization Heinkenschloss, Matthias; Ridzal, Denis SIAM Journal on Optimization, Vol. 24, Issue 3 https://doi.org/10.1137/130921738	journal	January 2014
Augmented Lagrange Multiplier Functions and Duality in Nonconvex Programming Rockafellar, R. Tyrrell SIAM Journal on Control, Vol. 12, Issue 2 https://doi.org/10.1137/0312021	journal	May 1974
Smoothing Methods and Semismooth Methods for Nondifferentiable Operator Equations Chen, Xiaojun; Nashed, Zuhair; Qi, Liqun SIAM Journal on Numerical Analysis, Vol. 38, Issue 4 https://doi.org/10.1137/S0036142999356719	journal	January 2000
Inexact Trust-Region Methods for PDE-Constrained Optimization Kouri, Drew P.; Ridzal, Denis Frontiers in PDE-Constrained Optimization https://doi.org/10.1007/978-1-4939-8636-1_3	book	January 2018
Control of an Elliptic Problem with Pointwise State Constraints Casas, Eduardo SIAM Journal on Control and Optimization, Vol. 24, Issue 6 https://doi.org/10.1137/0324078	journal	November 1986
Augmented Lagrangian-SQP-Methods in Hilbert Spaces and Application to Control in the Coefficients Problems Ito, Kazufumi; Kunisch, Karl SIAM Journal on Optimization, Vol. 6, Issue 1 https://doi.org/10.1137/0806007	journal	February 1996
Lagrange Multiplier Approach to Variational Problems and Applications Ito, Kazufumi; Kunisch, Karl https://doi.org/10.1137/1.9780898718614	book	January 2008
A Global Convergence Theory for General Trust-Region-Based Algorithms for Equality Constrained Optimization Dennis, J. E.; El-Alem, Mahmoud; Maciel, Maria C. SIAM Journal on Optimization, Vol. 7, Issue 1 https://doi.org/10.1137/S1052623492238881	journal	February 1997
Regularity and stability for the mathematical programming problem in Banach spaces Zowe, J.; Kurcyusz, S. Applied Mathematics & Optimization, Vol. 5, Issue 1 https://doi.org/10.1007/BF01442543	journal	March 1979
The SQP method for control constrained optimal control of the Burgers equation Tröltzsch, Fredi; Volkwein, Stefan ESAIM: Control, Optimisation and Calculus of Variations, Vol. 6 https://doi.org/10.1051/cocv:2001127	journal	January 2001
Certain Generalizations in the Analysis of Variance Wilks, S. S. Biometrika, Vol. 24, Issue 3-4 https://doi.org/10.1093/biomet/24.3-4.471	journal	January 1932
Mesh-Independence for an Augmented Lagrangian-SQP Method in Hilbert Spaces Volkwein, S. SIAM Journal on Control and Optimization, Vol. 38, Issue 3 https://doi.org/10.1137/S0363012998334468	journal	January 2000
An augmented Lagrangian technique for variational inequalities Ito, K.; Kunisch, K. Applied Mathematics & Optimization, Vol. 21, Issue 1 https://doi.org/10.1007/BF01445164	journal	January 1990
An Augmented Lagrangian Method for Optimization Problems in Banach Spaces Kanzow, Christian; Steck, Daniel; Wachsmuth, Daniel SIAM Journal on Control and Optimization, Vol. 56, Issue 1 https://doi.org/10.1137/16M1107103	journal	January 2018
On the Solution of Equality Constrained Quadratic Programming Problems Arising in Optimization Gould, Nicholas I. M.; Hribar, Mary E.; Nocedal, Jorge SIAM Journal on Scientific Computing, Vol. 23, Issue 4 https://doi.org/10.1137/S1064827598345667	journal	January 2001
Augmented Lagrangian method for distributed optimal control problems with state constraints Bergounioux, M. Journal of Optimization Theory and Applications, Vol. 78, Issue 3 https://doi.org/10.1007/BF00939879	journal	September 1993
Augmented Lagrangians and Applications of the Proximal Point Algorithm in Convex Programming Rockafellar, R. T. Mathematics of Operations Research, Vol. 1, Issue 2 https://doi.org/10.1287/moor.1.2.97	journal	May 1976
Rényi Divergence and Kullback-Leibler Divergence van Erven, Tim; Harremoes, Peter IEEE Transactions on Information Theory, Vol. 60, Issue 7 https://doi.org/10.1109/TIT.2014.2320500	journal	July 2014
Solution of Sparse Indefinite Systems of Linear Equations Paige, C. C.; Saunders, M. A. SIAM Journal on Numerical Analysis, Vol. 12, Issue 4 https://doi.org/10.1137/0712047	journal	September 1975
The multiplier method of Hestenes and Powell applied to convex programming Rockafellar, R. T. Journal of Optimization Theory and Applications, Vol. 12, Issue 6 https://doi.org/10.1007/BF00934777	journal	December 1973
The Conjugate Gradient Method and Trust Regions in Large Scale Optimization Steihaug, Trond SIAM Journal on Numerical Analysis, Vol. 20, Issue 3 https://doi.org/10.1137/0720042	journal	June 1983
A New Algorithm for Unconstrained Optimization Powell, M. J. D. Nonlinear Programming https://doi.org/10.1016/B978-0-12-597050-1.50006-3	book	January 1970
Finite Element Discretization of State-Constrained Elliptic Optimal Control Problems with Semilinear State Equation Neitzel, Ira; Pfefferer, Johannes; Rösch, Arnd SIAM Journal on Control and Optimization, Vol. 53, Issue 2 https://doi.org/10.1137/140960645	journal	January 2015
Local and Global Analysis of Multiplier Methods for Constrained Optimization in Banach Spaces Börgens, Eike; Kanzow, Christian; Steck, Daniel SIAM Journal on Control and Optimization, Vol. 57, Issue 6 https://doi.org/10.1137/19M1240186	journal	January 2019
Convex Analysis and Monotone Operator Theory in Hilbert Spaces Bauschke, Heinz H.; Combettes, Patrick L. CMS Books in Mathematics https://doi.org/10.1007/978-1-4419-9467-7	book	December 2013
Practical Augmented Lagrangian Methods for Constrained Optimization Birgin, E. G.; Martínez, J. M. https://doi.org/10.1137/1.9781611973365	book	January 2014
The augmented lagrangian method for equality and inequality constraints in hilbert spaces Ito, K.; Kunisch, K. Mathematical Programming, Vol. 46, Issue 1-3 https://doi.org/10.1007/BF01585750	journal	January 1990
Multiplier and gradient methods Hestenes, Magnus R. Journal of Optimization Theory and Applications, Vol. 4, Issue 5 https://doi.org/10.1007/BF00927673	journal	November 1969
Sequential equality-constrained optimization for nonlinear programming Birgin, E. G.; Bueno, L. F.; Martínez, J. M. Computational Optimization and Applications, Vol. 65, Issue 3 https://doi.org/10.1007/s10589-016-9849-6	journal	May 2016
Safeguarded Augmented Lagrangian Methods in Banach Spaces Kanzow, Christian; Karl, Veronika; Wachsmuth, Daniel Daniel Steck, International Series of Numerical Mathematics https://doi.org/10.1007/978-3-030-79393-7_10	book	June 2021
A Globally Convergent Augmented Lagrangian Algorithm for Optimization with General Constraints and Simple Bounds Conn, Andrew R.; Gould, Nicholas I. M.; Toint, Philippe SIAM Journal on Numerical Analysis, Vol. 28, Issue 2 https://doi.org/10.1137/0728030	journal	April 1991
Quasi-Variational Inequalities in Banach Spaces: Theory and Augmented Lagrangian Methods Kanzow, Christian; Steck, Daniel SIAM Journal on Optimization, Vol. 29, Issue 4 https://doi.org/10.1137/18M1230475	journal	January 2019
Adaptive Multilevel Inexact SQP Methods for PDE-Constrained Optimization Ziems, J. Carsten; Ulbrich, Stefan SIAM Journal on Optimization, Vol. 21, Issue 1 https://doi.org/10.1137/080743160	journal	January 2011
Trust Region Methods Conn, Andrew R.; Gould, Nicholas I. M.; Toint, Philippe L. MOS-SIAM Series on Optimization https://doi.org/10.1137/1.9780898719857	book	January 2000
Augemented Lagrangian Techniques for Elliptic State Constrained Optimal Control Problems Bergounioux, Maitine; Kunisch, Karl SIAM Journal on Control and Optimization, Vol. 35, Issue 5 https://doi.org/10.1137/S036301299529330X	journal	September 1997

Similar Records

A Strongly and Superlinearly Convergent SQP Algorithm for Optimization Problems with Linear Complementarity Constraints

Journal Article · 2006 · Applied Mathematics and Optimization · OSTI ID:21064208

Jianling, Li; Xingde, Mo

Sequential quadratic programming algorithms for optimization

Technical Report · 1989 · OSTI ID:5325989

Prieto, F. J.

Large-scale sequential quadratic programming algorithms

Technical Report · 1992 · OSTI ID:10102731

Eldersveld, S K

Related Subjects

97 MATHEMATICS AND COMPUTING
ALESQP
augmented Lagrangian
composite step trust-region method
SQQP
convergence analysis
constraint decomposition
nonlinear constraints

Title: ALESQP: An Augmented Lagrangian Equality-Constrained SQP Method for Optimization with General Constraints

Citation Formats

References (38)

Similar Records

Related Subjects