On the Derivation of Quasi-Newton Formulas for Optimization in Function Spaces
Newton’s method is usually preferred when solving optimization problems due to its superior convergence properties compared to gradient-based or derivative-free optimization algorithms. However, deriving and computing second-order derivatives needed by Newton’s method often is not trivial and, in some cases, not possible. In such cases quasi-Newton algorithms are a great alternative. In this paper, we provide a new derivation of well-known quasi-Newton formulas in an infinite-dimensional Hilbert space setting. Furthermore, it is known that quasi-Newton update formulas are solutions to certain variational problems over the space of symmetric matrices. In this paper, we formulate similar variational problems over the space of bounded symmetric operators in Hilbert spaces. By changing the constraints of the variational problem we obtain updates (for the Hessian and Hessian inverse) not only for the Broyden-Fletcher-Goldfarb-Shanno (BFGS) quasi-Newton method but also for Davidon–Fletcher–Powell (DFP), Symmetric Rank One (SR1), and Powell-Symmetric-Broyden (PSB). In addition, for an inverse problem governed by a partial differential equation (PDE), we derive DFP and BFGS “structured” secant formulas that explicitly use the derivative of the regularization and only approximates the second derivative of the misfit term. We show numerical results that demonstrate the desired mesh-independence property and superior performance of the resulting quasi-Newton methods.
- Research Organization:
- Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States)
- Sponsoring Organization:
- USDOE National Nuclear Security Administration (NNSA); National Science Foundation (NSF)
- Grant/Contract Number:
- AC52-07NA27344; 1654311
- OSTI ID:
- 1804288
- Report Number(s):
- LLNL-JRNL-818069; 1027937
- Journal Information:
- Numerical Functional Analysis and Optimization, Vol. 41, Issue 13; ISSN 0163-0563
- Publisher:
- Taylor & FrancisCopyright Statement
- Country of Publication:
- United States
- Language:
- English
Similar Records
A Structured Quasi-Newton Algorithm for Optimizing with Incomplete Hessian Information
Extension of quasi-Newton methods to constrained optimization and to general systems of nonlinear equations and inequalities. Final report, February 1, 1976--January 31, 1977