A randomized sketching trust-region secant method for low-memory dynamic optimization

Vuchkov, Radoslav Georgiev; Kouri, Drew Philip

doi:10.1007/s11590-025-02216-2

A randomized sketching trust-region secant method for low-memory dynamic optimization

Journal Article · Wed Jun 25 00:00:00 EDT 2025 · Optimization Letters

DOI:https://doi.org/10.1007/s11590-025-02216-2· OSTI ID:2999462

^[1]; ^[1]

Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)

The numerical solution of dynamic optimization problems is often limited by the memory required to store the state trajectory, which is used to evaluate the objective function and its derivatives. Recently, [R. Muthukumar et al., SIAM Journal on Optimization 31(2), pp. 1242–1275 (2021)] introduced a trust-region method for dynamic optimization that employs randomized sketching to compress the state trajectory, resulting in inexact derivative computations. By adaptively learning the sketch rank, the trust-region algorithm achieves rigorous convergence guarantees. Here, we extend this approach to use secant Hessian approximations. Due to the randomness introduced by the sketch, the traditional secant update formulae can produce poor Hessian approximations. In particular, the difference of two gradients, computed from two different sketches, may be inconsistent. To overcome this, we employ a sketched approximation of the Hessian application, in lieu of computing the gradient difference. We numerically demonstrate the improved stability of this approach on an example from PDE-constrained optimization.

View Accepted Manuscript (DOE)

Research Organization:: Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA)

Grant/Contract Number:: NA0003525

OSTI ID:: 2999462

Report Number(s):: SAND--2025-12569J; 1742859

Journal Information:: Optimization Letters, Journal Name: Optimization Letters; ISSN 1862-4472; ISSN 1862-4480

Publisher:: Springer NatureCopyright Statement

Country of Publication:: United States

Language:: English

References (27)

Reduced Order Modelling Approaches to PDE-Constrained Optimization Based on Proper Orthogonal Decomposition Fahl, Marco; Sachs, Ekkehard W. Large-Scale PDE-Constrained Optimization https://doi.org/10.1007/978-3-642-55508-4_16	book	January 2003
Representations of quasi-Newton matrices and their use in limited memory methods Byrd, Richard H.; Nocedal, Jorge; Schnabel, Robert B. Mathematical Programming, Vol. 63, Issue 1-3 https://doi.org/10.1007/BF01582063	journal	January 1994
On the limited memory BFGS method for large scale optimization Liu, Dong C.; Nocedal, Jorge Mathematical Programming, Vol. 45, Issue 1-3 https://doi.org/10.1007/BF01589116	journal	August 1989
Numerical Optimization Nocedal, Jorge; Wright, Stephen J. Springer Series in Operations Research and Financial Engineering https://doi.org/10.1007/b98874	book	January 1999
A proximal trust-region method for nonsmooth optimization with inexact function and gradient evaluations Baraldi, Robert J.; Kouri, Drew P. Mathematical Programming, Vol. 201, Issue 1-2 https://doi.org/10.1007/s10107-022-01915-3	journal	December 2022
Efficient proximal subproblem solvers for a nonsmooth trust-region method Baraldi, Robert J.; Kouri, Drew P. Computational Optimization and Applications, Vol. 90, Issue 1 https://doi.org/10.1007/s10589-024-00628-x	journal	January 2025
Optical tomography using the time-independent equation of radiative transfer—Part 2: inverse model Klose, Alexander D.; Hielscher, Andreas H. Journal of Quantitative Spectroscopy and Radiative Transfer, Vol. 72, Issue 5 https://doi.org/10.1016/S0022-4073(01)00151-0	journal	March 2002
A Mathematical Model Illustrating the Theory of Turbulence Burgers, J. M. Advances in Applied Mechanics https://doi.org/10.1016/S0065-2156(08)70100-5	book	January 1948
A numerical study of limited memory BFGS methods Morales, J. L. Applied Mathematics Letters, Vol. 15, Issue 4 https://doi.org/10.1016/S0893-9659(01)00162-8	journal	May 2002
Optical tomography in medical imaging Arridge, S. R. Inverse Problems, Vol. 15, Issue 2 https://doi.org/10.1088/0266-5611/15/2/022	journal	January 1999
A Newton-CG method for large-scale three-dimensional elastic full-waveform seismic inversion Epanomeritakis, I.; Akçelik, V.; Ghattas, O. Inverse Problems, Vol. 24, Issue 3 https://doi.org/10.1088/0266-5611/24/3/034015	journal	May 2008
On the Global Convergence of Trust Region Algorithms Using Inexact Gradient Information Carter, Richard G. SIAM Journal on Numerical Analysis, Vol. 28, Issue 1 https://doi.org/10.1137/0728014	journal	February 1991
Minimal Repetition Dynamic Checkpointing Algorithm for Unsteady Adjoint Calculation Wang, Qiqi; Moin, Parviz; Iaccarino, Gianluca SIAM Journal on Scientific Computing, Vol. 31, Issue 4 https://doi.org/10.1137/080727890	journal	January 2009
New Algorithms for Optimal Online Checkpointing Stumm, Philipp; Walther, Andrea SIAM Journal on Scientific Computing, Vol. 32, Issue 2 https://doi.org/10.1137/080742439	journal	January 2010
Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions Halko, N.; Martinsson, P. G.; Tropp, J. A. SIAM Review, Vol. 53, Issue 2 https://doi.org/10.1137/090771806	journal	January 2011
Quasi-Newton Methods, Motivation and Theory Dennis, Jr., J. E.; Moré, Jorge J. SIAM Review, Vol. 19, Issue 1 https://doi.org/10.1137/1019005	journal	January 1977
A Trust-Region Algorithm with Adaptive Stochastic Collocation for PDE Optimization under Uncertainty Kouri, D. P.; Heinkenschloss, M.; Ridzal, D. SIAM Journal on Scientific Computing, Vol. 35, Issue 4 https://doi.org/10.1137/120892362	journal	January 2013
A Stochastic Quasi-Newton Method for Large-Scale Optimization Byrd, R. H.; Hansen, S. L.; Nocedal, Jorge SIAM Journal on Optimization, Vol. 26, Issue 2 https://doi.org/10.1137/140954362	journal	January 2016
Optimal Multistage Algorithm for Adjoint Computation Aupy, Guillaume; Herrmann, Julien; Hovland, Paul SIAM Journal on Scientific Computing, Vol. 38, Issue 3 https://doi.org/10.1137/15M1019222	journal	January 2016
Practical Sketching Algorithms for Low-Rank Matrix Approximation Tropp, Joel A.; Yurtsever, Alp; Udell, Madeleine SIAM Journal on Matrix Analysis and Applications, Vol. 38, Issue 4 https://doi.org/10.1137/17M1111590	journal	January 2017
Streaming Low-Rank Matrix Approximation with an Application to Scientific Simulation Tropp, Joel A.; Yurtsever, Alp; Udell, Madeleine SIAM Journal on Scientific Computing, Vol. 41, Issue 4 https://doi.org/10.1137/18M1201068	journal	January 2019
An Efficient, Globally Convergent Method for Optimization Under Uncertainty Using Adaptive Model Reduction and Sparse Grids Zahr, Matthew J.; Carlberg, Kevin T.; Kouri, Drew P. SIAM/ASA Journal on Uncertainty Quantification, Vol. 7, Issue 3 https://doi.org/10.1137/18M1220996	journal	January 2019
Low-Rank Tucker Approximation of a Tensor from Streaming Data Sun, Yiming; Guo, Yang; Luo, Charlene SIAM Journal on Mathematics of Data Science, Vol. 2, Issue 4 https://doi.org/10.1137/19M1257718	journal	January 2020
Randomized Sketching Algorithms for Low-Memory Dynamic Optimization Muthukumar, Ramchandran; Kouri, Drew P.; Udell, Madeleine SIAM Journal on Optimization, Vol. 31, Issue 2 https://doi.org/10.1137/19M1272561	journal	January 2021
Analysis of a Symmetric Rank-One Trust Region Method Byrd, Richard H.; Khalfan, Humaid Fayez; Schnabel, Robert B. SIAM Journal on Optimization, Vol. 6, Issue 4 https://doi.org/10.1137/S1052623493252985	journal	November 1996
Algorithm 799: revolve: an implementation of checkpointing for the reverse or adjoint mode of computational differentiation Griewank, Andreas; Walther, Andrea ACM Transactions on Mathematical Software, Vol. 26, Issue 1 https://doi.org/10.1145/347837.347846	journal	March 2000
Fast full-wavefield seismic inversion using encoded sources Krebs, Jerome R.; Anderson, John E.; Hinkley, David GEOPHYSICS, Vol. 74, Issue 6 https://doi.org/10.1190/1.3230502	journal	November 2009

Similar Records

Randomized Sketching Algorithms for Low-Memory Dynamic Optimization

Journal Article · Sun May 09 20:00:00 EDT 2021 · SIAM Journal on Optimization · OSTI ID:1784618

An inexact semismooth Newton method with application to adaptive randomized sketching for dynamic optimization

Journal Article · Tue Oct 17 20:00:00 EDT 2023 · Finite Elements in Analysis and Design · OSTI ID:2311364

An investigation of Newton-Sketch and subsampled Newton methods

Journal Article · Tue Feb 11 19:00:00 EST 2020 · Optimization Methods and Software · OSTI ID:1657509

Related Subjects

dynamic optimization
quasi-Newton
randomized sketching
trust regions

A randomized sketching trust-region secant method for low-memory dynamic optimization

Citation Formats

References (27)

Similar Records

Related Subjects