Stochastic Trust-Region Algorithm in Random Subspaces with Convergence and Expected Complexity Analyses

Journal Article · Thu Jul 25 00:00:00 EDT 2024 · SIAM Journal on Optimization

DOI:https://doi.org/10.1137/22m1524072· OSTI ID:2480304

Dzahini, K. J. ^[1]; Wild, S. M. ^[2]

---

1. Argonne National Laboratory (ANL), Argonne, IL (United States)
2. Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)

Here, this work proposes a framework for large-scale stochastic derivative-free optimization (DFO) by introducing STARS, a trust-region method based on iterative minimization in random subspaces. This framework is both an algorithmic and theoretical extension of a random subspace derivative-free optimization (RSDFO) framework, and an algorithm for stochastic optimization with random models (STORM). Moreover, like RSDFO, STARS achieves scalability by minimizing interpolation models that approximate the objective in low-dimensional affine subspaces, thus significantly reducing per-iteration costs in terms of function evaluations and yielding strong performance on largescale stochastic DFO problems. The user-determined dimension of these subspaces, when the latter are defined, for example, by the columns of so-called Johnson-Lindenstrauss transforms, turns out to be independent of the dimension of the problem. For convergence purposes, inspired by the analyses of RSDFO and STORM, both a particular quality of the subspace and the accuracies of random function estimates and models are required to hold with sufficiently high, but fixed, probabilities. Using martingale theory under the latter assumptions, an almost sure global convergence of STARS to a first-order stationary point is shown, and the expected number of iterations required to reach a desired first-order accuracy is proved to be similar to that of STORM and other stochastic DFO algorithms, up to constants.

• Research Organization: Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)
• Sponsoring Organization: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR). Scientific Discovery through Advanced Computing (SciDAC)
• Grant/Contract Number: AC02-05CH11231; AC02-06CH11357
• OSTI ID: 2480304
• Journal Information: SIAM Journal on Optimization, Journal Name: SIAM Journal on Optimization Journal Issue: 3 Vol. 34; ISSN 1052-6234
• Publisher: Society for Industrial and Applied Mathematics (SIAM)
• Country of Publication: United States
• Language: English

Related Subjects

97 MATHEMATICS AND COMPUTING
randomized subspace methods
stochastic derivative-free optimization
zeroth-order methods