Stochastic average model methods

Journal Article · 24 April 2024 · Computational Optimization and Applications

DOI: https://doi.org/10.1007/s10589-024-00563-x · OSTI ID:2476911

Menickelly, Matt  ^[1]; Wild, Stefan M.  ^[2]

---

1. Argonne National Laboratory (ANL), Argonne, IL (United States)
2. Argonne National Laboratory (ANL), Argonne, IL (United States); Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)

We consider the solution of finite-sum minimization problems, such as those appearing in nonlinear least-squares or general empirical risk minimization problems. We are motivated by problems in which the summand functions are computationally expensive and evaluating all summands on every iteration of an optimization method may be undesirable. Here we present the idea of stochastic average model (SAM) methods, inspired by stochastic average gradient methods. SAM methods sample component functions on each iteration of a trust-region method according to a discrete probability distribution on component functions; the distribution is designed to minimize an upper bound on the variance of the resulting stochastic model. We present promising numerical results concerning an implemented variant extending the derivative-free model-based trust-region solver POUNDERS, which we name SAM-POUNDERS.

• Research Organization: Argonne National Laboratory (ANL), Argonne, IL (United States)
• Sponsoring Organization: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR). Scientific Discovery through Advanced Computing (SciDAC)
• Grant/Contract Number: AC02-06CH11357
• OSTI ID: 2476911
• Journal Information: Computational Optimization and Applications, Journal Name: Computational Optimization and Applications Journal Issue: 2 Vol. 88; ISSN 0926-6003
• Publisher: SpringerCopyright Statement
• Country of Publication: United States
• Language: English