Adaptive sampling quasi-Newton methods for zeroth-order stochastic optimization

Bollapragada, Raghu; Wild, Stefan M.

doi:10.1007/s12532-023-00233-9

Adaptive sampling quasi-Newton methods for zeroth-order stochastic optimization

Journal Article · Tue Mar 14 00:00:00 EDT 2023 · Mathematical Programming Computation

DOI:https://doi.org/10.1007/s12532-023-00233-9· OSTI ID:2001234

^[1]; ^[2]

University of Texas at Austin, TX (United States)
Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)

Here, we consider unconstrained stochastic optimization problems with no available gradient information. Such problems arise in settings from derivative-free simulation optimization to reinforcement learning. We propose an adaptive sampling quasi-Newton method where we estimate the gradients using finite differences of stochastic function evaluations within a common random number framework. We develop modified versions of a norm test and an inner product quasi-Newton test to control the sample sizes used in the stochastic approximations and provide global convergence results to the neighborhood of a locally optimal solution. We present numerical experiments on simulation optimization problems to illustrate the performance of the proposed algorithm. When compared with classical zeroth-order stochastic gradient methods, we observe that our strategies of adapting the sample sizes significantly improve performance in terms of the number of stochastic function evaluations required.

View Accepted Manuscript (DOE)

Research Organization:: Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)

Grant/Contract Number:: AC02-05CH11231

OSTI ID:: 2001234

Journal Information:: Mathematical Programming Computation, Journal Name: Mathematical Programming Computation Journal Issue: 2 Vol. 15; ISSN 1867-2949

Publisher:: SpringerCopyright Statement

Country of Publication:: United States

Language:: English

References (44)

A Guide to Sample Average Approximation Kim, Sujin; Pasupathy, Raghu; Henderson, Shane G. International Series in Operations Research & Management Science https://doi.org/10.1007/978-1-4939-1384-8_8	book	September 2014
Derivative-Free and Blackbox Optimization Audet, Charles; Hare, Warren Springer Series in Operations Research and Financial Engineering https://doi.org/10.1007/978-3-319-68913-5	book	January 2017
Variable-Number Sample-Path Optimization Deng, Geng; Ferris, Michael C. Mathematical Programming, Vol. 117, Issue 1-2 https://doi.org/10.1007/s10107-007-0164-y	journal	July 2007
Sample size selection in optimization methods for machine learning Byrd, Richard H.; Chin, Gillian M.; Nocedal, Jorge Mathematical Programming, Vol. 134, Issue 1 https://doi.org/10.1007/s10107-012-0572-5	journal	June 2012
Global convergence rate analysis of unconstrained optimization methods based on probabilistic models Cartis, C.; Scheinberg, K. Mathematical Programming, Vol. 169, Issue 2 https://doi.org/10.1007/s10107-017-1137-4	journal	April 2017
Stochastic optimization using a trust-region method and random models Chen, R.; Menickelly, M.; Scheinberg, K. Mathematical Programming, Vol. 169, Issue 2 https://doi.org/10.1007/s10107-017-1141-8	journal	April 2017
Conditional gradient type methods for composite nonlinear and stochastic optimization Ghadimi, Saeed Mathematical Programming, Vol. 173, Issue 1-2 https://doi.org/10.1007/s10107-017-1225-5	journal	January 2018
Sub-sampled Newton methods Roosta-Khorasani, Farbod; Mahoney, Michael W. Mathematical Programming, Vol. 174, Issue 1-2 https://doi.org/10.1007/s10107-018-1346-5	journal	November 2018
Newton-type methods for non-convex optimization under inexact Hessian information Xu, Peng; Roosta, Fred; Mahoney, Michael W. Mathematical Programming, Vol. 184, Issue 1-2 https://doi.org/10.1007/s10107-019-01405-z	journal	May 2019
Random Gradient-Free Minimization of Convex Functions Nesterov, Yurii; Spokoiny, Vladimir Foundations of Computational Mathematics, Vol. 17, Issue 2 https://doi.org/10.1007/s10208-015-9296-2	journal	November 2015
Zeroth-Order Nonconvex Stochastic Optimization: Handling Constraints, High Dimensionality, and Saddle Points Balasubramanian, Krishnakumar; Ghadimi, Saeed Foundations of Computational Mathematics, Vol. 22, Issue 1 https://doi.org/10.1007/s10208-021-09499-8	journal	March 2021
A Theoretical and Empirical Comparison of Gradient Approximations in Derivative-Free Optimization Berahas, Albert S.; Cao, Liyuan; Choromanski, Krzysztof Foundations of Computational Mathematics, Vol. 22, Issue 2 https://doi.org/10.1007/s10208-021-09513-z	journal	May 2021
Stochastic derivative-free optimization using a trust region framework Larson, Jeffrey; Billups, Stephen C. Computational Optimization and Applications, Vol. 64, Issue 3 https://doi.org/10.1007/s10589-016-9827-z	journal	February 2016
Stochastic mesh adaptive direct search for blackbox optimization using probabilistic estimates Audet, Charles; Dzahini, Kwassi Joseph; Kokkolaras, Michael Computational Optimization and Applications, Vol. 79, Issue 1 https://doi.org/10.1007/s10589-020-00249-0	journal	March 2021
A zeroth order method for stochastic weakly convex optimization Kungurtsev, V.; Rinaldi, F. Computational Optimization and Applications, Vol. 80, Issue 3 https://doi.org/10.1007/s10589-021-00313-3	journal	September 2021
Optimization with hidden constraints and embedded Monte Carlo computations Chen, Xiaojun; Kelley, C. T. Optimization and Engineering, Vol. 17, Issue 1 https://doi.org/10.1007/s11081-015-9302-1	journal	December 2015
Stochastic Nelder–Mead simplex method – A new globally convergent direct search method for simulation optimization Chang, Kuo-Hao European Journal of Operational Research, Vol. 220, Issue 3 https://doi.org/10.1016/j.ejor.2012.02.028	journal	August 2012
Derivative-free optimization methods Larson, Jeffrey; Menickelly, Matt; Wild, Stefan M. Acta Numerica, Vol. 28 https://doi.org/10.1017/s0962492919000060	journal	May 2019
A robust multi-batch L-BFGS method for machine learning Berahas, Albert S.; Takáč, Martin Optimization Methods and Software, Vol. 35, Issue 1 https://doi.org/10.1080/10556788.2019.1658107	journal	August 2019
Optimization and supervised machine learning methods for fitting numerical physics models without derivatives Bollapragada, Raghu; Menickelly, Matt; Nazarewicz, Witold Journal of Physics G: Nuclear and Particle Physics, Vol. 48, Issue 2 https://doi.org/10.1088/1361-6471/abd009	journal	January 2021
Exact and inexact subsampled Newton methods for optimization Bollapragada, Raghu; Byrd, Richard H.; Nocedal, Jorge IMA Journal of Numerical Analysis, Vol. 39, Issue 2 https://doi.org/10.1093/imanum/dry009	journal	April 2018
Optimal Rates for Zero-Order Convex Optimization: The Power of Two Function Evaluations Duchi, John C.; Jordan, Michael I.; Wainwright, Martin J. IEEE Transactions on Information Theory, Vol. 61, Issue 5 https://doi.org/10.1109/TIT.2015.2409256	journal	May 2015
Adaptation of the Uobyqa Algorithm for Noisy Functions Deng, Geng; Ferris, Michael Proceedings of the 2006 Winter Simulation Conference https://doi.org/10.1109/wsc.2006.323088	conference	December 2006
ASTRO-DF: Adaptive sampling trust-region optimization algorithms, heuristics, and numerical experience Shashaani, Sara; Hunter, Susan R.; Pasupathy, Raghu 2016 Winter Simulation Conference (WSC) https://doi.org/10.1109/wsc.2016.7822121	conference	December 2016
Stochastic online optimization. Single-point and multi-point non-linear multi-armed bandits. Convex and strongly-convex case Gasnikov, A. V.; Krymova, E. A.; Lagunovskaya, A. A. Automation and Remote Control, Vol. 78, Issue 2 https://doi.org/10.1134/S0005117917020035	journal	February 2017
Benchmarking Derivative-Free Optimization Algorithms Moré, Jorge J.; Wild, Stefan M. SIAM Journal on Optimization, Vol. 20, Issue 1 https://doi.org/10.1137/080724083	journal	January 2009
Estimating Computational Noise Moré, Jorge J.; Wild, Stefan M. SIAM Journal on Scientific Computing, Vol. 33, Issue 3 https://doi.org/10.1137/100786125	journal	January 2011
Stochastic First- and Zeroth-Order Methods for Nonconvex Stochastic Programming Ghadimi, Saeed; Lan, Guanghui SIAM Journal on Optimization, Vol. 23, Issue 4 https://doi.org/10.1137/120880811	journal	January 2013
On Sampling Rates in Simulation-Based Recursions Pasupathy, Raghu; Glynn, Peter; Ghosh, Soumyadip SIAM Journal on Optimization, Vol. 28, Issue 1 https://doi.org/10.1137/140951679	journal	January 2018
ASTRO-DF: A Class of Adaptive Sampling Trust-Region Algorithms for Derivative-Free Stochastic Optimization Shashaani, Sara; Hashemi, Fatemeh S.; Pasupathy, Raghu SIAM Journal on Optimization, Vol. 28, Issue 4 https://doi.org/10.1137/15m1042425	journal	January 2018
Optimization Methods for Large-Scale Machine Learning Bottou, Léon; Curtis, Frank E.; Nocedal, Jorge SIAM Review, Vol. 60, Issue 2 https://doi.org/10.1137/16m1080173	journal	January 2018
A Smoothing Direct Search Method for Monte Carlo-Based Bound Constrained Composite Nonsmooth Optimization Chen, Xiaojun; Kelley, C. T.; Xu, Fengmin SIAM Journal on Scientific Computing, Vol. 40, Issue 4 https://doi.org/10.1137/17m1116714	journal	January 2018
Adaptive Sampling Strategies for Stochastic Optimization Bollapragada, Raghu; Byrd, Richard; Nocedal, Jorge SIAM Journal on Optimization, Vol. 28, Issue 4 https://doi.org/10.1137/17m1154679	journal	January 2018
Derivative-Free Optimization of Noisy Functions via Quasi-Newton Methods Berahas, Albert S.; Byrd, Richard H.; Nocedal, Jorge SIAM Journal on Optimization, Vol. 29, Issue 2 https://doi.org/10.1137/18m1177718	journal	January 2019
Analysis of the BFGS Method with Errors Xie, Yuchen; Byrd, Richard H.; Nocedal, Jorge SIAM Journal on Optimization, Vol. 30, Issue 1 https://doi.org/10.1137/19m1240794	journal	January 2020
Budget-Dependent Convergence Rate of Stochastic Approximation L'Ecuyer, Pierre; Yin, George SIAM Journal on Optimization, Vol. 8, Issue 1 https://doi.org/10.1137/S1052623495270723	journal	February 1998
Estimating Derivatives of Noisy Simulations Moré, Jorge J.; Wild, Stefan M. ACM Transactions on Mathematical Software, Vol. 38, Issue 3 https://doi.org/10.1145/2168773.2168777	journal	April 2012
CUTEr and SifDec: A constrained and unconstrained testing environment, revisited Gould, Nicholas I. M.; Orban, Dominique; Toint, Philippe L. ACM Transactions on Mathematical Software, Vol. 29, Issue 4 https://doi.org/10.1145/962437.962439	journal	December 2003
Multidimensional Stochastic Approximation Methods Blum, Julius R. The Annals of Mathematical Statistics, Vol. 25, Issue 4 https://doi.org/10.1214/aoms/1177728659	journal	December 1954
Stochastic Estimation of the Maximum of a Regression Function Kiefer, J.; Wolfowitz, J. The Annals of Mathematical Statistics, Vol. 23, Issue 3 https://doi.org/10.1214/aoms/1177729392	journal	September 1952
A Stochastic Approximation Method Robbins, Herbert; Monro, Sutton The Annals of Mathematical Statistics, Vol. 22, Issue 3 https://doi.org/10.1214/aoms/1177729586	journal	September 1951
Convergence Rate Analysis of a Stochastic Trust-Region Method via Supermartingales Blanchet, Jose; Cartis, Coralia; Menickelly, Matt INFORMS Journal on Optimization, Vol. 1, Issue 2 https://doi.org/10.1287/ijoo.2019.0016	journal	April 2019
Simulation-Based Optimization with Stochastic Approximation Using Common Random Numbers Kleinman, Nathan L.; Spall, James C.; Naiman, Daniel Q. Management Science, Vol. 45, Issue 11 https://doi.org/10.1287/mnsc.45.11.1570	journal	November 1999
Faster Gradient-Free Proximal Stochastic Methods for Nonconvex Nonsmooth Optimization Huang, Feihu; Gu, Bin; Huo, Zhouyuan Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33, Issue 01 https://doi.org/10.1609/aaai.v33i01.33011503	journal	July 2019