DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Derivative-free optimization methods

Abstract

In many optimization problems arising from scientific, engineering and artificial intelligence applications, objective and constraint functions are available only as the output of a black-box or simulation oracle that does not provide derivative information. Such settings necessitate the use of methods for derivative-free, or zeroth-order, optimization. We provide a review and perspectives on developments in these methods, with an emphasis on highlighting recent developments and on unifying treatment of such problems in the non-linear optimization and machine learning literature. We categorize methods based on assumed properties of the black-box functions, as well as features of the methods. We first overview the primary setting of deterministic methods applied to unconstrained, non-convex optimization problems where the objective function is defined by a deterministic black-box oracle. We then discuss developments in randomized methods, methods that assume some additional structure about the objective (including convexity, separability and general non-smooth compositions), methods for problems where the output of the black-box oracle is stochastic, and methods for handling different types of constraints.

Authors:
 [1];  [1];  [1]
  1. Argonne National Lab. (ANL), Argonne, IL (United States)
Publication Date:
Research Org.:
Argonne National Laboratory (ANL), Argonne, IL (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
OSTI Identifier:
1545343
Grant/Contract Number:  
AC02-06CH11357
Resource Type:
Accepted Manuscript
Journal Name:
Acta Numerica
Additional Journal Information:
Journal Volume: 28; Journal Issue: 2010; Journal ID: ISSN 0962-4929
Publisher:
Cambridge University Press
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; Optimization

Citation Formats

Larson, Jeffrey, Menickelly, Matt, and Wild, Stefan M. Derivative-free optimization methods. United States: N. p., 2019. Web. doi:10.1017/S0962492919000060.
Larson, Jeffrey, Menickelly, Matt, & Wild, Stefan M. Derivative-free optimization methods. United States. https://doi.org/10.1017/S0962492919000060
Larson, Jeffrey, Menickelly, Matt, and Wild, Stefan M. Wed . "Derivative-free optimization methods". United States. https://doi.org/10.1017/S0962492919000060. https://www.osti.gov/servlets/purl/1545343.
@article{osti_1545343,
title = {Derivative-free optimization methods},
author = {Larson, Jeffrey and Menickelly, Matt and Wild, Stefan M.},
abstractNote = {In many optimization problems arising from scientific, engineering and artificial intelligence applications, objective and constraint functions are available only as the output of a black-box or simulation oracle that does not provide derivative information. Such settings necessitate the use of methods for derivative-free, or zeroth-order, optimization. We provide a review and perspectives on developments in these methods, with an emphasis on highlighting recent developments and on unifying treatment of such problems in the non-linear optimization and machine learning literature. We categorize methods based on assumed properties of the black-box functions, as well as features of the methods. We first overview the primary setting of deterministic methods applied to unconstrained, non-convex optimization problems where the objective function is defined by a deterministic black-box oracle. We then discuss developments in randomized methods, methods that assume some additional structure about the objective (including convexity, separability and general non-smooth compositions), methods for problems where the output of the black-box oracle is stochastic, and methods for handling different types of constraints.},
doi = {10.1017/S0962492919000060},
journal = {Acta Numerica},
number = 2010,
volume = 28,
place = {United States},
year = {2019},
month = {5}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Citation Metrics:
Cited by: 129 works
Citation information provided by
Web of Science

Save / Share:

Works referenced in this record:

Sequential Design and Analysis of High-Accuracy and Low-Accuracy Computer Codes
journal, August 2012


Manifold Sampling for $\ell_1$ Nonconvex Optimization
journal, January 2016

  • Larson, Jeffrey; Menickelly, Matt; Wild, Stefan M.
  • SIAM Journal on Optimization, Vol. 26, Issue 4
  • DOI: 10.1137/15M1042097

Mesh adaptive direct search algorithms for mixed variable optimization
journal, June 2008


A trust-region derivative-free algorithm for constrained optimization
journal, April 2015


Convergence estimates for iterative minimization methods
journal, January 1974


Derivative-Free and Blackbox Optimization
book, January 2017

  • Audet, Charles; Hare, Warren
  • Springer Series in Operations Research and Financial Engineering
  • DOI: 10.1007/978-3-319-68913-5

Algorithm 897: VTDIRECT95: Serial and parallel codes for the global optimization algorithm direct
journal, July 2009

  • He, Jian; Watson, Layne T.; Sosonkina, Masha
  • ACM Transactions on Mathematical Software, Vol. 36, Issue 3
  • DOI: 10.1145/1527286.1527291

A Pattern Search Filter Method for Nonlinear Programming without Derivatives
journal, January 2004


On the Local Convergence of Pattern Search
journal, January 2003

  • Dolan, Elizabeth D.; Lewis, Robert Michael; Torczon, Virginia
  • SIAM Journal on Optimization, Vol. 14, Issue 2
  • DOI: 10.1137/S1052623400374495

Global Convergence of Radial Basis Function Trust-Region Algorithms for Derivative-Free Optimization
journal, January 2013

  • Wild, Stefan M.; Shoemaker, Christine
  • SIAM Review, Vol. 55, Issue 2
  • DOI: 10.1137/120902434

An active-set trust-region method for derivative-free nonlinear bound-constrained optimization
journal, October 2011

  • Gratton, Serge; Toint, Philippe L.; Tröltzsch, Anke
  • Optimization Methods and Software, Vol. 26, Issue 4-5
  • DOI: 10.1080/10556788.2010.549231

Multiobjective Optimization Through a Series of Single-Objective Formulations
journal, January 2008

  • Audet, Charles; Savard, Gilles; Zghal, Walid
  • SIAM Journal on Optimization, Vol. 19, Issue 1
  • DOI: 10.1137/060677513

A Merit Function Approach for Direct Search
journal, January 2014

  • Gratton, S.; Vicente, L. N.
  • SIAM Journal on Optimization, Vol. 24, Issue 4
  • DOI: 10.1137/130917661

A View of Unconstrained Minimization Algorithms that Do Not Require Derivatives
journal, June 1975


Simulation-Based Optimization with Stochastic Approximation Using Common Random Numbers
journal, November 1999

  • Kleinman, Nathan L.; Spall, James C.; Naiman, Daniel Q.
  • Management Science, Vol. 45, Issue 11
  • DOI: 10.1287/mnsc.45.11.1570

Self-Correcting Geometry in Model-Based Algorithms for Derivative-Free Unconstrained Optimization
journal, January 2010

  • Scheinberg, K.; Toint, Ph. L.
  • SIAM Journal on Optimization, Vol. 20, Issue 6
  • DOI: 10.1137/090748536

A Derivative-Free Algorithm for Linearly Constrained Finite Minimax Problems
journal, January 2006

  • Liuzzi, G.; Lucidi, S.; Sciandrone, M.
  • SIAM Journal on Optimization, Vol. 16, Issue 4
  • DOI: 10.1137/040615821

Robust optimization of noisy blackbox problems using the Mesh Adaptive Direct Search algorithm
journal, January 2018

  • Audet, Charles; Ihaddadene, Amina; Le Digabel, Sébastien
  • Optimization Letters, Vol. 12, Issue 4
  • DOI: 10.1007/s11590-017-1226-6

Some aspects of the sequential design of experiments
journal, January 1952


On the local convergence of a derivative-free algorithm for least-squares minimization
journal, October 2010

  • Zhang, Hongchao; Conn, Andrew R.
  • Computational Optimization and Applications, Vol. 51, Issue 2
  • DOI: 10.1007/s10589-010-9367-x

A trust-region-based derivative free algorithm for mixed integer programming
journal, April 2014


Random Gradient-Free Minimization of Convex Functions
journal, November 2015

  • Nesterov, Yurii; Spokoiny, Vladimir
  • Foundations of Computational Mathematics, Vol. 17, Issue 2
  • DOI: 10.1007/s10208-015-9296-2

A grid algorithm for bound constrained optimization of noisy functions
journal, January 1995

  • Elster, Clemens; Neumaier, Arnold
  • IMA Journal of Numerical Analysis, Vol. 15, Issue 4
  • DOI: 10.1093/imanum/15.4.585

Efficient calculation of regular simplex gradients
journal, January 2019


Superlinear Convergence and Implicit Filtering
journal, January 2000


Estimating Computational Noise
journal, January 2011

  • Moré, Jorge J.; Wild, Stefan M.
  • SIAM Journal on Scientific Computing, Vol. 33, Issue 3
  • DOI: 10.1137/100786125

A projected derivative-free algorithm for nonlinear equations with convex constraints
journal, July 2012


Sequential Search: A Method for Solving Constrained Optimization Problems
journal, January 1965


Asymptotically efficient adaptive allocation rules
journal, March 1985


Worst case complexity of direct search under convexity
journal, November 2014


Optimization with hidden constraints and embedded Monte Carlo computations
journal, December 2015


Multidimensional Stochastic Approximation Methods
journal, December 1954


A Nonderivative Version of the Gradient Sampling Algorithm for Nonsmooth Nonconvex Optimization
journal, January 2010

  • Kiwiel, Krzysztof C.
  • SIAM Journal on Optimization, Vol. 20, Issue 4
  • DOI: 10.1137/090748408

Pattern Search Algorithms for Bound Constrained Minimization
journal, January 1999


Kernel-based methods for Bandit convex optimization
conference, January 2017

  • Bubeck, Sébastien; Lee, Yin Tat; Eldan, Ronen
  • Proceedings of the 49th Annual ACM SIGACT Symposium on Theory of Computing - STOC 2017
  • DOI: 10.1145/3055399.3055403

Direct search algorithm for bilevel programming problems
journal, September 2009

  • Mersha, Ayalew Getachew; Dempe, Stephan
  • Computational Optimization and Applications, Vol. 49, Issue 1
  • DOI: 10.1007/s10589-009-9295-9

A method for the solution of certain non-linear problems in least squares
journal, January 1944

  • Levenberg, Kenneth
  • Quarterly of Applied Mathematics, Vol. 2, Issue 2
  • DOI: 10.1090/qam/10666

Spectral residual method without gradient information for solving large-scale nonlinear systems of equations
journal, April 2006


The solution of variational and boundary value problems by the method of local variations
journal, January 1966

  • Banichuk, N. V.; Petrov, V. M.; Chernous'ko, F. L.
  • USSR Computational Mathematics and Mathematical Physics, Vol. 6, Issue 6
  • DOI: 10.1016/0041-5553(66)90158-3

Algorithm 856: APPSPACK 4.0: asynchronous parallel pattern search for derivative-free optimization
journal, September 2006

  • Gray, Genetha A.; Kolda, Tamara G.
  • ACM Transactions on Mathematical Software, Vol. 32, Issue 3
  • DOI: 10.1145/1163641.1163647

Compositions of convex functions and fully linear models
journal, February 2017


Global Convergence of Radial Basis Function Trust Region Derivative-Free Algorithms
journal, July 2011

  • Wild, Stefan M.; Shoemaker, Christine
  • SIAM Journal on Optimization, Vol. 21, Issue 3
  • DOI: 10.1137/09074927X

Sample mean based index policies by O (log n ) regret for the multi-armed bandit problem
journal, December 1995

  • Agrawal, Rajeev
  • Advances in Applied Probability, Vol. 27, Issue 4
  • DOI: 10.2307/1427934

Variable-Number Sample-Path Optimization
journal, July 2007


A Derivative-Free Trust-Region Method for Biobjective Optimization
journal, January 2014

  • Ryu, Jong-hyun; Kim, Sujin
  • SIAM Journal on Optimization, Vol. 24, Issue 1
  • DOI: 10.1137/120864738

Conditions for convergence of trust region algorithms for nonsmooth optimization
journal, June 1985


Convergence Results for Generalized Pattern Search Algorithms are Tight
journal, June 2004


Constrained derivative-free optimization on thin domains
journal, July 2012


Direct search based on probabilistic feasible descent for bound and linearly constrained problems
journal, January 2019

  • Gratton, S.; Royer, C. W.; Vicente, L. N.
  • Computational Optimization and Applications, Vol. 72, Issue 3
  • DOI: 10.1007/s10589-019-00062-4

Stochastic online optimization. Single-point and multi-point non-linear multi-armed bandits. Convex and strongly-convex case
journal, February 2017

  • Gasnikov, A. V.; Krymova, E. A.; Lagunovskaya, A. A.
  • Automation and Remote Control, Vol. 78, Issue 2
  • DOI: 10.1134/S0005117917020035

A Derivative-Free Algorithm for Inequality Constrained Nonlinear Programming via Smoothing of an $\ell_\infty$ Penalty Function
journal, January 2009

  • Liuzzi, G.; Lucidi, S.
  • SIAM Journal on Optimization, Vol. 20, Issue 1
  • DOI: 10.1137/070711451

Design and implementation of a massively parallel version of DIRECT
journal, October 2007

  • He, Jian; Verstak, Alex; Watson, Layne T.
  • Computational Optimization and Applications, Vol. 40, Issue 2
  • DOI: 10.1007/s10589-007-9092-2

Approximation Methods which Converge with Probability one
journal, June 1954


Derivative-Free Methods for Mixed-Integer Constrained Optimization Problems
journal, July 2014

  • Liuzzi, Giampaolo; Lucidi, Stefano; Rinaldi, Francesco
  • Journal of Optimization Theory and Applications, Vol. 164, Issue 3
  • DOI: 10.1007/s10957-014-0617-4

MISO: mixed-integer surrogate optimization framework
journal, June 2015


Asynchronous Parallel Pattern Search for Nonlinear Optimization
journal, January 2001

  • Hough, Patricia D.; Kolda, Tamara G.; Torczon, Virginia J.
  • SIAM Journal on Scientific Computing, Vol. 23, Issue 1
  • DOI: 10.1137/S1064827599365823

On the convergence of a wide range of trust region methods for unconstrained optimization
journal, September 2009


Parallel Space Decomposition of the Mesh Adaptive Direct Search Algorithm
journal, January 2008

  • Audet, Charles; Dennis, J. E.; Digabel, Sébastien Le
  • SIAM Journal on Optimization, Vol. 19, Issue 3
  • DOI: 10.1137/070707518

Incorporating minimum Frobenius norm models in direct search
journal, August 2009

  • Custódio, A. L.; Rocha, H.; Vicente, L. N.
  • Computational Optimization and Applications, Vol. 46, Issue 2
  • DOI: 10.1007/s10589-009-9283-0

A batch, derivative-free algorithm for finding multiple local minima
journal, October 2015


Generalized pattern searches with derivative information
journal, November 2003

  • Abramson, Mark A.; Audet, Charles; Dennis, J. E.
  • Mathematical Programming, Vol. 100, Issue 1
  • DOI: 10.1007/s10107-003-0484-5

A derivative-free trust-region algorithm for composite nonsmooth optimization
journal, December 2014

  • Grapiglia, Geovani Nunes; Yuan, Jinyun; Yuan, Ya-xiang
  • Computational and Applied Mathematics, Vol. 35, Issue 2
  • DOI: 10.1007/s40314-014-0201-4

Pattern search ranking and selection algorithms for mixed variable simulation-based optimization
journal, November 2009

  • Sriver, Todd A.; Chrissis, James W.; Abramson, Mark A.
  • European Journal of Operational Research, Vol. 198, Issue 3
  • DOI: 10.1016/j.ejor.2008.10.020

Derivative-free optimization: a review of algorithms and comparison of software implementations
journal, July 2012


Mesh Adaptive Direct Search Algorithms for Constrained Optimization
journal, January 2006

  • Audet, Charles; Dennis, J. E.
  • SIAM Journal on Optimization, Vol. 17, Issue 1
  • DOI: 10.1137/040603371

A trust-region framework for managing the use of approximation models in optimization
journal, February 1998

  • Alexandrov, N. M.; Dennis, J. E.; Lewis, R. M.
  • Structural Optimization, Vol. 15, Issue 1
  • DOI: 10.1007/BF01197433

On the Oracle Complexity of First-Order and Derivative-Free Algorithms for Smooth Nonconvex Minimization
journal, January 2012

  • Cartis, Coralia; Gould, Nicholas I. M.; Toint, Philippe L.
  • SIAM Journal on Optimization, Vol. 22, Issue 1
  • DOI: 10.1137/100812276

A stochastic method for global optimization
journal, December 1982

  • Boender, C. G. E.; Rinnooy Kan, A. H. G.; Timmer, G. T.
  • Mathematical Programming, Vol. 22, Issue 1
  • DOI: 10.1007/BF01581033

A class of derivative-free methods for large-scale nonlinear monotone equations
journal, January 2011


A Convergent Variant of the Nelder–Mead Algorithm
journal, April 2002

  • Price, C. J.; Coope, I. D.; Byatt, D.
  • Journal of Optimization Theory and Applications, Vol. 113, Issue 1
  • DOI: 10.1023/A:1014849028575

VXQR: derivative-free unconstrained optimization based on QR factorizations
journal, September 2010


Geometry of interpolation sets in derivative free optimization
journal, December 2006

  • Conn, A. R.; Scheinberg, K.; Vicente, Luís N.
  • Mathematical Programming, Vol. 111, Issue 1-2
  • DOI: 10.1007/s10107-006-0073-5

On the Number of Iterations of Piyavskii's Global Optimization Algorithm
journal, May 1991

  • Hansen, P.; Jaumard, B.; Lu, S. -H.
  • Mathematics of Operations Research, Vol. 16, Issue 2
  • DOI: 10.1287/moor.16.2.334

A derivative-free line search and global convergence of Broyden-like method for nonlinear equations
journal, January 2000


Beyond symmetric Broyden for updating quadratic models in minimization without derivatives
journal, January 2012


Performance Modeling and Analysis of a Massively Parallel Direct—Part 1
journal, February 2009

  • Jian He, ; Verstak, Alex; Watson, L. T.
  • The International Journal of High Performance Computing Applications, Vol. 23, Issue 1
  • DOI: 10.1177/1094342008098462

A subclass of generating set search with convergence to second-order stationary points
journal, June 2013

  • Abramson, Mark A.; Frimannslund, Lennart; Steihaug, Trond
  • Optimization Methods and Software, Vol. 29, Issue 5
  • DOI: 10.1080/10556788.2013.808198

Performance Modeling and Analysis of a Massively Parallel Direct—Part 2
journal, February 2009

  • Jian He, ; Verstak, Alex; Sosonkina, M.
  • The International Journal of High Performance Computing Applications, Vol. 23, Issue 1
  • DOI: 10.1177/1094342008098463

Direct search methods: then and now
journal, December 2000

  • Lewis, Robert Michael; Torczon, Virginia; Trosset, Michael W.
  • Journal of Computational and Applied Mathematics, Vol. 124, Issue 1-2
  • DOI: 10.1016/S0377-0427(00)00423-4

OrthoMADS: A Deterministic MADS Instance with Orthogonal Directions
journal, January 2009

  • Abramson, Mark A.; Audet, Charles; Dennis,, J. E.
  • SIAM Journal on Optimization, Vol. 20, Issue 2
  • DOI: 10.1137/080716980

Constrained optimization involving expensive function evaluations: A sequential approach
journal, January 2005

  • Brekelmans, Ruud; Driessen, Lonneke; Hamers, Herbert
  • European Journal of Operational Research, Vol. 160, Issue 1
  • DOI: 10.1016/j.ejor.2003.10.009

Boosters: A Derivative-Free Algorithm Based on Radial Basis Functions
journal, January 2009


Two-Point Step Size Gradient Methods
journal, January 1988

  • Barzilai, Jonathan; Borwein, Jonathan M.
  • IMA Journal of Numerical Analysis, Vol. 8, Issue 1
  • DOI: 10.1093/imanum/8.1.141

Stochastic Nelder–Mead simplex method – A new globally convergent direct search method for simulation optimization
journal, August 2012


Stationarity Results for Generating Set Search for Linearly Constrained Optimization
journal, January 2007

  • Kolda, Tamara G.; Lewis, Robert Michael; Torczon, Virginia
  • SIAM Journal on Optimization, Vol. 17, Issue 4
  • DOI: 10.1137/S1052623403433638

An Implicit Filtering Algorithm for Optimization of Functions with Many Local Minima
journal, May 1995

  • Gilmore, P.; Kelley, C. T.
  • SIAM Journal on Optimization, Vol. 5, Issue 2
  • DOI: 10.1137/0805015

Optimization by Direct Search: New Perspectives on Some Classical and Modern Methods
journal, January 2003

  • Kolda, Tamara G.; Lewis, Robert Michael; Torczon, Virginia
  • SIAM Review, Vol. 45, Issue 3
  • DOI: 10.1137/S003614450242889

Inexact Restoration Method for Derivative-Free Optimization with Smooth Constraints
journal, January 2013

  • Bueno, L. F.; Friedlander, A.; Martínez, J. M.
  • SIAM Journal on Optimization, Vol. 23, Issue 2
  • DOI: 10.1137/110856253

A Derivative-Free Approach to Constrained Multiobjective Nonsmooth Optimization
journal, January 2016

  • Liuzzi, G.; Lucidi, S.; Rinaldi, F.
  • SIAM Journal on Optimization, Vol. 26, Issue 4
  • DOI: 10.1137/15M1037810

Global convergence and stabilization of unconstrained minimization methods without derivatives
journal, March 1988

  • Grippo, L.; Lampariello, F.; Lucidi, S.
  • Journal of Optimization Theory and Applications, Vol. 56, Issue 3
  • DOI: 10.1007/BF00939550

A derivative-free comirror algorithm for convex optimization
journal, October 2014

  • Bauschke, Heinz H.; Hare, Warren L.; Moursi, Walaa M.
  • Optimization Methods and Software, Vol. 30, Issue 4
  • DOI: 10.1080/10556788.2014.968158

An Algorithm for Least-Squares Estimation of Nonlinear Parameters
journal, June 1963

  • Marquardt, Donald W.
  • Journal of the Society for Industrial and Applied Mathematics, Vol. 11, Issue 2
  • DOI: 10.1137/0111030

Trust-Region Methods Without Using Derivatives: Worst Case Complexity and the NonSmooth Case
journal, January 2016

  • Garmanjani, R.; Júdice, D.; Vicente, L. N.
  • SIAM Journal on Optimization, Vol. 26, Issue 4
  • DOI: 10.1137/151005683

A derivative-free algorithm for linearly constrained optimization problems
journal, October 2013

  • Gumma, E. A. E.; Hashim, M. H. A.; Ali, M. Montaz
  • Computational Optimization and Applications, Vol. 57, Issue 3
  • DOI: 10.1007/s10589-013-9607-y

Computing a Trust Region Step
journal, September 1983

  • Moré, Jorge J.; Sorensen, D. C.
  • SIAM Journal on Scientific and Statistical Computing, Vol. 4, Issue 3
  • DOI: 10.1137/0904038

MILP models for the selection of a small set of well-distributed points
journal, January 2017

  • D'Ambrosio, Claudia; Nannicini, Giacomo; Sartor, Giorgio
  • Operations Research Letters, Vol. 45, Issue 1
  • DOI: 10.1016/j.orl.2016.11.004

Lipschitzian optimization without the Lipschitz constant
journal, October 1993

  • Jones, D. R.; Perttunen, C. D.; Stuckman, B. E.
  • Journal of Optimization Theory and Applications, Vol. 79, Issue 1
  • DOI: 10.1007/BF00941892

`` Direct Search'' Solution of Numerical and Statistical Problems
journal, April 1961


A Globally Convergent Augmented Lagrangian Algorithm for Optimization with General Constraints and Simple Bounds
journal, April 1991

  • Conn, Andrew R.; Gould, Nicholas I. M.; Toint, Philippe
  • SIAM Journal on Numerical Analysis, Vol. 28, Issue 2
  • DOI: 10.1137/0728030

A Trust-Region Algorithm for Bi-Objective Stochastic Optimization
journal, January 2011


Optimal Rates for Zero-Order Convex Optimization: The Power of Two Function Evaluations
journal, May 2015

  • Duchi, John C.; Jordan, Michael I.; Wainwright, Martin J.
  • IEEE Transactions on Information Theory, Vol. 61, Issue 5
  • DOI: 10.1109/TIT.2015.2409256

On the convergence of trust region algorithms for unconstrained minimization without derivatives
journal, April 2012


Pattern search methods for finite minimax problems
journal, March 2009


Worst case complexity of direct search
journal, December 2012


Derivative-free optimization methods for finite minimax problems
journal, April 2013


On sequential and parallel non-monotone derivative-free algorithms for box constrained optimization
journal, June 2012

  • García-Palomares, Ubaldo M.; García-Urrea, Ildemaro J.; Rodríguez-Hernández, Pedro S.
  • Optimization Methods and Software, Vol. 28, Issue 6
  • DOI: 10.1080/10556788.2012.693926

Global convergence of a derivative-free inexact restoration filter algorithm for nonlinear programming
journal, November 2016


Optimizing partially separable functions without derivatives
journal, August 2005


Convergence of Trust-Region Methods Based on Probabilistic Models
journal, January 2014

  • Bandeira, A. S.; Scheinberg, K.; Vicente, L. N.
  • SIAM Journal on Optimization, Vol. 24, Issue 3
  • DOI: 10.1137/130915984

CONDOR, a new parallel, constrained extension of Powell's UOBYQA algorithm: Experimental results and comparison with the DFO algorithm
journal, September 2005

  • Vanden Berghen, Frank; Bersini, Hugues
  • Journal of Computational and Applied Mathematics, Vol. 181, Issue 1
  • DOI: 10.1016/j.cam.2004.11.029

Asynchronously parallel optimization solver for finding multiple minima
journal, February 2018


Provably Convergent Multifidelity Optimization Algorithm Not Requiring High-Fidelity Derivatives
journal, May 2012

  • March, Andrew; Willcox, Karen
  • AIAA Journal, Vol. 50, Issue 5
  • DOI: 10.2514/1.J051125

On the convergence of the UOBYQA method
journal, March 2004

  • Han, Lixing; Liu, Guanghui
  • Journal of Applied Mathematics and Computing, Vol. 16, Issue 1-2
  • DOI: 10.1007/BF02936156

Additive Scaling and the DIRECT Algorithm
journal, June 2006


Bayesian Guided Pattern Search for Robust Local Optimization
journal, November 2009

  • Taddy, Matthew A.; Lee, Herbert K. H.; Gray, Genetha A.
  • Technometrics, Vol. 51, Issue 4
  • DOI: 10.1198/TECH.2009.08007

Dud, A Derivative-Free Algorithm for Nonlinear Least Squares
journal, February 1978


An implicit filtering algorithm for derivative-free multiobjective optimization with box constraints
journal, October 2017

  • Cocchi, G.; Liuzzi, G.; Papini, A.
  • Computational Optimization and Applications, Vol. 69, Issue 2
  • DOI: 10.1007/s10589-017-9953-2

Sequential Application of Simplex Designs in Optimisation and Evolutionary Operation
journal, November 1962


Stochastic derivative-free optimization using a trust region framework
journal, February 2016

  • Larson, Jeffrey; Billups, Stephen C.
  • Computational Optimization and Applications, Vol. 64, Issue 3
  • DOI: 10.1007/s10589-016-9827-z

On the optimal order of worst case complexity of direct search
journal, June 2015


Stopping criteria for linesearch methods without derivatives
journal, October 1984

  • De Leone, Renato; Gaudioso, Manlio; Grippo, Luigi
  • Mathematical Programming, Vol. 30, Issue 3
  • DOI: 10.1007/BF02591934

Derivative-Free Optimization of Expensive Functions with Computational Error Using Weighted Regression
journal, January 2013

  • Billups, Stephen C.; Larson, Jeffrey; Graf, Peter
  • SIAM Journal on Optimization, Vol. 23, Issue 1
  • DOI: 10.1137/100814688

Using Sampling and Simplex Derivatives in Pattern Search Methods
journal, January 2007

  • Custódio, A. L.; Vicente, L. N.
  • SIAM Journal on Optimization, Vol. 18, Issue 2
  • DOI: 10.1137/050646706

Solving Nonlinear Programs Without Using Analytic Derivatives
journal, June 1979


A second-order globally convergent direct-search method and its worst-case complexity
journal, October 2015


The Nonstochastic Multiarmed Bandit Problem
journal, January 2002

  • Auer, Peter; Cesa-Bianchi, Nicolò; Freund, Yoav
  • SIAM Journal on Computing, Vol. 32, Issue 1
  • DOI: 10.1137/S0097539701398375

Robust optimization with simulated annealing
journal, December 2009


A superlinearly convergent algorithm for minimization without evaluating derivatives
journal, December 1975


SNOBFIT -- Stable Noisy Optimization by Branch and Fit
journal, July 2008

  • Huyer, Waltraud; Neumaier, Arnold
  • ACM Transactions on Mathematical Software, Vol. 35, Issue 2
  • DOI: 10.1145/1377612.1377613

Pure adaptive search in global optimization
journal, January 1992

  • Zabinsky, Zelda B.; Smith, Robert L.
  • Mathematical Programming, Vol. 53, Issue 1-3
  • DOI: 10.1007/BF01585710

Fortified-Descent Simplicial Search Method: A General Approach
journal, January 1999


Do you trust derivatives or differences?
journal, September 2014


A derivative-free approximate gradient sampling algorithm for finite minimax problems
journal, March 2013


Evaluating Derivatives
book, January 2008


A sequential quadratic programming algorithm for equality-constrained optimization without derivatives
journal, November 2014


Stochastic optimization using a trust-region method and random models
journal, April 2017


Generalized Pattern Search methods for a class of nonsmooth optimization problems with structure
journal, July 2009

  • Bogani, C.; Gasparo, M. G.; Papini, A.
  • Journal of Computational and Applied Mathematics, Vol. 229, Issue 1
  • DOI: 10.1016/j.cam.2008.10.047

A Linesearch-Based Derivative-Free Approach for Nonsmooth Constrained Optimization
journal, January 2014

  • Fasano, G.; Liuzzi, G.; Lucidi, S.
  • SIAM Journal on Optimization, Vol. 24, Issue 3
  • DOI: 10.1137/130940037

GLODS: Global and Local Optimization using Direct Search
journal, August 2014


Multivariate stochastic approximation using a simultaneous perturbation gradient approximation
journal, March 1992

  • Spall, J. C.
  • IEEE Transactions on Automatic Control, Vol. 37, Issue 3
  • DOI: 10.1109/9.119632

Second-Order Behavior of Pattern Search
journal, January 2005

  • Abramson, Mark A.
  • SIAM Journal on Optimization, Vol. 16, Issue 2
  • DOI: 10.1137/04060367X

The calculus of simplex gradients
journal, October 2014


New horizons in sphere-packing theory, part II: lattice-based derivative-free optimization via global surrogates
journal, March 2012


Grid Restrained Nelder-Mead Algorithm
journal, March 2006

  • Bűrmen, Árpád; Puhan, Janez; Tuma, Tadej
  • Computational Optimization and Applications, Vol. 34, Issue 3
  • DOI: 10.1007/s10589-005-3912-z

Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems
journal, January 2012

  • Bubeck, Sébastien
  • Foundations and Trends® in Machine Learning, Vol. 5, Issue 1
  • DOI: 10.1561/2200000024

Bilevel derivative-free optimization and its application to robust optimization
journal, June 2012


Computing Forward-Difference Intervals for Numerical Optimization
journal, June 1983

  • Gill, Philip E.; Murray, Walter; Saunders, Michael A.
  • SIAM Journal on Scientific and Statistical Computing, Vol. 4, Issue 2
  • DOI: 10.1137/0904025

On Lipschitz optimization based on gray-box piecewise linearization
journal, July 2015

  • Griewank, Andreas; Walther, Andrea; Fiege, Sabrina
  • Mathematical Programming, Vol. 158, Issue 1-2
  • DOI: 10.1007/s10107-015-0934-x

A mesh adaptive direct search algorithm for multiobjective optimization
journal, August 2010

  • Audet, Charles; Savard, Gilles; Zghal, Walid
  • European Journal of Operational Research, Vol. 204, Issue 3
  • DOI: 10.1016/j.ejor.2009.11.010

Direct Multisearch for Multiobjective Optimization
journal, July 2011

  • Custódio, A. L.; Madeira, J. F. A.; Vaz, A. I. F.
  • SIAM Journal on Optimization, Vol. 21, Issue 3
  • DOI: 10.1137/10079731X

Optimization of Convex Functions with Random Pursuit
journal, January 2013

  • Stich, S. U.; Müller, C. L.; Gärtner, B.
  • SIAM Journal on Optimization, Vol. 23, Issue 2
  • DOI: 10.1137/110853613

Exploiting Known Structures to Approximate Normal Cones
journal, November 2013


On fast trust region methods for quadratic models with linear constraints
journal, June 2015


BFO, A Trainable Derivative-free Brute Force Optimizer for Nonlinear Bound-constrained Optimization and Equilibrium Computations with Continuous and Discrete Variables
journal, June 2017

  • Porcelli, Margherita; Toint, Philippe L.
  • ACM Transactions on Mathematical Software, Vol. 44, Issue 1
  • DOI: 10.1145/3085592

Piecewise Partially Separable Functions and a Derivative-free Algorithm for Large Scale Nonsmooth Optimization
journal, June 2006


Global Convergence of General Derivative-Free Trust-Region Algorithms to First- and Second-Order Critical Points
journal, January 2009

  • Conn, Andrew R.; Scheinberg, Katya; Vicente, Luís N.
  • SIAM Journal on Optimization, Vol. 20, Issue 1
  • DOI: 10.1137/060673424

A Derivative-Free Algorithm for Least-Squares Minimization
journal, January 2010

  • Zhang, Hongchao; Conn, Andrew R.; Scheinberg, Katya
  • SIAM Journal on Optimization, Vol. 20, Issue 6
  • DOI: 10.1137/09075531X

An adaptive multiquadric radial basis function method for expensive black-box mixed-integer nonlinear constrained optimization
journal, February 2013


A derivative-free nonmonotone line-search technique for unconstrained optimization
journal, October 2008

  • Diniz-Ehrhardt, M. A.; Martínez, J. M.; Raydan, M.
  • Journal of Computational and Applied Mathematics, Vol. 219, Issue 2
  • DOI: 10.1016/j.cam.2007.07.017

A method for stochastic constrained optimization using derivative-free surrogate pattern search and collocation
journal, June 2010

  • Sankaran, Sethuraman; Audet, Charles; Marsden, Alison L.
  • Journal of Computational Physics, Vol. 229, Issue 12
  • DOI: 10.1016/j.jcp.2010.03.005

New Sequential and Parallel Derivative-Free Algorithms for Unconstrained Minimization
journal, January 2002


On the Convergence of the Multidirectional Search Algorithm
journal, February 1991

  • Torczon, Virginia
  • SIAM Journal on Optimization, Vol. 1, Issue 1
  • DOI: 10.1137/0801010

Derivative free analogues of the Levenberg-Marquardt and Gauss algorithms for nonlinear least squares approximation
journal, August 1971

  • Brown, Kenneth M.; Dennis, J. E.
  • Numerische Mathematik, Vol. 18, Issue 4
  • DOI: 10.1007/BF01404679

On Sampling Rates in Simulation-Based Recursions
journal, January 2018

  • Pasupathy, Raghu; Glynn, Peter; Ghosh, Soumyadip
  • SIAM Journal on Optimization, Vol. 28, Issue 1
  • DOI: 10.1137/140951679

ORBIT: Optimization by Radial Basis Function Interpolation in Trust-Regions
journal, January 2008

  • Wild, Stefan M.; Regis, Rommel G.; Shoemaker, Christine A.
  • SIAM Journal on Scientific Computing, Vol. 30, Issue 6
  • DOI: 10.1137/070691814

A Stochastic Approximation Method
journal, September 1951

  • Robbins, Herbert; Monro, Sutton
  • The Annals of Mathematical Statistics, Vol. 22, Issue 3
  • DOI: 10.1214/aoms/1177729586

UOBYQA: unconstrained optimization by quadratic approximation
journal, May 2002


Estimating Derivatives of Noisy Simulations
journal, April 2012

  • Moré, Jorge J.; Wild, Stefan M.
  • ACM Transactions on Mathematical Software, Vol. 38, Issue 3
  • DOI: 10.1145/2168773.2168777

Chapter 40: POUNDERS in TAO: Solving Derivative-Free Nonlinear Least-Squares Problems with POUNDERS
book, April 2017

  • Wild, Stefan M.; Terlaky, Tamás; Anjos, Miguel F.
  • Advances and Trends in Optimization with Engineering Applications
  • DOI: 10.1137/1.9781611974683.ch40

On the construction of quadratic models for derivative-free trust-region algorithms
journal, February 2017

  • Verdério, Adriano; Karas, Elizabeth W.; Pedroso, Lucas G.
  • EURO Journal on Computational Optimization, Vol. 5, Issue 4
  • DOI: 10.1007/s13675-017-0081-7

Approximate norm descent methods for constrained nonlinear systems
journal, May 2017

  • Morini, Benedetta; Porcelli, Margherita; Toint, Philippe L.
  • Mathematics of Computation, Vol. 87, Issue 311
  • DOI: 10.1090/mcom/3251

SO-MI: A surrogate model algorithm for computationally expensive nonlinear mixed-integer black-box global optimization problems
journal, May 2013

  • Müller, Juliane; Shoemaker, Christine A.; Piché, Robert
  • Computers & Operations Research, Vol. 40, Issue 5
  • DOI: 10.1016/j.cor.2012.08.022

An inexact restoration derivative-free filter method for nonlinear programming
journal, July 2015

  • Echebest, N.; Schuverdt, M. L.; Vignau, R. P.
  • Computational and Applied Mathematics, Vol. 36, Issue 1
  • DOI: 10.1007/s40314-015-0253-0

Taking the Human Out of the Loop: A Review of Bayesian Optimization
journal, January 2016


Precision Control for Generalized Pattern Search Algorithms with Adaptive Precision Function Evaluations
journal, January 2006

  • Polak, Elijah; Wetter, Michael
  • SIAM Journal on Optimization, Vol. 16, Issue 3
  • DOI: 10.1137/040605527

Analysis of direct searches for discontinuous functions
journal, December 2010


Algorithm 909: NOMAD: Nonlinear Optimization with the MADS Algorithm
journal, February 2011


On the use of quadratic models in unconstrained minimization without derivatives
journal, June 2004


Linear equalities in blackbox optimization
journal, October 2014

  • Audet, Charles; Le Digabel, Sébastien; Peyrega, Mathilde
  • Computational Optimization and Applications, Vol. 61, Issue 1
  • DOI: 10.1007/s10589-014-9708-2

Efficient Global Optimization of Expensive Black-Box Functions
journal, January 1998

  • Jones, Donald R.; Schonlau, Matthias; Welch, William J.
  • Journal of Global Optimization, Vol. 13, Issue 4, p. 455-492
  • DOI: 10.1023/A:1008306431147

Stochastic First- and Zeroth-Order Methods for Nonconvex Stochastic Programming
journal, January 2013

  • Ghadimi, Saeed; Lan, Guanghui
  • SIAM Journal on Optimization, Vol. 23, Issue 4
  • DOI: 10.1137/120880811

A trust region-based two phase algorithm for constrained black-box and grey-box optimization with infeasible initial point
journal, August 2018


Direct Search Methods on Parallel Machines
journal, November 1991

  • Dennis, Jr., J. E.; Torczon, Virginia
  • SIAM Journal on Optimization, Vol. 1, Issue 4
  • DOI: 10.1137/0801027

Coordinate search algorithms in multilevel optimization
journal, November 2013


Derivative-free methods for bound constrained mixed-integer optimization
journal, April 2011

  • Liuzzi, G.; Lucidi, S.; Rinaldi, F.
  • Computational Optimization and Applications, Vol. 53, Issue 2
  • DOI: 10.1007/s10589-011-9405-3

Surrogate-based methods for black-box optimization: Surrogate-based methods for black-box optimization
journal, April 2016

  • Vu, Ky Khac; D'Ambrosio, Claudia; Hamadi, Youssef
  • International Transactions in Operational Research, Vol. 24, Issue 3
  • DOI: 10.1111/itor.12292

A Stochastic Radial Basis Function Method for the Global Optimization of Expensive Functions
journal, November 2007

  • Regis, Rommel G.; Shoemaker, Christine A.
  • INFORMS Journal on Computing, Vol. 19, Issue 4
  • DOI: 10.1287/ijoc.1060.0182

An Application of Chung's Lemma to the Kiefer-Wolfowitz Stochastic Approximation Procedure
journal, June 1956


Optimal Condenser Design by Geometric Programming
journal, April 1967

  • Avriel, Mordecai; Wilde, D. J.
  • Industrial & Engineering Chemistry Process Design and Development, Vol. 6, Issue 2
  • DOI: 10.1021/i260022a018

Nelder-Mead Simplex Modifications for Simulation Optimization
journal, July 1996


CONORBIT: constrained optimization by radial basis function interpolation in trust regions
journal, August 2016


A multigrid approach to discretized optimization problems
journal, January 2000


Frame Based Methods for Unconstrained Optimization
journal, November 2000

  • Coope, I. D.; Price, C. J.
  • Journal of Optimization Theory and Applications, Vol. 107, Issue 2
  • DOI: 10.1023/A:1026429319405

Mesh adaptive direct search with second directional derivative-based Hessian update
journal, April 2015

  • Bűrmen, Árpád; Olenšek, Jernej; Tuma, Tadej
  • Computational Optimization and Applications, Vol. 62, Issue 3
  • DOI: 10.1007/s10589-015-9753-5

On the properties of positive spanning sets and positive bases
journal, September 2015


A Derivative-Free Trust-Region Algorithm for the Optimization of Functions Smoothed via Gaussian Convolution Using Adaptive Multiple Importance Sampling
journal, January 2018

  • Maggiar, Alvaro; Wächter, Andreas; Dolinskaya, Irina S.
  • SIAM Journal on Optimization, Vol. 28, Issue 2
  • DOI: 10.1137/15M1031679

Positive bases for linear spaces
journal, January 1962


Sobolev seminorm of quadratic functions with applications to derivative-free optimization
journal, May 2013


Use of quadratic models with mesh-adaptive direct search for constrained black box optimization
journal, February 2013


A Globally Convergent Filter Method for Nonlinear Programming
journal, January 2004

  • Gonzaga, Clóvis C.; Karas, Elizabeth; Vanti, Márcia
  • SIAM Journal on Optimization, Vol. 14, Issue 3
  • DOI: 10.1137/S1052623401399320

Benchmarking Derivative-Free Optimization Algorithms
journal, January 2009

  • Moré, Jorge J.; Wild, Stefan M.
  • SIAM Journal on Optimization, Vol. 20, Issue 1
  • DOI: 10.1137/080724083

Stochastic Estimation of the Maximum of a Regression Function
journal, September 1952


Wedge trust region methods for derivative free optimization
journal, January 2002


Convergence of the Restricted Nelder--Mead Algorithm in Two Dimensions
journal, January 2012

  • Lagarias, Jeffrey C.; Poonen, Bjorn; Wright, Margaret H.
  • SIAM Journal on Optimization, Vol. 22, Issue 2
  • DOI: 10.1137/110830150

Using QR Decomposition to Obtain a New Instance of Mesh Adaptive Direct Search with Uniformly Distributed Polling Directions
journal, June 2013

  • Van Dyke, Benjamin; Asaki, Thomas J.
  • Journal of Optimization Theory and Applications, Vol. 159, Issue 3
  • DOI: 10.1007/s10957-013-0356-y

Reducing the Number of Function Evaluations in Mesh Adaptive Direct Search Algorithms
journal, January 2014

  • Audet, Charles; Ianni, Andrea; Le Digabel, Sébastien
  • SIAM Journal on Optimization, Vol. 24, Issue 2
  • DOI: 10.1137/120895056

Objective-derivative-free methods for constrained optimization
journal, March 2002

  • Lucidi, S.; Sciandrone, M.; Tseng, P.
  • Mathematical Programming, Vol. 92, Issue 1
  • DOI: 10.1007/s101070100266

Introduction to Derivative-Free Optimization
book, January 2009


A generating set search method using curvature information
journal, May 2007

  • Frimannslund, Lennart; Steihaug, Trond
  • Computational Optimization and Applications, Vol. 38, Issue 1
  • DOI: 10.1007/s10589-007-9038-8

Derivative-Free Optimization Via Proximal Point Methods
journal, June 2013


Smoothing and worst-case complexity for direct-search methods in nonsmooth optimization
journal, October 2012

  • Garmanjani, R.; Vicente, L. N.
  • IMA Journal of Numerical Analysis, Vol. 33, Issue 3
  • DOI: 10.1093/imanum/drs027

Developments of NEWUOA for minimization without derivatives
journal, February 2008


A Progressive Barrier for Derivative-Free Nonlinear Programming
journal, January 2009

  • Audet, Charles; Dennis, J. E.
  • SIAM Journal on Optimization, Vol. 20, Issue 1
  • DOI: 10.1137/070692662

Rates of Convergence for Stochastic Approximation Type Algorithms
journal, September 1979

  • Kushner, Harold J.; Huang, Hai
  • SIAM Journal on Control and Optimization, Vol. 17, Issue 5
  • DOI: 10.1137/0317043

Estimating Tangent and Normal Cones Without Calculus
journal, November 2005


Pattern Search Algorithms for Mixed Variable Programming
journal, January 2001


Theory of Positive Linear Dependence
journal, October 1954

  • Davis, Chandler
  • American Journal of Mathematics, Vol. 76, Issue 4
  • DOI: 10.2307/2372648

Exploiting problem structure in pattern search methods for unconstrained optimization
journal, June 2006


A polynomial case of convex integer quadratic programming problems with box integer constraints
journal, January 2015


Pattern Search Methods for Linearly Constrained Minimization
journal, January 2000


Sequential Penalty Derivative-Free Methods for Nonlinear Constrained Optimization
journal, January 2010

  • Liuzzi, Giampaolo; Lucidi, Stefano; Sciandrone, Marco
  • SIAM Journal on Optimization, Vol. 20, Issue 5
  • DOI: 10.1137/090750639

A derivative-free trust-funnel method for equality-constrained nonlinear optimization
journal, November 2014

  • Sampaio, Ph. R.; Toint, Ph. L.
  • Computational Optimization and Applications, Vol. 61, Issue 1
  • DOI: 10.1007/s10589-014-9715-3

Complexity and global rates of trust-region methods based on probabilistic models
journal, August 2017

  • Gratton, Serge; Royer, Clément W.; Vicente, Luís N.
  • IMA Journal of Numerical Analysis, Vol. 38, Issue 3
  • DOI: 10.1093/imanum/drx043

Nonmonotone derivative-free methods for nonlinear equations
journal, March 2007


Numerical experience with a derivative-free trust-funnel method for nonlinear optimization problems with general nonlinear constraints
journal, February 2016


Computation of sparse low degree interpolating polynomials and their application to derivative-free optimization
journal, July 2012


Derivative-Free Robust Optimization for Circuit Design
journal, October 2013

  • Ciccazzo, Angelo; Latorre, Vittorio; Liuzzi, Giampaolo
  • Journal of Optimization Theory and Applications, Vol. 164, Issue 3
  • DOI: 10.1007/s10957-013-0441-2

Conditional gradient type methods for composite nonlinear and stochastic optimization
journal, January 2018


Simulation optimization: a review of algorithms and applications
journal, September 2015

  • Amaran, Satyajith; Sahinidis, Nikolaos V.; Sharda, Bikram
  • Annals of Operations Research, Vol. 240, Issue 1
  • DOI: 10.1007/s10479-015-2019-x

Asymptotic Distribution of Stochastic Approximation Procedures
journal, June 1958


Calibration by optimization without using derivatives
journal, July 2016


Using simplex gradients of nonsmooth functions in direct search methods
journal, February 2008

  • Custodio, A. L.; Dennis, J. E.; Vicente, L. N.
  • IMA Journal of Numerical Analysis, Vol. 28, Issue 4
  • DOI: 10.1093/imanum/drn045

GOSAC: global optimization with surrogate approximation of constraints
journal, January 2017


Geometry of sample sets in derivative-free optimization: polynomial regression and underdetermined interpolation
journal, February 2008

  • Conn, A. R.; Scheinberg, K.; Vicente, L. N.
  • IMA Journal of Numerical Analysis, Vol. 28, Issue 4
  • DOI: 10.1093/imanum/drn046

On the Lagrange functions of quadratic models that are defined by interpolation *
journal, January 2001


Convergence Properties of the Nelder--Mead Simplex Method in Low Dimensions
journal, January 1998

  • Lagarias, Jeffrey C.; Reeds, James A.; Wright, Margaret H.
  • SIAM Journal on Optimization, Vol. 9, Issue 1
  • DOI: 10.1137/S1052623496303470

Global convergence rate analysis of unconstrained optimization methods based on probabilistic models
journal, April 2017


Function Minimization by Interpolation in a Data Table
journal, January 1973


Budget-Dependent Convergence Rate of Stochastic Approximation
journal, February 1998


Bilevel direct search method for leader–follower problems and application in health insurance
journal, January 2014


On the Global Convergence of Derivative-Free Methods for Unconstrained Optimization
journal, January 2002


A class of derivative-free nonmonotone optimization algorithms employing coordinate rotations and gradient approximations
journal, June 2014


On the geometry phase in model-based algorithms for derivative-free optimization
journal, February 2009

  • Fasano, Giovanni; Morales, José Luis; Nocedal, Jorge
  • Optimization Methods and Software, Vol. 24, Issue 1
  • DOI: 10.1080/10556780802409296

A Globally Convergent Augmented Lagrangian Pattern Search Algorithm for Optimization with General Constraints and Simple Bounds
journal, January 2002


On trust region methods for unconstrained minimization without derivatives
journal, August 2003


Least Frobenius norm updating of quadratic models that satisfy interpolation conditions
journal, November 2003


Convergence of the Nelder--Mead Simplex Method to a Nonstationary Point
journal, January 1998


Robust Optimization for Unconstrained Simulation-Based Problems
journal, February 2010

  • Bertsimas, Dimitris; Nohadani, Omid; Teo, Kwong Meng
  • Operations Research, Vol. 58, Issue 1
  • DOI: 10.1287/opre.1090.0715

Pure adaptive search in monte carlo optimization
journal, January 1989

  • Patel, Nitin R.; Smith, Robert L.; Zabinsky, Zelda B.
  • Mathematical Programming, Vol. 43, Issue 1-3
  • DOI: 10.1007/BF01582296

Stochastic Trust-Region Response-Surface Method (STRONG)—A New Response-Surface Framework for Simulation Optimization
journal, May 2013

  • Chang, Kuo-Hao; Hong, L. Jeff; Wan, Hong
  • INFORMS Journal on Computing, Vol. 25, Issue 2
  • DOI: 10.1287/ijoc.1120.0498

Stochastic global optimization methods part I: Clustering methods
journal, September 1987

  • Rinnooy Kan, A. H. G.; Timmer, G. T.
  • Mathematical Programming, Vol. 39, Issue 1
  • DOI: 10.1007/BF02592070

Implementing the Nelder-Mead simplex algorithm with adaptive parameters
journal, May 2010


Stochastic global optimization methods part II: Multi level methods
journal, September 1987

  • Rinnooy Kan, A. H. G.; Timmer, G. T.
  • Mathematical Programming, Vol. 39, Issue 1
  • DOI: 10.1007/BF02592071

Efficient solution of quadratically constrained quadratic subproblems within the mesh adaptive direct search algorithm
journal, July 2018

  • Amaioua, Nadir; Audet, Charles; Conn, Andrew R.
  • European Journal of Operational Research, Vol. 268, Issue 1
  • DOI: 10.1016/j.ejor.2017.10.058

Nonlinear programming without a penalty function or a filter
journal, September 2008


Analysis of Generalized Pattern Searches
journal, January 2002


Convergence of Mesh Adaptive Direct Search to Second‐Order Stationary Points
journal, January 2006

  • Abramson, Mark A.; Audet, Charles
  • SIAM Journal on Optimization, Vol. 17, Issue 2
  • DOI: 10.1137/050638382

Direct Search Based on Probabilistic Descent
journal, January 2015

  • Gratton, S.; Royer, C. W.; Vicente, L. N.
  • SIAM Journal on Optimization, Vol. 25, Issue 3
  • DOI: 10.1137/140961602

Detection and Remediation of Stagnation in the Nelder--Mead Algorithm Using a Sufficient Decrease Condition
journal, January 1999


Scientific uses of the MANIAC
journal, June 1986

  • Anderson, H. L.
  • Journal of Statistical Physics, Vol. 43, Issue 5-6
  • DOI: 10.1007/BF02628301

Discrete Gradient Method: Derivative-Free Method for Nonsmooth Optimization
journal, December 2007

  • Bagirov, A. M.; Karasözen, B.; Sezer, M.
  • Journal of Optimization Theory and Applications, Vol. 137, Issue 2
  • DOI: 10.1007/s10957-007-9335-5

A Class of Trust-Region Methods for Parallel Optimization
journal, January 2002


SO-I: a surrogate model algorithm for expensive nonlinear integer programming problems including global optimization applications
journal, September 2013

  • Müller, Juliane; Shoemaker, Christine A.; Piché, Robert
  • Journal of Global Optimization, Vol. 59, Issue 4
  • DOI: 10.1007/s10898-013-0101-y

Convergence Rate Analysis of a Stochastic Trust-Region Method via Supermartingales
journal, April 2019

  • Blanchet, Jose; Cartis, Coralia; Menickelly, Matt
  • INFORMS Journal on Optimization, Vol. 1, Issue 2
  • DOI: 10.1287/ijoo.2019.0016

Multivariate Stochastic Process Models for Correlated Responses of Mixed Type
journal, September 2016

  • Pourmohamad, Tony; Lee, Herbert K. H.
  • Bayesian Analysis, Vol. 11, Issue 3
  • DOI: 10.1214/15-BA976

Google Vizier: A Service for Black-Box Optimization
conference, August 2017

  • Golovin, Daniel; Solnik, Benjamin; Moitra, Subhodeep
  • KDD '17: The 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
  • DOI: 10.1145/3097983.3098043

Derivative free methodologies for circuit worst case analysis
journal, December 2018


Recent progress in unconstrained nonlinear optimization without derivatives
journal, October 1997

  • Conn, A. R.; Scheinberg, K.; Toint, Ph. L.
  • Mathematical Programming, Vol. 79, Issue 1-3
  • DOI: 10.1007/BF02614326

Surrogate Optimization of Computationally Expensive Black-Box Problems with Hidden Constraints
journal, October 2019


Stochastic root finding via retrospective approximation
journal, March 2001