skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Derivative-free optimization methods

Abstract

In many optimization problems arising from scientific, engineering and artificial intelligence applications, objective and constraint functions are available only as the output of a black-box or simulation oracle that does not provide derivative information. Such settings necessitate the use of methods for derivative-free, or zeroth-order, optimization. We provide a review and perspectives on developments in these methods, with an emphasis on highlighting recent developments and on unifying treatment of such problems in the non-linear optimization and machine learning literature. We categorize methods based on assumed properties of the black-box functions, as well as features of the methods. We first overview the primary setting of deterministic methods applied to unconstrained, non-convex optimization problems where the objective function is defined by a deterministic black-box oracle. We then discuss developments in randomized methods, methods that assume some additional structure about the objective (including convexity, separability and general non-smooth compositions), methods for problems where the output of the black-box oracle is stochastic, and methods for handling different types of constraints.

Authors:
 [1];  [1];  [1]
  1. Argonne National Lab. (ANL), Argonne, IL (United States)
Publication Date:
Research Org.:
Argonne National Lab. (ANL), Argonne, IL (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)
OSTI Identifier:
1545343
Grant/Contract Number:  
AC02-06CH11357
Resource Type:
Accepted Manuscript
Journal Name:
Acta Numerica
Additional Journal Information:
Journal Volume: 28; Journal Issue: 2010; Journal ID: ISSN 0962-4929
Publisher:
Cambridge University Press
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; Optimization

Citation Formats

Larson, Jeffrey, Menickelly, Matt, and Wild, Stefan M. Derivative-free optimization methods. United States: N. p., 2019. Web. doi:10.1017/S0962492919000060.
Larson, Jeffrey, Menickelly, Matt, & Wild, Stefan M. Derivative-free optimization methods. United States. doi:10.1017/S0962492919000060.
Larson, Jeffrey, Menickelly, Matt, and Wild, Stefan M. Wed . "Derivative-free optimization methods". United States. doi:10.1017/S0962492919000060. https://www.osti.gov/servlets/purl/1545343.
@article{osti_1545343,
title = {Derivative-free optimization methods},
author = {Larson, Jeffrey and Menickelly, Matt and Wild, Stefan M.},
abstractNote = {In many optimization problems arising from scientific, engineering and artificial intelligence applications, objective and constraint functions are available only as the output of a black-box or simulation oracle that does not provide derivative information. Such settings necessitate the use of methods for derivative-free, or zeroth-order, optimization. We provide a review and perspectives on developments in these methods, with an emphasis on highlighting recent developments and on unifying treatment of such problems in the non-linear optimization and machine learning literature. We categorize methods based on assumed properties of the black-box functions, as well as features of the methods. We first overview the primary setting of deterministic methods applied to unconstrained, non-convex optimization problems where the objective function is defined by a deterministic black-box oracle. We then discuss developments in randomized methods, methods that assume some additional structure about the objective (including convexity, separability and general non-smooth compositions), methods for problems where the output of the black-box oracle is stochastic, and methods for handling different types of constraints.},
doi = {10.1017/S0962492919000060},
journal = {Acta Numerica},
number = 2010,
volume = 28,
place = {United States},
year = {2019},
month = {5}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Citation Metrics:
Cited by: 2 works
Citation information provided by
Web of Science

Save / Share:

Works referenced in this record:

Discrete Gradient Method: Derivative-Free Method for Nonsmooth Optimization
journal, December 2007

  • Bagirov, A. M.; Karasözen, B.; Sezer, M.
  • Journal of Optimization Theory and Applications, Vol. 137, Issue 2
  • DOI: 10.1007/s10957-007-9335-5

Piecewise Partially Separable Functions and a Derivative-free Algorithm for Large Scale Nonsmooth Optimization
journal, June 2006


On fast trust region methods for quadratic models with linear constraints
journal, June 2015


Beyond symmetric Broyden for updating quadratic models in minimization without derivatives
journal, January 2012


On the convergence of trust region algorithms for unconstrained minimization without derivatives
journal, April 2012


On the convergence of a wide range of trust region methods for unconstrained optimization
journal, September 2009


Robust Optimization for Unconstrained Simulation-Based Problems
journal, February 2010

  • Bertsimas, Dimitris; Nohadani, Omid; Teo, Kwong Meng
  • Operations Research, Vol. 58, Issue 1
  • DOI: 10.1287/opre.1090.0715

Robust optimization with simulated annealing
journal, December 2009


New horizons in sphere-packing theory, part II: lattice-based derivative-free optimization via global surrogates
journal, March 2012


A derivative-free comirror algorithm for convex optimization
journal, October 2014

  • Bauschke, Heinz H.; Hare, Warren L.; Moursi, Walaa M.
  • Optimization Methods and Software, Vol. 30, Issue 4
  • DOI: 10.1080/10556788.2014.968158

Two-Point Step Size Gradient Methods
journal, January 1988

  • Barzilai, Jonathan; Borwein, Jonathan M.
  • IMA Journal of Numerical Analysis, Vol. 8, Issue 1
  • DOI: 10.1093/imanum/8.1.141

Nelder-Mead Simplex Modifications for Simulation Optimization
journal, July 1996


ASTRO-DF: A Class of Adaptive Sampling Trust-Region Algorithms for Derivative-Free Stochastic Optimization
journal, January 2018

  • Shashaani, Sara; Hashemi, Fatemeh S.; Pasupathy, Raghu
  • SIAM Journal on Optimization, Vol. 28, Issue 4
  • DOI: 10.1137/15M1042425

Taking the Human Out of the Loop: A Review of Bayesian Optimization
journal, January 2016


Self-Correcting Geometry in Model-Based Algorithms for Derivative-Free Unconstrained Optimization
journal, January 2010

  • Scheinberg, K.; Toint, Ph. L.
  • SIAM Journal on Optimization, Vol. 20, Issue 6
  • DOI: 10.1137/090748536

A method for stochastic constrained optimization using derivative-free surrogate pattern search and collocation
journal, June 2010

  • Sankaran, Sethuraman; Audet, Charles; Marsden, Alison L.
  • Journal of Computational Physics, Vol. 229, Issue 12
  • DOI: 10.1016/j.jcp.2010.03.005

Numerical experience with a derivative-free trust-funnel method for nonlinear optimization problems with general nonlinear constraints
journal, February 2016


A derivative-free trust-funnel method for equality-constrained nonlinear optimization
journal, November 2014

  • Sampaio, Ph. R.; Toint, Ph. L.
  • Computational Optimization and Applications, Vol. 61, Issue 1
  • DOI: 10.1007/s10589-014-9715-3

Developments of NEWUOA for minimization without derivatives
journal, February 2008


On the use of quadratic models in unconstrained minimization without derivatives
journal, June 2004


Least Frobenius norm updating of quadratic models that satisfy interpolation conditions
journal, November 2003


On trust region methods for unconstrained minimization without derivatives
journal, August 2003


UOBYQA: unconstrained optimization by quadratic approximation
journal, May 2002


On the Lagrange functions of quadratic models that are defined by interpolation *
journal, January 2001


The solution of variational and boundary value problems by the method of local variations
journal, January 1966

  • Banichuk, N. V.; Petrov, V. M.; Chernous'ko, F. L.
  • USSR Computational Mathematics and Mathematical Physics, Vol. 6, Issue 6
  • DOI: 10.1016/0041-5553(66)90158-3

Convergence of Trust-Region Methods Based on Probabilistic Models
journal, January 2014

  • Bandeira, A. S.; Scheinberg, K.; Vicente, L. N.
  • SIAM Journal on Optimization, Vol. 24, Issue 3
  • DOI: 10.1137/130915984

Computation of sparse low degree interpolating polynomials and their application to derivative-free optimization
journal, July 2012


A trust region-based two phase algorithm for constrained black-box and grey-box optimization with infeasible initial point
journal, August 2018


A View of Unconstrained Minimization Algorithms that Do Not Require Derivatives
journal, June 1975


BFO, A Trainable Derivative-free Brute Force Optimizer for Nonlinear Bound-constrained Optimization and Equilibrium Computations with Continuous and Discrete Variables
journal, June 2017

  • Porcelli, Margherita; Toint, Philippe L.
  • ACM Transactions on Mathematical Software, Vol. 44, Issue 1
  • DOI: 10.1145/3085592

Precision Control for Generalized Pattern Search Algorithms with Adaptive Precision Function Evaluations
journal, January 2006

  • Polak, Elijah; Wetter, Michael
  • SIAM Journal on Optimization, Vol. 16, Issue 3
  • DOI: 10.1137/040605527

Optimal Condenser Design by Geometric Programming
journal, April 1967

  • Avriel, Mordecai; Wilde, D. J.
  • Industrial & Engineering Chemistry Process Design and Development, Vol. 6, Issue 2
  • DOI: 10.1021/i260022a018

The Nonstochastic Multiarmed Bandit Problem
journal, January 2002

  • Auer, Peter; Cesa-Bianchi, Nicolò; Freund, Yoav
  • SIAM Journal on Computing, Vol. 32, Issue 1
  • DOI: 10.1137/S0097539701398375

A mesh adaptive direct search algorithm for multiobjective optimization
journal, August 2010

  • Audet, Charles; Savard, Gilles; Zghal, Walid
  • European Journal of Operational Research, Vol. 204, Issue 3
  • DOI: 10.1016/j.ejor.2009.11.010

Multiobjective Optimization Through a Series of Single-Objective Formulations
journal, January 2008

  • Audet, Charles; Savard, Gilles; Zghal, Walid
  • SIAM Journal on Optimization, Vol. 19, Issue 1
  • DOI: 10.1137/060677513

Linear equalities in blackbox optimization
journal, October 2014

  • Audet, Charles; Le Digabel, Sébastien; Peyrega, Mathilde
  • Computational Optimization and Applications, Vol. 61, Issue 1
  • DOI: 10.1007/s10589-014-9708-2

Pure adaptive search in monte carlo optimization
journal, January 1989

  • Patel, Nitin R.; Smith, Robert L.; Zabinsky, Zelda B.
  • Mathematical Programming, Vol. 43, Issue 1-3
  • DOI: 10.1007/BF01582296

On Sampling Rates in Simulation-Based Recursions
journal, January 2018

  • Pasupathy, Raghu; Glynn, Peter; Ghosh, Soumyadip
  • SIAM Journal on Optimization, Vol. 28, Issue 1
  • DOI: 10.1137/140951679

Robust optimization of noisy blackbox problems using the Mesh Adaptive Direct Search algorithm
journal, January 2018

  • Audet, Charles; Ihaddadene, Amina; Le Digabel, Sébastien
  • Optimization Letters, Vol. 12, Issue 4
  • DOI: 10.1007/s11590-017-1226-6

Reducing the Number of Function Evaluations in Mesh Adaptive Direct Search Algorithms
journal, January 2014

  • Audet, Charles; Ianni, Andrea; Le Digabel, Sébastien
  • SIAM Journal on Optimization, Vol. 24, Issue 2
  • DOI: 10.1137/120895056

Parallel Space Decomposition of the Mesh Adaptive Direct Search Algorithm
journal, January 2008

  • Audet, Charles; Dennis, J. E.; Digabel, Sébastien Le
  • SIAM Journal on Optimization, Vol. 19, Issue 3
  • DOI: 10.1137/070707518

A Progressive Barrier for Derivative-Free Nonlinear Programming
journal, January 2009

  • Audet, Charles; Dennis, J. E.
  • SIAM Journal on Optimization, Vol. 20, Issue 1
  • DOI: 10.1137/070692662

Mesh Adaptive Direct Search Algorithms for Constrained Optimization
journal, January 2006

  • Audet, Charles; Dennis, J. E.
  • SIAM Journal on Optimization, Vol. 17, Issue 1
  • DOI: 10.1137/040603371

A Pattern Search Filter Method for Nonlinear Programming without Derivatives
journal, January 2004


Analysis of Generalized Pattern Searches
journal, January 2002


Pattern Search Algorithms for Mixed Variable Programming
journal, January 2001


Boosters: A Derivative-Free Algorithm Based on Radial Basis Functions
journal, January 2009


A trust-region-based derivative free algorithm for mixed integer programming
journal, April 2014


VXQR: derivative-free unconstrained optimization based on QR factorizations
journal, September 2010


Random Gradient-Free Minimization of Convex Functions
journal, November 2015

  • Nesterov, Yurii; Spokoiny, Vladimir
  • Foundations of Computational Mathematics, Vol. 17, Issue 2
  • DOI: 10.1007/s10208-015-9296-2

Objective-derivative-free methods for constrained optimization
journal, March 2002

  • Lucidi, S.; Sciandrone, M.; Tseng, P.
  • Mathematical Programming, Vol. 92, Issue 1
  • DOI: 10.1007/s101070100266

A multigrid approach to discretized optimization problems
journal, January 2000


On the Global Convergence of Derivative-Free Methods for Unconstrained Optimization
journal, January 2002


Convergence Results for Generalized Pattern Search Algorithms are Tight
journal, June 2004


Sequential Penalty Derivative-Free Methods for Nonlinear Constrained Optimization
journal, January 2010

  • Liuzzi, Giampaolo; Lucidi, Stefano; Sciandrone, Marco
  • SIAM Journal on Optimization, Vol. 20, Issue 5
  • DOI: 10.1137/090750639

A Derivative-Free Algorithm for Linearly Constrained Finite Minimax Problems
journal, January 2006

  • Liuzzi, G.; Lucidi, S.; Sciandrone, M.
  • SIAM Journal on Optimization, Vol. 16, Issue 4
  • DOI: 10.1137/040615821

Scientific uses of the MANIAC
journal, June 1986

  • Anderson, H. L.
  • Journal of Statistical Physics, Vol. 43, Issue 5-6
  • DOI: 10.1007/BF02628301

Simulation optimization: a review of algorithms and applications
journal, September 2015

  • Amaran, Satyajith; Sahinidis, Nikolaos V.; Sharda, Bikram
  • Annals of Operations Research, Vol. 240, Issue 1
  • DOI: 10.1007/s10479-015-2019-x

A Derivative-Free Approach to Constrained Multiobjective Nonsmooth Optimization
journal, January 2016

  • Liuzzi, G.; Lucidi, S.; Rinaldi, F.
  • SIAM Journal on Optimization, Vol. 26, Issue 4
  • DOI: 10.1137/15M1037810

Efficient solution of quadratically constrained quadratic subproblems within the mesh adaptive direct search algorithm
journal, July 2018

  • Amaioua, Nadir; Audet, Charles; Conn, Andrew R.
  • European Journal of Operational Research, Vol. 268, Issue 1
  • DOI: 10.1016/j.ejor.2017.10.058

Derivative-Free Methods for Mixed-Integer Constrained Optimization Problems
journal, July 2014

  • Liuzzi, Giampaolo; Lucidi, Stefano; Rinaldi, Francesco
  • Journal of Optimization Theory and Applications, Vol. 164, Issue 3
  • DOI: 10.1007/s10957-014-0617-4

A trust-region framework for managing the use of approximation models in optimization
journal, February 1998

  • Alexandrov, N. M.; Dennis, J. E.; Lewis, R. M.
  • Structural Optimization, Vol. 15, Issue 1
  • DOI: 10.1007/BF01197433

Derivative-free methods for bound constrained mixed-integer optimization
journal, April 2011

  • Liuzzi, G.; Lucidi, S.; Rinaldi, F.
  • Computational Optimization and Applications, Vol. 53, Issue 2
  • DOI: 10.1007/s10589-011-9405-3

A Derivative-Free Algorithm for Inequality Constrained Nonlinear Programming via Smoothing of an $\ell_\infty$ Penalty Function
journal, January 2009

  • Liuzzi, G.; Lucidi, S.
  • SIAM Journal on Optimization, Vol. 20, Issue 1
  • DOI: 10.1137/070711451

Sample mean based index policies by O (log n ) regret for the multi-armed bandit problem
journal, December 1995

  • Agrawal, Rajeev
  • Advances in Applied Probability, Vol. 27, Issue 4
  • DOI: 10.2307/1427934

SO-MI: A surrogate model algorithm for computationally expensive nonlinear mixed-integer black-box global optimization problems
journal, May 2013

  • Müller, Juliane; Shoemaker, Christine A.; Piché, Robert
  • Computers & Operations Research, Vol. 40, Issue 5
  • DOI: 10.1016/j.cor.2012.08.022

SO-I: a surrogate model algorithm for expensive nonlinear integer programming problems including global optimization applications
journal, September 2013

  • Müller, Juliane; Shoemaker, Christine A.; Piché, Robert
  • Journal of Global Optimization, Vol. 59, Issue 4
  • DOI: 10.1007/s10898-013-0101-y

GOSAC: global optimization with surrogate approximation of constraints
journal, January 2017


MISO: mixed-integer surrogate optimization framework
journal, June 2015


Approximate norm descent methods for constrained nonlinear systems
journal, May 2017

  • Morini, Benedetta; Porcelli, Margherita; Toint, Philippe L.
  • Mathematics of Computation, Vol. 87, Issue 311
  • DOI: 10.1090/mcom/3251

Do you trust derivatives or differences?
journal, September 2014


Estimating Derivatives of Noisy Simulations
journal, April 2012

  • Moré, Jorge J.; Wild, Stefan M.
  • ACM Transactions on Mathematical Software, Vol. 38, Issue 3
  • DOI: 10.1145/2168773.2168777

A polynomial case of convex integer quadratic programming problems with box integer constraints
journal, January 2015


Estimating Computational Noise
journal, January 2011

  • Moré, Jorge J.; Wild, Stefan M.
  • SIAM Journal on Scientific Computing, Vol. 33, Issue 3
  • DOI: 10.1137/100786125

A class of derivative-free methods for large-scale nonlinear monotone equations
journal, January 2011


A derivative-free line search and global convergence of Broyden-like method for nonlinear equations
journal, January 2000


Direct search methods: then and now
journal, December 2000

  • Lewis, Robert Michael; Torczon, Virginia; Trosset, Michael W.
  • Journal of Computational and Applied Mathematics, Vol. 124, Issue 1-2
  • DOI: 10.1016/S0377-0427(00)00423-4

A Globally Convergent Augmented Lagrangian Pattern Search Algorithm for Optimization with General Constraints and Simple Bounds
journal, January 2002


Pattern Search Algorithms for Bound Constrained Minimization
journal, January 1999


Benchmarking Derivative-Free Optimization Algorithms
journal, January 2009

  • Moré, Jorge J.; Wild, Stefan M.
  • SIAM Journal on Optimization, Vol. 20, Issue 1
  • DOI: 10.1137/080724083

Computing a Trust Region Step
journal, September 1983

  • Moré, Jorge J.; Sorensen, D. C.
  • SIAM Journal on Scientific and Statistical Computing, Vol. 4, Issue 3
  • DOI: 10.1137/0904038

A superlinearly convergent algorithm for minimization without evaluating derivatives
journal, December 1975


Direct search algorithm for bilevel programming problems
journal, September 2009

  • Mersha, Ayalew Getachew; Dempe, Stephan
  • Computational Optimization and Applications, Vol. 49, Issue 1
  • DOI: 10.1007/s10589-009-9295-9

A method for the solution of certain non-linear problems in least squares
journal, January 1944

  • Levenberg, Kenneth
  • Quarterly of Applied Mathematics, Vol. 2, Issue 2
  • DOI: 10.1090/qam/10666

Convergence of the Nelder--Mead Simplex Method to a Nonstationary Point
journal, January 1998


Positive bases for linear spaces
journal, January 1962


Budget-Dependent Convergence Rate of Stochastic Approximation
journal, February 1998


Pattern Search Methods for Linearly Constrained Minimization
journal, January 2000


Algorithm 909: NOMAD: Nonlinear Optimization with the MADS Algorithm
journal, February 2011


Calibration by optimization without using derivatives
journal, July 2016


Manifold Sampling for $\ell_1$ Nonconvex Optimization
journal, January 2016

  • Larson, Jeffrey; Menickelly, Matt; Wild, Stefan M.
  • SIAM Journal on Optimization, Vol. 26, Issue 4
  • DOI: 10.1137/15M1042097

Solving Nonlinear Programs Without Using Analytic Derivatives
journal, June 1979


Constrained derivative-free optimization on thin domains
journal, July 2012


An Algorithm for Least-Squares Estimation of Nonlinear Parameters
journal, June 1963

  • Marquardt, Donald W.
  • Journal of the Society for Industrial and Applied Mathematics, Vol. 11, Issue 2
  • DOI: 10.1137/0111030

Provably Convergent Multifidelity Optimization Algorithm Not Requiring High-Fidelity Derivatives
journal, May 2012

  • March, Andrew; Willcox, Karen
  • AIAA Journal, Vol. 50, Issue 5
  • DOI: 10.2514/1.J051125

Wedge trust region methods for derivative free optimization
journal, January 2002


A Derivative-Free Trust-Region Algorithm for the Optimization of Functions Smoothed via Gaussian Convolution Using Adaptive Multiple Importance Sampling
journal, January 2018

  • Maggiar, Alvaro; Wächter, Andreas; Dolinskaya, Irina S.
  • SIAM Journal on Optimization, Vol. 28, Issue 2
  • DOI: 10.1137/15M1031679

Asynchronously parallel optimization solver for finding multiple minima
journal, February 2018


Pattern search methods for finite minimax problems
journal, March 2009


A batch, derivative-free algorithm for finding multiple local minima
journal, October 2015


Stochastic derivative-free optimization using a trust region framework
journal, February 2016

  • Larson, Jeffrey; Billups, Stephen C.
  • Computational Optimization and Applications, Vol. 64, Issue 3
  • DOI: 10.1007/s10589-016-9827-z

Asymptotically efficient adaptive allocation rules
journal, March 1985


Convergence Properties of the Nelder--Mead Simplex Method in Low Dimensions
journal, January 1998

  • Lagarias, Jeffrey C.; Reeds, James A.; Wright, Margaret H.
  • SIAM Journal on Optimization, Vol. 9, Issue 1
  • DOI: 10.1137/S1052623496303470

Convergence of the Restricted Nelder--Mead Algorithm in Two Dimensions
journal, January 2012

  • Lagarias, Jeffrey C.; Poonen, Bjorn; Wright, Margaret H.
  • SIAM Journal on Optimization, Vol. 22, Issue 2
  • DOI: 10.1137/110830150

Spectral residual method without gradient information for solving large-scale nonlinear systems of equations
journal, April 2006


A projected derivative-free algorithm for nonlinear equations with convex constraints
journal, July 2012


Rates of Convergence for Stochastic Approximation Type Algorithms
journal, September 1979

  • Kushner, Harold J.; Huang, Hai
  • SIAM Journal on Control and Optimization, Vol. 17, Issue 5
  • DOI: 10.1137/0317043

Optimization by Direct Search: New Perspectives on Some Classical and Modern Methods
journal, January 2003

  • Kolda, Tamara G.; Lewis, Robert Michael; Torczon, Virginia
  • SIAM Review, Vol. 45, Issue 3
  • DOI: 10.1137/S003614450242889

Stationarity Results for Generating Set Search for Linearly Constrained Optimization
journal, January 2007

  • Kolda, Tamara G.; Lewis, Robert Michael; Torczon, Virginia
  • SIAM Journal on Optimization, Vol. 17, Issue 4
  • DOI: 10.1137/S1052623403433638

Simulation-Based Optimization with Stochastic Approximation Using Common Random Numbers
journal, November 1999

  • Kleinman, Nathan L.; Spall, James C.; Naiman, Daniel Q.
  • Management Science, Vol. 45, Issue 11
  • DOI: 10.1287/mnsc.45.11.1570

A Nonderivative Version of the Gradient Sampling Algorithm for Nonsmooth Nonconvex Optimization
journal, January 2010

  • Kiwiel, Krzysztof C.
  • SIAM Journal on Optimization, Vol. 20, Issue 4
  • DOI: 10.1137/090748408

A Trust-Region Algorithm for Bi-Objective Stochastic Optimization
journal, January 2011


Stochastic Estimation of the Maximum of a Regression Function
journal, September 1952


Manifold Sampling for Optimization of Nonconvex Functions That Are Piecewise Linear Compositions of Smooth Components
journal, January 2018

  • Khan, Kamil A.; Larson, Jeffrey; Wild, Stefan M.
  • SIAM Journal on Optimization, Vol. 28, Issue 4
  • DOI: 10.1137/17M114741X

Detection and Remediation of Stagnation in the Nelder--Mead Algorithm Using a Sufficient Decrease Condition
journal, January 1999


Convergence estimates for iterative minimization methods
journal, January 1974


Lipschitzian optimization without the Lipschitz constant
journal, October 1993

  • Jones, D. R.; Perttunen, C. D.; Stuckman, B. E.
  • Journal of Optimization Theory and Applications, Vol. 79, Issue 1
  • DOI: 10.1007/BF00941892

A subclass of generating set search with convergence to second-order stationary points
journal, June 2013

  • Abramson, Mark A.; Frimannslund, Lennart; Steihaug, Trond
  • Optimization Methods and Software, Vol. 29, Issue 5
  • DOI: 10.1080/10556788.2013.808198

OrthoMADS: A Deterministic MADS Instance with Orthogonal Directions
journal, January 2009

  • Abramson, Mark A.; Audet, Charles; Dennis,, J. E.
  • SIAM Journal on Optimization, Vol. 20, Issue 2
  • DOI: 10.1137/080716980

Mesh adaptive direct search algorithms for mixed variable optimization
journal, June 2008


Generalized pattern searches with derivative information
journal, November 2003

  • Abramson, Mark A.; Audet, Charles; Dennis, J. E.
  • Mathematical Programming, Vol. 100, Issue 1
  • DOI: 10.1007/s10107-003-0484-5

Convergence of Mesh Adaptive Direct Search to Second‐Order Stationary Points
journal, January 2006

  • Abramson, Mark A.; Audet, Charles
  • SIAM Journal on Optimization, Vol. 17, Issue 2
  • DOI: 10.1137/050638382

Second-Order Behavior of Pattern Search
journal, January 2005

  • Abramson, Mark A.
  • SIAM Journal on Optimization, Vol. 16, Issue 2
  • DOI: 10.1137/04060367X

SNOBFIT -- Stable Noisy Optimization by Branch and Fit
journal, July 2008

  • Huyer, Waltraud; Neumaier, Arnold
  • ACM Transactions on Mathematical Software, Vol. 35, Issue 2
  • DOI: 10.1145/1377612.1377613

Asynchronous Parallel Pattern Search for Nonlinear Optimization
journal, January 2001

  • Hough, Patricia D.; Kolda, Tamara G.; Torczon, Virginia J.
  • SIAM Journal on Scientific Computing, Vol. 23, Issue 1
  • DOI: 10.1137/S1064827599365823

A Class of Trust-Region Methods for Parallel Optimization
journal, January 2002


A second-order globally convergent direct-search method and its worst-case complexity
journal, October 2015


`` Direct Search'' Solution of Numerical and Statistical Problems
journal, April 1961


A Merit Function Approach for Direct Search
journal, January 2014

  • Gratton, S.; Vicente, L. N.
  • SIAM Journal on Optimization, Vol. 24, Issue 4
  • DOI: 10.1137/130917661

A derivative-free trust-region algorithm for composite nonsmooth optimization
journal, December 2014

  • Grapiglia, Geovani Nunes; Yuan, Jinyun; Yuan, Ya-xiang
  • Computational and Applied Mathematics, Vol. 35, Issue 2
  • DOI: 10.1007/s40314-014-0201-4

Nonlinear programming without a penalty function or a filter
journal, September 2008


A Globally Convergent Filter Method for Nonlinear Programming
journal, January 2004

  • Gonzaga, Clóvis C.; Karas, Elizabeth; Vanti, Márcia
  • SIAM Journal on Optimization, Vol. 14, Issue 3
  • DOI: 10.1137/S1052623401399320

Sequential Search: A Method for Solving Constrained Optimization Problems
journal, January 1965


An Implicit Filtering Algorithm for Optimization of Functions with Many Local Minima
journal, May 1995

  • Gilmore, P.; Kelley, C. T.
  • SIAM Journal on Optimization, Vol. 5, Issue 2
  • DOI: 10.1137/0805015

Computing Forward-Difference Intervals for Numerical Optimization
journal, June 1983

  • Gill, Philip E.; Murray, Walter; Saunders, Michael A.
  • SIAM Journal on Scientific and Statistical Computing, Vol. 4, Issue 2
  • DOI: 10.1137/0904025

Algorithm 897: VTDIRECT95: Serial and parallel codes for the global optimization algorithm direct
journal, July 2009

  • He, Jian; Watson, Layne T.; Sosonkina, Masha
  • ACM Transactions on Mathematical Software, Vol. 36, Issue 3
  • DOI: 10.1145/1527286.1527291

Performance Modeling and Analysis of a Massively Parallel Direct—Part 1
journal, February 2009

  • Jian He, ; Verstak, Alex; Watson, L. T.
  • The International Journal of High Performance Computing Applications, Vol. 23, Issue 1
  • DOI: 10.1177/1094342008098462

Design and implementation of a massively parallel version of DIRECT
journal, October 2007

  • He, Jian; Verstak, Alex; Watson, Layne T.
  • Computational Optimization and Applications, Vol. 40, Issue 2
  • DOI: 10.1007/s10589-007-9092-2

Performance Modeling and Analysis of a Massively Parallel Direct—Part 2
journal, February 2009

  • Jian He, ; Verstak, Alex; Sosonkina, M.
  • The International Journal of High Performance Computing Applications, Vol. 23, Issue 1
  • DOI: 10.1177/1094342008098463

A derivative-free approximate gradient sampling algorithm for finite minimax problems
journal, March 2013


Derivative-free optimization methods for finite minimax problems
journal, April 2013


Derivative-Free Optimization Via Proximal Point Methods
journal, June 2013


Stochastic First- and Zeroth-Order Methods for Nonconvex Stochastic Programming
journal, January 2013

  • Ghadimi, Saeed; Lan, Guanghui
  • SIAM Journal on Optimization, Vol. 23, Issue 4
  • DOI: 10.1137/120880811

Estimating Tangent and Normal Cones Without Calculus
journal, November 2005


Conditional gradient type methods for composite nonlinear and stochastic optimization
journal, January 2018


Stochastic online optimization. Single-point and multi-point non-linear multi-armed bandits. Convex and strongly-convex case
journal, February 2017

  • Gasnikov, A. V.; Krymova, E. A.; Lagunovskaya, A. A.
  • Automation and Remote Control, Vol. 78, Issue 2
  • DOI: 10.1134/S0005117917020035

Trust-Region Methods Without Using Derivatives: Worst Case Complexity and the NonSmooth Case
journal, January 2016

  • Garmanjani, R.; Júdice, D.; Vicente, L. N.
  • SIAM Journal on Optimization, Vol. 26, Issue 4
  • DOI: 10.1137/151005683

Smoothing and worst-case complexity for direct-search methods in nonsmooth optimization
journal, October 2012

  • Garmanjani, R.; Vicente, L. N.
  • IMA Journal of Numerical Analysis, Vol. 33, Issue 3
  • DOI: 10.1093/imanum/drs027

On sequential and parallel non-monotone derivative-free algorithms for box constrained optimization
journal, June 2012

  • García-Palomares, Ubaldo M.; García-Urrea, Ildemaro J.; Rodríguez-Hernández, Pedro S.
  • Optimization Methods and Software, Vol. 28, Issue 6
  • DOI: 10.1080/10556788.2012.693926

New Sequential and Parallel Derivative-Free Algorithms for Unconstrained Minimization
journal, January 2002


Implementing the Nelder-Mead simplex algorithm with adaptive parameters
journal, May 2010


Compositions of convex functions and fully linear models
journal, February 2017


On the Number of Iterations of Piyavskii's Global Optimization Algorithm
journal, May 1991

  • Hansen, P.; Jaumard, B.; Lu, S. -H.
  • Mathematics of Operations Research, Vol. 16, Issue 2
  • DOI: 10.1287/moor.16.2.334

On the convergence of the UOBYQA method
journal, March 2004

  • Han, Lixing; Liu, Guanghui
  • Journal of Applied Mathematics and Computing, Vol. 16, Issue 1-2
  • DOI: 10.1007/BF02936156

A derivative-free algorithm for linearly constrained optimization problems
journal, October 2013

  • Gumma, E. A. E.; Hashim, M. H. A.; Ali, M. Montaz
  • Computational Optimization and Applications, Vol. 57, Issue 3
  • DOI: 10.1007/s10589-013-9607-y

Global convergence and stabilization of unconstrained minimization methods without derivatives
journal, March 1988

  • Grippo, L.; Lampariello, F.; Lucidi, S.
  • Journal of Optimization Theory and Applications, Vol. 56, Issue 3
  • DOI: 10.1007/BF00939550

Nonmonotone derivative-free methods for nonlinear equations
journal, March 2007


A generating set search method using curvature information
journal, May 2007

  • Frimannslund, Lennart; Steihaug, Trond
  • Computational Optimization and Applications, Vol. 38, Issue 1
  • DOI: 10.1007/s10589-007-9038-8

Coordinate search algorithms in multilevel optimization
journal, November 2013


A class of derivative-free nonmonotone optimization algorithms employing coordinate rotations and gradient approximations
journal, June 2014


On Lipschitz optimization based on gray-box piecewise linearization
journal, July 2015

  • Griewank, Andreas; Walther, Andrea; Fiege, Sabrina
  • Mathematical Programming, Vol. 158, Issue 1-2
  • DOI: 10.1007/s10107-015-0934-x

Algorithm 856: APPSPACK 4.0: asynchronous parallel pattern search for derivative-free optimization
journal, September 2006

  • Gray, Genetha A.; Kolda, Tamara G.
  • ACM Transactions on Mathematical Software, Vol. 32, Issue 3
  • DOI: 10.1145/1163641.1163647

An active-set trust-region method for derivative-free nonlinear bound-constrained optimization
journal, October 2011

  • Gratton, Serge; Toint, Philippe L.; Tröltzsch, Anke
  • Optimization Methods and Software, Vol. 26, Issue 4-5
  • DOI: 10.1080/10556788.2010.549231

Direct search based on probabilistic feasible descent for bound and linearly constrained problems
journal, January 2019

  • Gratton, S.; Royer, C. W.; Vicente, L. N.
  • Computational Optimization and Applications, Vol. 72, Issue 3
  • DOI: 10.1007/s10589-019-00062-4

Complexity and global rates of trust-region methods based on probabilistic models
journal, August 2017

  • Gratton, Serge; Royer, Clément W.; Vicente, Luís N.
  • IMA Journal of Numerical Analysis, Vol. 38, Issue 3
  • DOI: 10.1093/imanum/drx043

Direct Search Based on Probabilistic Descent
journal, January 2015

  • Gratton, S.; Royer, C. W.; Vicente, L. N.
  • SIAM Journal on Optimization, Vol. 25, Issue 3
  • DOI: 10.1137/140961602

Additive Scaling and the DIRECT Algorithm
journal, June 2006


Global convergence of a derivative-free inexact restoration filter algorithm for nonlinear programming
journal, November 2016


On the geometry phase in model-based algorithms for derivative-free optimization
journal, February 2009

  • Fasano, Giovanni; Morales, José Luis; Nocedal, Jorge
  • Optimization Methods and Software, Vol. 24, Issue 1
  • DOI: 10.1080/10556780802409296

A Linesearch-Based Derivative-Free Approach for Nonsmooth Constrained Optimization
journal, January 2014

  • Fasano, G.; Liuzzi, G.; Lucidi, S.
  • SIAM Journal on Optimization, Vol. 24, Issue 3
  • DOI: 10.1137/130940037

A grid algorithm for bound constrained optimization of noisy functions
journal, January 1995

  • Elster, Clemens; Neumaier, Arnold
  • IMA Journal of Numerical Analysis, Vol. 15, Issue 4
  • DOI: 10.1093/imanum/15.4.585

An inexact restoration derivative-free filter method for nonlinear programming
journal, July 2015

  • Echebest, N.; Schuverdt, M. L.; Vignau, R. P.
  • Computational and Applied Mathematics, Vol. 36, Issue 1
  • DOI: 10.1007/s40314-015-0253-0

Optimal Rates for Zero-Order Convex Optimization: The Power of Two Function Evaluations
journal, May 2015

  • Duchi, John C.; Jordan, Michael I.; Wainwright, Martin J.
  • IEEE Transactions on Information Theory, Vol. 61, Issue 5
  • DOI: 10.1109/TIT.2015.2409256

On the Local Convergence of Pattern Search
journal, January 2003

  • Dolan, Elizabeth D.; Lewis, Robert Michael; Torczon, Virginia
  • SIAM Journal on Optimization, Vol. 14, Issue 2
  • DOI: 10.1137/S1052623400374495

On the optimal order of worst case complexity of direct search
journal, June 2015


Worst case complexity of direct search under convexity
journal, November 2014


A derivative-free nonmonotone line-search technique for unconstrained optimization
journal, October 2008

  • Diniz-Ehrhardt, M. A.; Martínez, J. M.; Raydan, M.
  • Journal of Computational and Applied Mathematics, Vol. 219, Issue 2
  • DOI: 10.1016/j.cam.2007.07.017

An effective algorithm for hyperparameter optimization of neural networks
journal, July 2017

  • Diaz, G. I.; Fokoue-Nkoutche, A.; Nannicini, G.
  • IBM Journal of Research and Development, Vol. 61, Issue 4/5
  • DOI: 10.1147/JRD.2017.2709578

An Application of Chung's Lemma to the Kiefer-Wolfowitz Stochastic Approximation Procedure
journal, June 1956


Direct Search Methods on Parallel Machines
journal, November 1991

  • Dennis, Jr., J. E.; Torczon, Virginia
  • SIAM Journal on Optimization, Vol. 1, Issue 4
  • DOI: 10.1137/0801027

Adaptation of the Uobyqa Algorithm for Noisy Functions
conference, December 2006


Variable-Number Sample-Path Optimization
journal, July 2007


Stopping criteria for linesearch methods without derivatives
journal, October 1984

  • De Leone, Renato; Gaudioso, Manlio; Grippo, Luigi
  • Mathematical Programming, Vol. 30, Issue 3
  • DOI: 10.1007/BF02591934

Exploiting Known Structures to Approximate Normal Cones
journal, November 2013


Theory of Positive Linear Dependence
journal, October 1954

  • Davis, Chandler
  • American Journal of Mathematics, Vol. 76, Issue 4
  • DOI: 10.2307/2372648

MILP models for the selection of a small set of well-distributed points
journal, January 2017

  • D'Ambrosio, Claudia; Nannicini, Giacomo; Sartor, Giorgio
  • Operations Research Letters, Vol. 45, Issue 1
  • DOI: 10.1016/j.orl.2016.11.004

Incorporating minimum Frobenius norm models in direct search
journal, August 2009

  • Custódio, A. L.; Rocha, H.; Vicente, L. N.
  • Computational Optimization and Applications, Vol. 46, Issue 2
  • DOI: 10.1007/s10589-009-9283-0

Direct Multisearch for Multiobjective Optimization
journal, July 2011

  • Custódio, A. L.; Madeira, J. F. A.; Vaz, A. I. F.
  • SIAM Journal on Optimization, Vol. 21, Issue 3
  • DOI: 10.1137/10079731X

Using simplex gradients of nonsmooth functions in direct search methods
journal, February 2008

  • Custodio, A. L.; Dennis, J. E.; Vicente, L. N.
  • IMA Journal of Numerical Analysis, Vol. 28, Issue 4
  • DOI: 10.1093/imanum/drn045

Using Sampling and Simplex Derivatives in Pattern Search Methods
journal, January 2007

  • Custódio, A. L.; Vicente, L. N.
  • SIAM Journal on Optimization, Vol. 18, Issue 2
  • DOI: 10.1137/050646706

GLODS: Global and Local Optimization using Direct Search
journal, August 2014


Efficient calculation of regular simplex gradients
journal, January 2019


Frame Based Methods for Unconstrained Optimization
journal, November 2000

  • Coope, I. D.; Price, C. J.
  • Journal of Optimization Theory and Applications, Vol. 107, Issue 2
  • DOI: 10.1023/A:1026429319405

Global Convergence of General Derivative-Free Trust-Region Algorithms to First- and Second-Order Critical Points
journal, January 2009

  • Conn, Andrew R.; Scheinberg, Katya; Vicente, Luís N.
  • SIAM Journal on Optimization, Vol. 20, Issue 1
  • DOI: 10.1137/060673424

Geometry of sample sets in derivative-free optimization: polynomial regression and underdetermined interpolation
journal, February 2008

  • Conn, A. R.; Scheinberg, K.; Vicente, L. N.
  • IMA Journal of Numerical Analysis, Vol. 28, Issue 4
  • DOI: 10.1093/imanum/drn046

Geometry of interpolation sets in derivative free optimization
journal, December 2006

  • Conn, A. R.; Scheinberg, K.; Vicente, Luís N.
  • Mathematical Programming, Vol. 111, Issue 1-2
  • DOI: 10.1007/s10107-006-0073-5

A Globally Convergent Augmented Lagrangian Algorithm for Optimization with General Constraints and Simple Bounds
journal, April 1991

  • Conn, Andrew R.; Gould, Nicholas I. M.; Toint, Philippe
  • SIAM Journal on Numerical Analysis, Vol. 28, Issue 2
  • DOI: 10.1137/0728030

Bilevel derivative-free optimization and its application to robust optimization
journal, June 2012


Use of quadratic models with mesh-adaptive direct search for constrained black box optimization
journal, February 2013


A trust-region derivative-free algorithm for constrained optimization
journal, April 2015


Optimizing partially separable functions without derivatives
journal, August 2005


An implicit filtering algorithm for derivative-free multiobjective optimization with box constraints
journal, October 2017

  • Cocchi, G.; Liuzzi, G.; Papini, A.
  • Computational Optimization and Applications, Vol. 69, Issue 2
  • DOI: 10.1007/s10589-017-9953-2

Derivative-Free Robust Optimization for Circuit Design
journal, October 2013

  • Ciccazzo, Angelo; Latorre, Vittorio; Liuzzi, Giampaolo
  • Journal of Optimization Theory and Applications, Vol. 164, Issue 3
  • DOI: 10.1007/s10957-013-0441-2

Superlinear Convergence and Implicit Filtering
journal, January 2000


Stochastic Nelder–Mead simplex method – A new globally convergent direct search method for simulation optimization
journal, August 2012


On the Oracle Complexity of First-Order and Derivative-Free Algorithms for Smooth Nonconvex Minimization
journal, January 2012

  • Cartis, Coralia; Gould, Nicholas I. M.; Toint, Philippe L.
  • SIAM Journal on Optimization, Vol. 22, Issue 1
  • DOI: 10.1137/100812276

Bilevel direct search method for leader–follower problems and application in health insurance
journal, January 2014


Pure adaptive search in global optimization
journal, January 1992

  • Zabinsky, Zelda B.; Smith, Robert L.
  • Mathematical Programming, Vol. 53, Issue 1-3
  • DOI: 10.1007/BF01585710

Conditions for convergence of trust region algorithms for nonsmooth optimization
journal, June 1985


Sequential Design and Analysis of High-Accuracy and Low-Accuracy Computer Codes
journal, August 2012


Function Minimization by Interpolation in a Data Table
journal, January 1973


Global convergence rate analysis of unconstrained optimization methods based on probabilistic models
journal, April 2017


Grid Restrained Nelder-Mead Algorithm
journal, March 2006

  • Bűrmen, Árpád; Puhan, Janez; Tuma, Tadej
  • Computational Optimization and Applications, Vol. 34, Issue 3
  • DOI: 10.1007/s10589-005-3912-z

Mesh adaptive direct search with second directional derivative-based Hessian update
journal, April 2015

  • Bűrmen, Árpád; Olenšek, Jernej; Tuma, Tadej
  • Computational Optimization and Applications, Vol. 62, Issue 3
  • DOI: 10.1007/s10589-015-9753-5

Inexact Restoration Method for Derivative-Free Optimization with Smooth Constraints
journal, January 2013

  • Bueno, L. F.; Friedlander, A.; Martínez, J. M.
  • SIAM Journal on Optimization, Vol. 23, Issue 2
  • DOI: 10.1137/110856253

ORBIT: Optimization by Radial Basis Function Interpolation in Trust-Regions
journal, January 2008

  • Wild, Stefan M.; Regis, Rommel G.; Shoemaker, Christine A.
  • SIAM Journal on Scientific Computing, Vol. 30, Issue 6
  • DOI: 10.1137/070691814

Global Convergence of Radial Basis Function Trust-Region Algorithms for Derivative-Free Optimization
journal, January 2013

  • Wild, Stefan M.; Shoemaker, Christine
  • SIAM Review, Vol. 55, Issue 2
  • DOI: 10.1137/120902434

Global Convergence of Radial Basis Function Trust Region Derivative-Free Algorithms
journal, July 2011

  • Wild, Stefan M.; Shoemaker, Christine
  • SIAM Journal on Optimization, Vol. 21, Issue 3
  • DOI: 10.1137/09074927X

Surrogate-based methods for black-box optimization: Surrogate-based methods for black-box optimization
journal, April 2016

  • Vu, Ky Khac; D'Ambrosio, Claudia; Hamadi, Youssef
  • International Transactions in Operational Research, Vol. 24, Issue 3
  • DOI: 10.1111/itor.12292

Asymptotic Distribution of Stochastic Approximation Procedures
journal, June 1958


Analysis of direct searches for discontinuous functions
journal, December 2010


A Derivative-Free Trust-Region Method for Biobjective Optimization
journal, January 2014

  • Ryu, Jong-hyun; Kim, Sujin
  • SIAM Journal on Optimization, Vol. 24, Issue 1
  • DOI: 10.1137/120864738

A Stochastic Approximation Method
journal, September 1951

  • Robbins, Herbert; Monro, Sutton
  • The Annals of Mathematical Statistics, Vol. 22, Issue 3
  • DOI: 10.1214/aoms/1177729586

Some aspects of the sequential design of experiments
journal, January 1952


Derivative-free optimization: a review of algorithms and comparison of software implementations
journal, July 2012


Kernel-based methods for Bandit convex optimization
conference, January 2017

  • Bubeck, Sébastien; Lee, Yin Tat; Eldan, Ronen
  • Proceedings of the 49th Annual ACM SIGACT Symposium on Theory of Computing - STOC 2017
  • DOI: 10.1145/3055399.3055403

Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems
journal, January 2012

  • Bubeck, Sébastien
  • Foundations and Trends® in Machine Learning, Vol. 5, Issue 1
  • DOI: 10.1561/2200000024

A Smoothing Direct Search Method for Monte Carlo-Based Bound Constrained Composite Nonsmooth Optimization
journal, January 2018

  • Chen, Xiaojun; Kelley, C. T.; Xu, Fengmin
  • SIAM Journal on Scientific Computing, Vol. 40, Issue 4
  • DOI: 10.1137/17M1116714

Optimization with hidden constraints and embedded Monte Carlo computations
journal, December 2015


Stochastic optimization using a trust-region method and random models
journal, April 2017


Sobolev seminorm of quadratic functions with applications to derivative-free optimization
journal, May 2013


A Derivative-Free Algorithm for Least-Squares Minimization
journal, January 2010

  • Zhang, Hongchao; Conn, Andrew R.; Scheinberg, Katya
  • SIAM Journal on Optimization, Vol. 20, Issue 6
  • DOI: 10.1137/09075531X

On the local convergence of a derivative-free algorithm for least-squares minimization
journal, October 2010

  • Zhang, Hongchao; Conn, Andrew R.
  • Computational Optimization and Applications, Vol. 51, Issue 2
  • DOI: 10.1007/s10589-010-9367-x

Derivative free analogues of the Levenberg-Marquardt and Gauss algorithms for nonlinear least squares approximation
journal, August 1971

  • Brown, Kenneth M.; Dennis, J. E.
  • Numerische Mathematik, Vol. 18, Issue 4
  • DOI: 10.1007/BF01404679

Stochastic Trust-Region Response-Surface Method (STRONG)—A New Response-Surface Framework for Simulation Optimization
journal, May 2013

  • Chang, Kuo-Hao; Hong, L. Jeff; Wan, Hong
  • INFORMS Journal on Computing, Vol. 25, Issue 2
  • DOI: 10.1287/ijoc.1120.0498

Constrained optimization involving expensive function evaluations: A sequential approach
journal, January 2005

  • Brekelmans, Ruud; Driessen, Lonneke; Hamers, Herbert
  • European Journal of Operational Research, Vol. 160, Issue 1
  • DOI: 10.1016/j.ejor.2003.10.009

Optimization Methods for Large-Scale Machine Learning
journal, January 2018

  • Bottou, Léon; Curtis, Frank E.; Nocedal, Jorge
  • SIAM Review, Vol. 60, Issue 2
  • DOI: 10.1137/16M1080173

Worst case complexity of direct search
journal, December 2012


On the construction of quadratic models for derivative-free trust-region algorithms
journal, February 2017

  • Verdério, Adriano; Karas, Elizabeth W.; Pedroso, Lucas G.
  • EURO Journal on Computational Optimization, Vol. 5, Issue 4
  • DOI: 10.1007/s13675-017-0081-7

CONDOR, a new parallel, constrained extension of Powell's UOBYQA algorithm: Experimental results and comparison with the DFO algorithm
journal, September 2005

  • Vanden Berghen, Frank; Bersini, Hugues
  • Journal of Computational and Applied Mathematics, Vol. 181, Issue 1
  • DOI: 10.1016/j.cam.2004.11.029

Using QR Decomposition to Obtain a New Instance of Mesh Adaptive Direct Search with Uniformly Distributed Polling Directions
journal, June 2013

  • Van Dyke, Benjamin; Asaki, Thomas J.
  • Journal of Optimization Theory and Applications, Vol. 159, Issue 3
  • DOI: 10.1007/s10957-013-0356-y

Fortified-Descent Simplicial Search Method: A General Approach
journal, January 1999


A sequential quadratic programming algorithm for equality-constrained optimization without derivatives
journal, November 2014


Stochastic global optimization methods part II: Multi level methods
journal, September 1987

  • Rinnooy Kan, A. H. G.; Timmer, G. T.
  • Mathematical Programming, Vol. 39, Issue 1
  • DOI: 10.1007/BF02592071

On the Convergence of Pattern Search Algorithms
journal, February 1997


Stochastic global optimization methods part I: Clustering methods
journal, September 1987

  • Rinnooy Kan, A. H. G.; Timmer, G. T.
  • Mathematical Programming, Vol. 39, Issue 1
  • DOI: 10.1007/BF02592070

CONORBIT: constrained optimization by radial basis function interpolation in trust regions
journal, August 2016


A Stochastic Radial Basis Function Method for the Global Optimization of Expensive Functions
journal, November 2007

  • Regis, Rommel G.; Shoemaker, Christine A.
  • INFORMS Journal on Computing, Vol. 19, Issue 4
  • DOI: 10.1287/ijoc.1060.0182

On the properties of positive spanning sets and positive bases
journal, September 2015


The calculus of simplex gradients
journal, October 2014


Generalized Pattern Search methods for a class of nonsmooth optimization problems with structure
journal, July 2009

  • Bogani, C.; Gasparo, M. G.; Papini, A.
  • Journal of Computational and Applied Mathematics, Vol. 229, Issue 1
  • DOI: 10.1016/j.cam.2008.10.047

A stochastic method for global optimization
journal, December 1982

  • Boender, C. G. E.; Rinnooy Kan, A. H. G.; Timmer, G. T.
  • Mathematical Programming, Vol. 22, Issue 1
  • DOI: 10.1007/BF01581033

Multidimensional Stochastic Approximation Methods
journal, December 1954


Approximation Methods which Converge with Probability one
journal, June 1954


Derivative-Free Optimization of Expensive Functions with Computational Error Using Weighted Regression
journal, January 2013

  • Billups, Stephen C.; Larson, Jeffrey; Graf, Peter
  • SIAM Journal on Optimization, Vol. 23, Issue 1
  • DOI: 10.1137/100814688

On the Convergence of the Multidirectional Search Algorithm
journal, February 1991

  • Torczon, Virginia
  • SIAM Journal on Optimization, Vol. 1, Issue 1
  • DOI: 10.1137/0801010

Bayesian Guided Pattern Search for Robust Local Optimization
journal, November 2009

  • Taddy, Matthew A.; Lee, Herbert K. H.; Gray, Genetha A.
  • Technometrics, Vol. 51, Issue 4
  • DOI: 10.1198/TECH.2009.08007

Optimization of Convex Functions with Random Pursuit
journal, January 2013

  • Stich, S. U.; Müller, C. L.; Gärtner, B.
  • SIAM Journal on Optimization, Vol. 23, Issue 2
  • DOI: 10.1137/110853613

Pattern search ranking and selection algorithms for mixed variable simulation-based optimization
journal, November 2009

  • Sriver, Todd A.; Chrissis, James W.; Abramson, Mark A.
  • European Journal of Operational Research, Vol. 198, Issue 3
  • DOI: 10.1016/j.ejor.2008.10.020

Sequential Application of Simplex Designs in Optimisation and Evolutionary Operation
journal, November 1962


Multivariate stochastic approximation using a simultaneous perturbation gradient approximation
journal, March 1992

  • Spall, J. C.
  • IEEE Transactions on Automatic Control, Vol. 37, Issue 3
  • DOI: 10.1109/9.119632

An adaptive multiquadric radial basis function method for expensive black-box mixed-integer nonlinear constrained optimization
journal, February 2013


Dud, A Derivative-Free Algorithm for Nonlinear Least Squares
journal, February 1978


A Convergent Variant of the Nelder–Mead Algorithm
journal, April 2002

  • Price, C. J.; Coope, I. D.; Byatt, D.
  • Journal of Optimization Theory and Applications, Vol. 113, Issue 1
  • DOI: 10.1023/A:1014849028575

Exploiting problem structure in pattern search methods for unconstrained optimization
journal, June 2006