skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

This content will become publicly available on May 1, 2020

Title: Derivative-free optimization methods

Abstract

In many optimization problems arising from scientific, engineering and artificial intelligence applications, objective and constraint functions are available only as the output of a black-box or simulation oracle that does not provide derivative information. Such settings necessitate the use of methods for derivative-free, or zeroth-order, optimization. We provide a review and perspectives on developments in these methods, with an emphasis on highlighting recent developments and on unifying treatment of such problems in the non-linear optimization and machine learning literature. We categorize methods based on assumed properties of the black-box functions, as well as features of the methods. We first overview the primary setting of deterministic methods applied to unconstrained, non-convex optimization problems where the objective function is defined by a deterministic black-box oracle. We then discuss developments in randomized methods, methods that assume some additional structure about the objective (including convexity, separability and general non-smooth compositions), methods for problems where the output of the black-box oracle is stochastic, and methods for handling different types of constraints.

Authors:
 [1];  [1];  [1]
  1. Argonne National Lab. (ANL), Argonne, IL (United States)
Publication Date:
Research Org.:
Argonne National Lab. (ANL), Argonne, IL (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)
OSTI Identifier:
1545343
Grant/Contract Number:  
AC02-06CH11357
Resource Type:
Accepted Manuscript
Journal Name:
Acta Numerica
Additional Journal Information:
Journal Volume: 28; Journal Issue: 2010; Journal ID: ISSN 0962-4929
Publisher:
Cambridge University Press
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; Optimization

Citation Formats

Larson, Jeffrey, Menickelly, Matt, and Wild, Stefan M. Derivative-free optimization methods. United States: N. p., 2019. Web. doi:10.1017/S0962492919000060.
Larson, Jeffrey, Menickelly, Matt, & Wild, Stefan M. Derivative-free optimization methods. United States. doi:10.1017/S0962492919000060.
Larson, Jeffrey, Menickelly, Matt, and Wild, Stefan M. Wed . "Derivative-free optimization methods". United States. doi:10.1017/S0962492919000060.
@article{osti_1545343,
title = {Derivative-free optimization methods},
author = {Larson, Jeffrey and Menickelly, Matt and Wild, Stefan M.},
abstractNote = {In many optimization problems arising from scientific, engineering and artificial intelligence applications, objective and constraint functions are available only as the output of a black-box or simulation oracle that does not provide derivative information. Such settings necessitate the use of methods for derivative-free, or zeroth-order, optimization. We provide a review and perspectives on developments in these methods, with an emphasis on highlighting recent developments and on unifying treatment of such problems in the non-linear optimization and machine learning literature. We categorize methods based on assumed properties of the black-box functions, as well as features of the methods. We first overview the primary setting of deterministic methods applied to unconstrained, non-convex optimization problems where the objective function is defined by a deterministic black-box oracle. We then discuss developments in randomized methods, methods that assume some additional structure about the objective (including convexity, separability and general non-smooth compositions), methods for problems where the output of the black-box oracle is stochastic, and methods for handling different types of constraints.},
doi = {10.1017/S0962492919000060},
journal = {Acta Numerica},
number = 2010,
volume = 28,
place = {United States},
year = {2019},
month = {5}
}

Journal Article:
Free Publicly Available Full Text
This content will become publicly available on May 1, 2020
Publisher's Version of Record

Save / Share: