Differentiable McCormick relaxations
Abstract
McCormick's classical relaxation technique constructs closed-form convex and concave relaxations of compositions of simple intrinsic functions. These relaxations have several properties which make them useful for lower bounding problems in global optimization: they can be evaluated automatically, accurately, and computationally inexpensively, and they converge rapidly to the relaxed function as the underlying domain is reduced in size. They may also be adapted to yield relaxations of certain implicit functions and differential equation solutions. However, McCormick's relaxations may be nonsmooth, and this nonsmoothness can create theoretical and computational obstacles when relaxations are to be deployed. This article presents a continuously differentiable variant of McCormick's original relaxations in the multivariate McCormick framework of Tsoukalas and Mitsos. Gradients of the new differentiable relaxations may be computed efficiently using the standard forward or reverse modes of automatic differentiation. Furthermore, extensions to differentiable relaxations of implicit functions and solutions of parametric ordinary differential equations are discussed. A C++ implementation based on the library MC++ is described and applied to a case study in nonsmooth nonconvex optimization.
- Authors:
-
- Argonne National Lab. (ANL), Lemont, IL (United States)
- Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States)
- Publication Date:
- Research Org.:
- Argonne National Laboratory (ANL), Argonne, IL (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC)
- OSTI Identifier:
- 1373702
- Grant/Contract Number:
- AC02-06CH11357
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Journal of Global Optimization
- Additional Journal Information:
- Journal Volume: 67; Journal Issue: 4; Journal ID: ISSN 0925-5001
- Publisher:
- Springer
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 97 MATHEMATICS AND COMPUTING; Nonconvex optimization; Convex underestimators; McCormick relaxations; Implicit functions
Citation Formats
Khan, Kamil A., Watson, Harry A. J., and Barton, Paul I. Differentiable McCormick relaxations. United States: N. p., 2016.
Web. doi:10.1007/s10898-016-0440-6.
Khan, Kamil A., Watson, Harry A. J., & Barton, Paul I. Differentiable McCormick relaxations. United States. https://doi.org/10.1007/s10898-016-0440-6
Khan, Kamil A., Watson, Harry A. J., and Barton, Paul I. Fri .
"Differentiable McCormick relaxations". United States. https://doi.org/10.1007/s10898-016-0440-6. https://www.osti.gov/servlets/purl/1373702.
@article{osti_1373702,
title = {Differentiable McCormick relaxations},
author = {Khan, Kamil A. and Watson, Harry A. J. and Barton, Paul I.},
abstractNote = {McCormick's classical relaxation technique constructs closed-form convex and concave relaxations of compositions of simple intrinsic functions. These relaxations have several properties which make them useful for lower bounding problems in global optimization: they can be evaluated automatically, accurately, and computationally inexpensively, and they converge rapidly to the relaxed function as the underlying domain is reduced in size. They may also be adapted to yield relaxations of certain implicit functions and differential equation solutions. However, McCormick's relaxations may be nonsmooth, and this nonsmoothness can create theoretical and computational obstacles when relaxations are to be deployed. This article presents a continuously differentiable variant of McCormick's original relaxations in the multivariate McCormick framework of Tsoukalas and Mitsos. Gradients of the new differentiable relaxations may be computed efficiently using the standard forward or reverse modes of automatic differentiation. Furthermore, extensions to differentiable relaxations of implicit functions and solutions of parametric ordinary differential equations are discussed. A C++ implementation based on the library MC++ is described and applied to a case study in nonsmooth nonconvex optimization.},
doi = {10.1007/s10898-016-0440-6},
journal = {Journal of Global Optimization},
number = 4,
volume = 67,
place = {United States},
year = {Fri May 27 00:00:00 EDT 2016},
month = {Fri May 27 00:00:00 EDT 2016}
}
Web of Science
Works referenced in this record:
The cluster problem in multivariate global optimization
journal, October 1994
- Du, Kaisheng; Kearfott, R. Baker
- Journal of Global Optimization, Vol. 5, Issue 3
Convex and concave relaxations of implicit functions
journal, June 2014
- Stuber, Matthew D.; Scott, Joseph K.; Barton, Paul I.
- Optimization Methods and Software, Vol. 30, Issue 3
Convergence rate of McCormick relaxations
journal, February 2011
- Bompadre, Agustín; Mitsos, Alexander
- Journal of Global Optimization, Vol. 52, Issue 1
Global optimization of nonconvex NLPs and MINLPs with applications in process design
journal, May 1995
- Ryoo, H. S.; Sahinidis, N. V.
- Computers & Chemical Engineering, Vol. 19, Issue 5
The cluster problem revisited
journal, March 2013
- Wechsung, Achim; Schaber, Spencer D.; Barton, Paul I.
- Journal of Global Optimization, Vol. 58, Issue 3
McCormick-Based Relaxations of Algorithms
journal, January 2009
- Mitsos, Alexander; Chachuat, Benoît; Barton, Paul I.
- SIAM Journal on Optimization, Vol. 20, Issue 2
Computability of global solutions to factorable nonconvex programs: Part I — Convex underestimating problems
journal, December 1976
- McCormick, Garth P.
- Mathematical Programming, Vol. 10, Issue 1
Global optimization of mixed-integer nonlinear programs: A theoretical and computational study
journal, April 2004
- Tawarmalani, Mohit; Sahinidis, Nikolaos V.
- Mathematical Programming, Vol. 99, Issue 3
A global optimization method, αBB, for general twice-differentiable constrained NLPs — I. Theoretical advances
journal, August 1998
- Adjiman, C. S.; Dallwig, S.; Floudas, C. A.
- Computers & Chemical Engineering, Vol. 22, Issue 9
Nonlinear convex and concave relaxations for the solutions of parametric ODEs: CONVEX RELAXATIONS FOR PARAMETRIC ODES
journal, January 2012
- Scott, Joseph K.; Chachuat, Benoit; Barton, Paul I.
- Optimal Control Applications and Methods, Vol. 34, Issue 2
Ordinary Differential Equations
journal, January 2002
- Hartman, Philip
- Classics in Applied Mathematics
The global solver in the LINDO API
journal, October 2009
- Lin, Youdong; Schrage, Linus
- Optimization Methods and Software, Vol. 24, Issue 4-5
Reverse propagation of McCormick relaxations
journal, April 2015
- Wechsung, Achim; Scott, Joseph K.; Watson, Harry A. J.
- Journal of Global Optimization, Vol. 63, Issue 1
Multivariate McCormick relaxations
journal, April 2014
- Tsoukalas, A.; Mitsos, A.
- Journal of Global Optimization, Vol. 59, Issue 2-3
Multistream heat exchanger modeling and design
journal, August 2015
- Watson, Harry A. J.; Khan, Kamil A.; Barton, Paul I.
- AIChE Journal, Vol. 61, Issue 10
Theoretical rate of convergence for interval inclusion functions
journal, June 2011
- Scholz, Daniel
- Journal of Global Optimization, Vol. 53, Issue 4
Numerical methods and software for sensitivity analysis of differential-algebraic systems
journal, February 1996
- Maly, Timothy; Petzold, Linda R.
- Applied Numerical Mathematics, Vol. 20, Issue 1-2
Smoothing Functions and Smoothing Newton Method for Complementarity and Variational Inequality Problems
journal, April 2002
- Qi, L.; Sun, D.
- Journal of Optimization Theory and Applications, Vol. 113, Issue 1
A simple characterization of solution sets of convex programs
journal, February 1988
- Mangasarian, O. L.
- Operations Research Letters, Vol. 7, Issue 1
Interval analysis: theory and applications
journal, September 2000
- Alefeld, Götz; Mayer, Günter
- Journal of Computational and Applied Mathematics, Vol. 121, Issue 1-2
Analytic extensions of differentiable functions defined in closed sets
journal, January 1934
- Whitney, Hassler
- Transactions of the American Mathematical Society, Vol. 36, Issue 1
SNOPT: An SQP Algorithm for Large-Scale Constrained Optimization
journal, January 2005
- Gill, Philip E.; Murray, Walter; Saunders, Michael A.
- SIAM Review, Vol. 47, Issue 1
Outer approximation algorithms for separable nonconvex mixed-integer nonlinear programs
journal, May 2004
- Kesavan, Padmanaban; Allgor, Russell J.; Gatzke, Edward P.
- Mathematical Programming, Vol. 100, Issue 3
Nonconvex Generalized Benders Decomposition for Stochastic Separable Mixed-Integer Nonlinear Programs
journal, July 2011
- Li, Xiang; Tomasgard, Asgeir; Barton, Paul I.
- Journal of Optimization Theory and Applications, Vol. 151, Issue 3
Synthesis of heat exchanger networks at subambient conditions with compression and expansion of process streams
journal, September 2010
- Wechsung, Achim; Aspelund, Audun; Gundersen, Truls
- AIChE Journal, Vol. 57, Issue 8
A rigorous disjunctive optimization model for simultaneous flowsheet optimization and heat integration
journal, March 1998
- Grossmann, Ignacio E.; Yeomans, Héctor; Kravanja, Zdravko
- Computers & Chemical Engineering, Vol. 22
Convex and Concave Relaxations for the Parametric Solutions of Semi-explicit Index-One Differential-Algebraic Equations
journal, August 2012
- Scott, Joseph K.; Barton, Paul I.
- Journal of Optimization Theory and Applications, Vol. 156, Issue 3
On the smoothness of convex envelopes
journal, February 1990
- Griewank, A.; Rabier, P. J.
- Transactions of the American Mathematical Society, Vol. 322, Issue 2
Efficient sensitivity analysis of large-scale differential-algebraic systems
journal, October 1997
- Feehery, William F.; Tolsma, John E.; Barton, Paul I.
- Applied Numerical Mathematics, Vol. 25, Issue 1
Improved relaxations for the parametric solutions of ODEs using differential inequalities
journal, May 2012
- Scott, Joseph K.; Barton, Paul I.
- Journal of Global Optimization, Vol. 57, Issue 1
SCIP: solving constraint integer programs
journal, January 2009
- Achterberg, Tobias
- Mathematical Programming Computation, Vol. 1, Issue 1
Convergence analysis of multivariate McCormick relaxations
journal, March 2016
- Najman, Jaromił; Mitsos, Alexander
- Journal of Global Optimization, Vol. 66, Issue 4
On the Local and Superlinear Convergence of Quasi-Newton Methods
journal, January 1973
- Broyden, C. G.; Dennis, J. E.; MorÉ, Jorge J.
- IMA Journal of Applied Mathematics, Vol. 12, Issue 3
ANTIGONE: Algorithms for coNTinuous / Integer Global Optimization of Nonlinear Equations
journal, March 2014
- Misener, Ruth; Floudas, Christodoulos A.
- Journal of Global Optimization, Vol. 59, Issue 2-3
Generalized McCormick relaxations
journal, February 2011
- Scott, Joseph K.; Stuber, Matthew D.; Barton, Paul I.
- Journal of Global Optimization, Vol. 51, Issue 4
Works referencing / citing this record:
Convergence-order analysis of branch-and-bound algorithms for constrained problems
journal, June 2017
- Kannan, Rohit; Barton, Paul I.
- Journal of Global Optimization, Vol. 71, Issue 4
Deterministic Global Optimization with Artificial Neural Networks Embedded
journal, October 2018
- Schweidtmann, Artur M.; Mitsos, Alexander
- Journal of Optimization Theory and Applications, Vol. 180, Issue 3
Computationally relevant generalized derivatives: theory, evaluation and applications
journal, September 2017
- Barton, Paul I.; Khan, Kamil A.; Stechlinski, Peter
- Optimization Methods and Software, Vol. 33, Issue 4-6
Convergence of Subtangent-Based Relaxations of Nonlinear Programs
journal, April 2019
- Cao, Huiyi; Song, Yingkai; Khan, Kamil A.
- Processes, Vol. 7, Issue 4