Differentiable McCormick relaxations

Khan, Kamil A.; Watson, Harry A. J.; Barton, Paul I.

doi:10.1007/s10898-016-0440-6

Differentiable McCormick relaxations

Journal Article · Fri May 27 00:00:00 EDT 2016 · Journal of Global Optimization

DOI:https://doi.org/10.1007/s10898-016-0440-6· OSTI ID:1373702

Khan, Kamil A. ^[1]; Watson, Harry A. J. ^[2]; Barton, Paul I. ^[2]

Argonne National Lab. (ANL), Lemont, IL (United States)
Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States)

McCormick's classical relaxation technique constructs closed-form convex and concave relaxations of compositions of simple intrinsic functions. These relaxations have several properties which make them useful for lower bounding problems in global optimization: they can be evaluated automatically, accurately, and computationally inexpensively, and they converge rapidly to the relaxed function as the underlying domain is reduced in size. They may also be adapted to yield relaxations of certain implicit functions and differential equation solutions. However, McCormick's relaxations may be nonsmooth, and this nonsmoothness can create theoretical and computational obstacles when relaxations are to be deployed. This article presents a continuously differentiable variant of McCormick's original relaxations in the multivariate McCormick framework of Tsoukalas and Mitsos. Gradients of the new differentiable relaxations may be computed efficiently using the standard forward or reverse modes of automatic differentiation. Furthermore, extensions to differentiable relaxations of implicit functions and solutions of parametric ordinary differential equations are discussed. A C++ implementation based on the library MC++ is described and applied to a case study in nonsmooth nonconvex optimization.

View Accepted Manuscript (DOE)

Research Organization:: Argonne National Lab. (ANL), Argonne, IL (United States)

Sponsoring Organization:: USDOE Office of Science (SC)

Grant/Contract Number:: AC02-06CH11357

OSTI ID:: 1373702

Journal Information:: Journal of Global Optimization, Journal Name: Journal of Global Optimization Journal Issue: 4 Vol. 67; ISSN 0925-5001

Publisher:: SpringerCopyright Statement

Country of Publication:: United States

Language:: English

References (43)

Synthesis of heat exchanger networks at subambient conditions with compression and expansion of process streams Wechsung, Achim; Aspelund, Audun; Gundersen, Truls AIChE Journal, Vol. 57, Issue 8 https://doi.org/10.1002/aic.12412	journal	September 2010
Multistream heat exchanger modeling and design Watson, Harry A. J.; Khan, Kamil A.; Barton, Paul I. AIChE Journal, Vol. 61, Issue 10 https://doi.org/10.1002/aic.14965	journal	August 2015
Nonlinear convex and concave relaxations for the solutions of parametric ODEs: CONVEX RELAXATIONS FOR PARAMETRIC ODES Scott, Joseph K.; Chachuat, Benoit; Barton, Paul I. Optimal Control Applications and Methods, Vol. 34, Issue 2 https://doi.org/10.1002/oca.2014	journal	January 2012
Convexification and Global Optimization in Continuous and Mixed-Integer Nonlinear Programming Tawarmalani, Mohit; Sahinidis, Nikolaos V. Nonconvex Optimization and Its Applications https://doi.org/10.1007/978-1-4757-3532-1	book	January 2002
Constraint Integer Programming: A New Approach to Integrate CP and MIP Achterberg, Tobias; Berthold, Timo; Koch, Thorsten Integration of AI and OR Techniques in Constraint Programming for Combinatorial Optimization Problems https://doi.org/10.1007/978-3-540-68155-7_4	book	January 2008
Adjoint Mode Computation of Subgradients for McCormick Relaxations Beckers, Markus; Mosenkis, Viktor; Naumann, Uwe Lecture Notes in Computational Science and Engineering https://doi.org/10.1007/978-3-642-30023-3_10	book	January 2012
BARON: A general purpose global optimization software package Sahinidis, Nikolaos V. Journal of Global Optimization, Vol. 8, Issue 2 https://doi.org/10.1007/BF00138693	journal	March 1996
The cluster problem in multivariate global optimization Du, Kaisheng; Kearfott, R. Baker Journal of Global Optimization, Vol. 5, Issue 3 https://doi.org/10.1007/BF01096455	journal	October 1994
Computability of global solutions to factorable nonconvex programs: Part I — Convex underestimating problems McCormick, Garth P. Mathematical Programming, Vol. 10, Issue 1 https://doi.org/10.1007/BF01580665	journal	December 1976
Global optimization of mixed-integer nonlinear programs: A theoretical and computational study Tawarmalani, Mohit; Sahinidis, Nikolaos V. Mathematical Programming, Vol. 99, Issue 3 https://doi.org/10.1007/s10107-003-0467-6	journal	April 2004
Outer approximation algorithms for separable nonconvex mixed-integer nonlinear programs Kesavan, Padmanaban; Allgor, Russell J.; Gatzke, Edward P. Mathematical Programming, Vol. 100, Issue 3 https://doi.org/10.1007/s10107-004-0503-1	journal	May 2004
Generalized McCormick relaxations Scott, Joseph K.; Stuber, Matthew D.; Barton, Paul I. Journal of Global Optimization, Vol. 51, Issue 4 https://doi.org/10.1007/s10898-011-9664-7	journal	February 2011
Convergence rate of McCormick relaxations Bompadre, Agustín; Mitsos, Alexander Journal of Global Optimization, Vol. 52, Issue 1 https://doi.org/10.1007/s10898-011-9685-2	journal	February 2011
Theoretical rate of convergence for interval inclusion functions Scholz, Daniel Journal of Global Optimization, Vol. 53, Issue 4 https://doi.org/10.1007/s10898-011-9735-9	journal	June 2011
Improved relaxations for the parametric solutions of ODEs using differential inequalities Scott, Joseph K.; Barton, Paul I. Journal of Global Optimization, Vol. 57, Issue 1 https://doi.org/10.1007/s10898-012-9909-0	journal	May 2012
The cluster problem revisited Wechsung, Achim; Schaber, Spencer D.; Barton, Paul I. Journal of Global Optimization, Vol. 58, Issue 3 https://doi.org/10.1007/s10898-013-0059-9	journal	March 2013
ANTIGONE: Algorithms for coNTinuous / Integer Global Optimization of Nonlinear Equations Misener, Ruth; Floudas, Christodoulos A. Journal of Global Optimization, Vol. 59, Issue 2-3 https://doi.org/10.1007/s10898-014-0166-2	journal	March 2014
Multivariate McCormick relaxations Tsoukalas, A.; Mitsos, A. Journal of Global Optimization, Vol. 59, Issue 2-3 https://doi.org/10.1007/s10898-014-0176-0	journal	April 2014
Reverse propagation of McCormick relaxations Wechsung, Achim; Scott, Joseph K.; Watson, Harry A. J. Journal of Global Optimization, Vol. 63, Issue 1 https://doi.org/10.1007/s10898-015-0303-6	journal	April 2015
Convergence analysis of multivariate McCormick relaxations Najman, Jaromił; Mitsos, Alexander Journal of Global Optimization, Vol. 66, Issue 4 https://doi.org/10.1007/s10898-016-0408-6	journal	March 2016
Nonconvex Generalized Benders Decomposition for Stochastic Separable Mixed-Integer Nonlinear Programs Li, Xiang; Tomasgard, Asgeir; Barton, Paul I. Journal of Optimization Theory and Applications, Vol. 151, Issue 3 https://doi.org/10.1007/s10957-011-9888-1	journal	July 2011
Convex and Concave Relaxations for the Parametric Solutions of Semi-explicit Index-One Differential-Algebraic Equations Scott, Joseph K.; Barton, Paul I. Journal of Optimization Theory and Applications, Vol. 156, Issue 3 https://doi.org/10.1007/s10957-012-0149-8	journal	August 2012
SCIP: solving constraint integer programs Achterberg, Tobias Mathematical Programming Computation, Vol. 1, Issue 1 https://doi.org/10.1007/s12532-008-0001-1	journal	January 2009
Global optimization of nonconvex NLPs and MINLPs with applications in process design Ryoo, H. S.; Sahinidis, N. V. Computers & Chemical Engineering, Vol. 19, Issue 5 https://doi.org/10.1016/0098-1354(94)00097-2	journal	May 1995
A simple characterization of solution sets of convex programs Mangasarian, O. L. Operations Research Letters, Vol. 7, Issue 1 https://doi.org/10.1016/0167-6377(88)90047-8	journal	February 1988
Numerical methods and software for sensitivity analysis of differential-algebraic systems Maly, Timothy; Petzold, Linda R. Applied Numerical Mathematics, Vol. 20, Issue 1-2 https://doi.org/10.1016/0168-9274(95)00117-4	journal	February 1996
A global optimization method, αBB, for general twice-differentiable constrained NLPs — I. Theoretical advances Adjiman, C. S.; Dallwig, S.; Floudas, C. A. Computers & Chemical Engineering, Vol. 22, Issue 9 https://doi.org/10.1016/S0098-1354(98)00027-1	journal	August 1998
A rigorous disjunctive optimization model for simultaneous flowsheet optimization and heat integration Grossmann, Ignacio E.; Yeomans, Héctor; Kravanja, Zdravko Computers & Chemical Engineering, Vol. 22 https://doi.org/10.1016/S0098-1354(98)00050-7	journal	March 1998
Efficient sensitivity analysis of large-scale differential-algebraic systems Feehery, William F.; Tolsma, John E.; Barton, Paul I. Applied Numerical Mathematics, Vol. 25, Issue 1 https://doi.org/10.1016/S0168-9274(97)00050-0	journal	October 1997
Interval analysis: theory and applications Alefeld, Götz; Mayer, Günter Journal of Computational and Applied Mathematics, Vol. 121, Issue 1-2 https://doi.org/10.1016/S0377-0427(00)00342-3	journal	September 2000
Convex Optimization Boyd, Stephen; Vandenberghe, Lieven https://doi.org/10.1017/CBO9780511804441	book	January 2004
Smoothing Functions and Smoothing Newton Method for Complementarity and Variational Inequality Problems Qi, L.; Sun, D. Journal of Optimization Theory and Applications, Vol. 113, Issue 1 https://doi.org/10.1023/A:1014861331301	journal	April 2002
The global solver in the LINDO API Lin, Youdong; Schrage, Linus Optimization Methods and Software, Vol. 24, Issue 4-5 https://doi.org/10.1080/10556780902753221	journal	October 2009
Convex and concave relaxations of implicit functions Stuber, Matthew D.; Scott, Joseph K.; Barton, Paul I. Optimization Methods and Software, Vol. 30, Issue 3 https://doi.org/10.1080/10556788.2014.924514	journal	June 2014
Analytic extensions of differentiable functions defined in closed sets Whitney, Hassler Transactions of the American Mathematical Society, Vol. 36, Issue 1 https://doi.org/10.1090/S0002-9947-1934-1501735-3	journal	January 1934
On the smoothness of convex envelopes Griewank, A.; Rabier, P. J. Transactions of the American Mathematical Society, Vol. 322, Issue 2 https://doi.org/10.1090/S0002-9947-1990-0986024-2	journal	February 1990
On the Local and Superlinear Convergence of Quasi-Newton Methods Broyden, C. G.; Dennis, J. E.; MorÉ, Jorge J. IMA Journal of Applied Mathematics, Vol. 12, Issue 3 https://doi.org/10.1093/imamat/12.3.223	journal	January 1973
McCormick-Based Relaxations of Algorithms Mitsos, Alexander; Chachuat, Benoît; Barton, Paul I. SIAM Journal on Optimization, Vol. 20, Issue 2 https://doi.org/10.1137/080717341	journal	January 2009
Evaluating Derivatives Griewank, Andreas; Walther, Andrea https://doi.org/10.1137/1.9780898717761	book	January 2008
Ordinary Differential Equations Hartman, Philip Classics in Applied Mathematics https://doi.org/10.1137/1.9780898719222	journal	January 2002
Iterative Solution of Nonlinear Equations in Several Variables Ortega, J. M.; Rheinboldt, W. C. Classics in Applied Mathematics https://doi.org/10.1137/1.9780898719468	book	January 2000
Optimization and Nonsmooth Analysis Clarke, Frank H. Classics in Applied Mathematics https://doi.org/10.1137/1.9781611971309	book	January 1990
SNOPT: An SQP Algorithm for Large-Scale Constrained Optimization Gill, Philip E.; Murray, Walter; Saunders, Michael A. SIAM Review, Vol. 47, Issue 1 https://doi.org/10.1137/S0036144504446096	journal	January 2005

Cited By (4)

Convergence-order analysis of branch-and-bound algorithms for constrained problems Kannan, Rohit; Barton, Paul I. Journal of Global Optimization, Vol. 71, Issue 4 https://doi.org/10.1007/s10898-017-0532-y	journal	June 2017
Deterministic Global Optimization with Artificial Neural Networks Embedded Schweidtmann, Artur M.; Mitsos, Alexander Journal of Optimization Theory and Applications, Vol. 180, Issue 3 https://doi.org/10.1007/s10957-018-1396-0	journal	October 2018
Computationally relevant generalized derivatives: theory, evaluation and applications Barton, Paul I.; Khan, Kamil A.; Stechlinski, Peter Optimization Methods and Software, Vol. 33, Issue 4-6 https://doi.org/10.1080/10556788.2017.1374385	journal	September 2017
Convergence of Subtangent-Based Relaxations of Nonlinear Programs Cao, Huiyi; Song, Yingkai; Khan, Kamil A. Processes, Vol. 7, Issue 4 https://doi.org/10.3390/pr7040221	journal	April 2019

Similar Records

McCormick envelopes in mixed-integer PDE-constrained optimization

Journal Article · Sun Jan 05 23:00:00 EST 2025 · Mathematical Programming · OSTI ID:2496737

Bounding the solutions of parametric weakly coupled second-order semilinear parabolic partial differential equations

Journal Article · Wed Sep 21 00:00:00 EDT 2016 · Optimal Control Applications and Methods · OSTI ID:1425976

Asynchronous relaxations for the numerical solution of differential equations by parallel processors

Journal Article · Wed Dec 31 23:00:00 EST 1986 · SIAM J. Sci. Stat. Comput.; (United States) · OSTI ID:6394813

Related Subjects

97 MATHEMATICS AND COMPUTING
Convex underestimators
Implicit functions
McCormick relaxations
Nonconvex optimization

Differentiable McCormick relaxations

Citation Formats

References (43)

Cited By (4)

Similar Records

Related Subjects