Practical aspects of the Moreau-Yosida regularization II: Numerical considerations
This paper is a follow-up of C. Lemarechal and C.A. Sagastizabal, {open_quotes}An approach to variable metric bundle methods{close_quotes}, in Proceedings IFIP`93 Systems Modelling and Optimization Conference, Springer-Verlag. To minimize a convex function f, consider the usual cutting-plane approximation {breve f}k(y) := max {sub i}[f(y){sub i} + g{sub i}{sup T} (y - y{sub i})]. In a variable metric bundle algorithm, the next iterate y{sub k+1} is the proximal point of the current stability center x{sub k}, associated with the model {breve f}{sub k} and a metric M{sub k}. In other words y{sub k+1} = argmin{sub y}[{breve f}{sub k}(y) + {1/2} (y - x{sub k}){sup T} M{sub k} (y - x{sub k})]. Observing that the proximal mapping has an explicit inverse, the Moreau-Yosida regularization of f itself (call it F{sub M{sub k}}) can be used to update M{sub k} at each descent-step. Indeed, for all y and g {element_of} {partial_derivative} f(y), it holds g = {del}F{sub M{sub k}} (y + M{sub k}{sup -1}g). Thus, having two points y{sub k} and y{sub k+1}, a quasi-Newton formula can be applied to M{sub k}, using the pair of vectors u := y{sub k+1} + M{sub k}{sup -1} g{sub k+1} - y{sub k} - M{sub k}{sup -1} g{sub k} and v := g{sub k+1} - g{sub k}. Thanks to the special form of the above u = {Delta}y + M{sub k}{sup -1}v, a standard line-search makes the symmetric rank-one update positive definite. We prove convergence of the resulting bundle method. We do the same job with the diagonal update, for which we prove superlinear convergence in the {open_quotes}ideal{close_quotes} situation when {breve f}{sub k} can be replaced by f to compute y{sub k+1}. Some numerical illustrations are also given.
- OSTI ID:
- 36207
- Report Number(s):
- CONF-9408161--
- Country of Publication:
- United States
- Language:
- English
Similar Records
Approximations for generalized bilevel programming problem
Subdifferential monotonicity as characterization of convex functions