A memory-distributed quasi-Newton solver for nonlinear programming problems with a small number of general constraints

Petra, Cosmin G.

doi:10.1016/j.jpdc.2018.10.009

Title: A memory-distributed quasi-Newton solver for nonlinear programming problems with a small number of general constraints

Full Record
Other Related Research

Abstract

We approach the problem of parallelizing state-of-the-art nonlinear programming optimization algorithms.Specifically, we focus on parallelizing quasi-Newton interior-point methods that use limited-memory secant Hessian approximations. Such interior-point methods are known to have better convergence properties and to be more effective on large-scale problems than gradient-based and derivative-free optimization algorithms. We target nonlinear and potentially nonconvex optimization problems with an arbitrary number of bound constraints and a small number of general equality and inequality constraints on the optimization variables. These problems occur for example in the form of optimal control, optimal design, and inverse problems governed by ordinary or partial differential equations, whenever they are expressed in a “reduced-space” optimization approach. We introduce and analyze the time and space complexity of a decomposition method for solving the quasi-Newton linear systems that leverages the fact that the quasi-Newton Hessian matrix has a small number of dense blocks that border a low-rank update of a diagonal matrix. This enables an efficient parallelization on memory-distributed computers of the iterations of the optimization algorithm, a state-of-the-art filter line-search interior-point algorithm by Wächter et. al. We illustrate the efficiency of the proposed method by solving structural topology optimization problems on up to 4608 cores on a parallelmore » machine.« less

Authors:

^[1]

Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)

Publication Date:: Thu Nov 28 00:00:00 EST 2019

Research Org.:: Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States)

Sponsoring Org.:: USDOE National Nuclear Security Administration (NNSA)

OSTI Identifier:: 1562383

Alternate Identifier(s):: OSTI ID: 1636866

Report Number(s):: LLNL-JRNL-739001
Journal ID: ISSN 0743-7315; 892535

Grant/Contract Number:: AC52-07NA27344; 16-ERD-025; 17-SI-005

Resource Type:: Accepted Manuscript

Journal Name:: Journal of Parallel and Distributed Computing

Additional Journal Information:: Journal Volume: 133; Journal Issue: C; Journal ID: ISSN 0743-7315

Publisher:: Elsevier

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING; Parallel optimization; Parallel interior-point; Quasi-Newton

Citation Formats


                    Petra, Cosmin G. A memory-distributed quasi-Newton solver for nonlinear programming problems with a small number of general constraints.  United States: N. p., 2019. 
Web.  doi:10.1016/j.jpdc.2018.10.009.

Copy to clipboard


                    Petra, Cosmin G. A memory-distributed quasi-Newton solver for nonlinear programming problems with a small number of general constraints.  United States.  https://doi.org/10.1016/j.jpdc.2018.10.009

Copy to clipboard


                    Petra, Cosmin G. Thu .  
"A memory-distributed quasi-Newton solver for nonlinear programming problems with a small number of general constraints".  United States.  https://doi.org/10.1016/j.jpdc.2018.10.009.  https://www.osti.gov/servlets/purl/1562383.

Copy to clipboard


                    
@article{osti_1562383,

  title        = {A memory-distributed quasi-Newton solver for nonlinear programming problems with a small number of general constraints},

  author       = {Petra, Cosmin G.},

  abstractNote = {We approach the problem of parallelizing state-of-the-art nonlinear programming optimization algorithms.Specifically, we focus on parallelizing quasi-Newton interior-point methods that use limited-memory secant Hessian approximations. Such interior-point methods are known to have better convergence properties and to be more effective on large-scale problems than gradient-based and derivative-free optimization algorithms. We target nonlinear and potentially nonconvex optimization problems with an arbitrary number of bound constraints and a small number of general equality and inequality constraints on the optimization variables. These problems occur for example in the form of optimal control, optimal design, and inverse problems governed by ordinary or partial differential equations, whenever they are expressed in a “reduced-space” optimization approach. We introduce and analyze the time and space complexity of a decomposition method for solving the quasi-Newton linear systems that leverages the fact that the quasi-Newton Hessian matrix has a small number of dense blocks that border a low-rank update of a diagonal matrix. This enables an efficient parallelization on memory-distributed computers of the iterations of the optimization algorithm, a state-of-the-art filter line-search interior-point algorithm by Wächter et. al. We illustrate the efficiency of the proposed method by solving structural topology optimization problems on up to 4608 cores on a parallel machine.},

  doi          = {10.1016/j.jpdc.2018.10.009},

  journal      = {Journal of Parallel and Distributed Computing},

  number       = C,

  volume       = 133,

  place        = {United States},

  year         = {Thu Nov 28 00:00:00 EST 2019},

  month        = {Thu Nov 28 00:00:00 EST 2019}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (Publisher)

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1016/j.jpdc.2018.10.009

Other availability

Search WorldCat to find libraries that may hold this journal

Citation Metrics:

Cited by: 11 works

Citation information provided by
Web of Science

Save / Share:

Export Metadata

Save to My Library

Similar Records in DOE PAGES and OSTI.GOV collections:

On the Derivation of Quasi-Newton Formulas for Optimization in Function Spaces

Journal Article Vuchkov, Radoslav G. ; Petra, Cosmin G. ; Petra, Noémi - Numerical Functional Analysis and Optimization

Newton’s method is usually preferred when solving optimization problems due to its superior convergence properties compared to gradient-based or derivative-free optimization algorithms. However, deriving and computing second-order derivatives needed by Newton’s method often is not trivial and, in some cases, not possible. In such cases quasi-Newton algorithms are a great alternative. In this paper, we provide a new derivation of well-known quasi-Newton formulas in an infinite-dimensional Hilbert space setting. Furthermore, it is known that quasi-Newton update formulas are solutions to certain variational problems over the space of symmetric matrices. In this paper, we formulate similar variational problems over the spacemore »« less
https://doi.org/10.1080/01630563.2020.1785496

Full Text Available
A study on Newton-related nonlinear methods in well test analysis, production schedule optimization and reservoir simulation

Miscellaneous Barua, J

This is a study on the use of alternative nonlinear methods in automated well test analysis, production and injection schedule optimization and in reservoir simulation. In automated well test analysis the advantages and disadvantages of second-order partial derivatives are investigated. Newton's method is shown to be prone to difficulties, however by adjusting the eigenvalues of the Hessian matrix, the performance can be substantially improved. In optimizing the cyclic steam injection process, Newton's method is compared with the Quasi-Newton method using a simplified model to simulate the process. The Quasi-Newton method does significantly better than Newton's method in saving function evaluations.more »« less
A study on Newton related nonlinear methods in well test analysis, production schedule optimization and reservoir simulation

Technical Report Barua, J

This is a study on the use of alternative nonlinear methods in automated well test analysis, production and injection schedule optimization and in reservoir simulation. In automated well test analysis the advantages and disadvantages of second-order partial derivatives are investigated. Newton's method is shown to be prone to difficulties, however by adjusting the eigenvalues of the Hessian matrix, the performance can be substantially improved. In optimizing the cyclic steam injection process, Newton's method is compared with the Quasi-Newton method using a simplified model to simulate the process. Specific operating strategies for the process are identified. The two methods are thenmore »« less
A STRUCTURED QUASI-NEWTON ALGORITHM FOR OPTIMIZING WITH INCOMPLETE HESSIAN INFORMATION

Journal Article Petra, Cosmin G. ; Chiang, Nai-Yuan ; Anitescu, Mihai - SIAM Journal on Optimization

We present a structured quasi-Newton algorithm for unconstrained optimization problems that have unavailable second-order derivatives or Hessian terms. We provide a formal derivation of the well-known Broyden-Fletcher-Goldfarb-Shanno (BFGS) secant update formula that approximates only the missing Hessian terms, and we propose a linesearch quasi-Newton algorithm based on a modification of Wolfe conditions that converges to first-order optimality conditions. We also analyze the local convergence properties of the structured BFGS algorithm and show that it achieves superlinear convergence under the standard assumptions used by quasi-Newton methods using secant updates. We provide a thorough study of the practical performance of the algorithmmore »« less
https://doi.org/10.1137/18M1167942
A Structured Quasi-Newton Algorithm for Optimizing with Incomplete Hessian Information

Journal Article Petra, Cosmin G. ; Chiang, Naiyuan ; Anitescu, Mihai - SIAM Journal on Optimization

We present a structured quasi-Newton algorithm for unconstrained optimization problems that have unavailable second-order derivatives or Hessian terms. We provide a formal derivation of the well-known Broyden--Fletcher--Goldfarb--Shanno (BFGS) secant update formula that approximates only the missing Hessian terms, and we propose a linesearch quasi-Newton algorithm based on a modification of Wolfe conditions that converges to first-order optimality conditions. We also analyze the local convergence properties of the structured BFGS algorithm and show that it achieves superlinear convergence under the standard assumptions used by quasi-Newton methods using secant updates. In conclusion, we provide a thorough study of the practical performance ofmore »« less
Cited by 5
https://doi.org/10.1137/18M1167942

Full Text Available

Similar Records