Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

On efficient Hessian computation using the edge pushing algorithm in Julia

Journal Article · · Optimization Methods and Software
 [1];  [2];  [3];  [3]
  1. Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
  2. Argonne National Lab. (ANL), Lemont, IL (United States)
  3. Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States)
+ Show Author Affiliations

Evaluating the Hessian matrix of second-order derivatives at a sequence of points can be costly when applying second-order methods for nonlinear optimization. In this work, we discuss our experiences implementing the recently proposed Edge Pushing (EP) method in Julia as an experimental replacement for the current colouring-based methods used by JuMP, an open-source algebraic modelling language. We propose an alternative data structure for sparse Hessians to avoid the use of hash tables and analyse the space and time complexity of EP method. In our benchmarks, we find that EP is very competitive in terms of both preprocessing time and Hessian evaluation time. As a result, we identify cases where EP closes the performance gap between JuMP's previous implementation and the implementation in AMPL, a commercial software package with similar functionality.

Research Organization:
Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States)
Sponsoring Organization:
USDOE National Nuclear Security Administration (NNSA)
Grant/Contract Number:
AC52-07NA27344
OSTI ID:
1474340
Report Number(s):
LLNL-JRNL--720198; 865490
Journal Information:
Optimization Methods and Software, Journal Name: Optimization Methods and Software Journal Issue: 4-6 Vol. 33; ISSN 1055-6788
Publisher:
Taylor & FrancisCopyright Statement
Country of Publication:
United States
Language:
English

References (13)

Partitioned variable metric updates for large structured optimization problems journal February 1982
Local convergence analysis for partitioned quasi-Newton updates journal October 1982
Capitalizing on live variables: new algorithms for efficient Hessian computation via automatic differentiation journal February 2016
A new framework for the computation of Hessians journal April 2012
New Acyclic and Star Coloring Algorithms with Application to Computing Hessians journal January 2007
Evaluating Derivatives book January 2008
Trust Region Methods book January 2000
JuMP: A Modeling Language for Mathematical Optimization journal January 2017
What Color Is Your Jacobian? Graph Coloring for Computing Derivatives journal January 2005
LIBSVM: A library for support vector machines journal April 2011
Computing the sparsity pattern of Hessians using automatic differentiation journal February 2014
ColPack: Software for graph coloring and related problems in scientific computing journal September 2013
Efficient Computation of Sparse Hessians Using Coloring and Automatic Differentiation journal May 2009

Similar Records

Edge Pushing is Equivalent to Vertex Elimination for Computing Hessians
Conference · Sat Oct 01 00:00:00 EDT 2016 · OSTI ID:1422527
Reducing measurement costs by recycling the Hessian in adaptive variational quantum algorithms
Journal Article · Sun Nov 17 23:00:00 EST 2024 · Quantum Science and Technology · OSTI ID:2477881

Related Subjects

97 MATHEMATICS AND COMPUTING
Automatic differentiation
JuMP
Julia
algebraic modelling language
edge pushing
sparse Hessian computation