skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: On efficient Hessian computation using the edge pushing algorithm in Julia

Abstract

Evaluating the Hessian matrix of second-order derivatives at a sequence of points can be costly when applying second-order methods for nonlinear optimization. In this work, we discuss our experiences implementing the recently proposed Edge Pushing (EP) method in Julia as an experimental replacement for the current colouring-based methods used by JuMP, an open-source algebraic modelling language. We propose an alternative data structure for sparse Hessians to avoid the use of hash tables and analyse the space and time complexity of EP method. In our benchmarks, we find that EP is very competitive in terms of both preprocessing time and Hessian evaluation time. As a result, we identify cases where EP closes the performance gap between JuMP's previous implementation and the implementation in AMPL, a commercial software package with similar functionality.

Authors:
 [1];  [2];  [3];  [3]
  1. Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
  2. Argonne National Lab. (ANL), Lemont, IL (United States)
  3. Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States)
Publication Date:
Research Org.:
Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
Sponsoring Org.:
USDOE National Nuclear Security Administration (NNSA)
OSTI Identifier:
1474340
Report Number(s):
LLNL-JRNL-720198
Journal ID: ISSN 1055-6788; 865490
Grant/Contract Number:  
AC52-07NA27344
Resource Type:
Accepted Manuscript
Journal Name:
Optimization Methods and Software
Additional Journal Information:
Journal Volume: 33; Journal Issue: 4-6; Journal ID: ISSN 1055-6788
Publisher:
Taylor & Francis
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; Automatic differentiation; sparse Hessian computation; edge pushing; algebraic modelling language; Julia; JuMP

Citation Formats

Petra, C. G., Qiang, F., Lubin, M., and Huchette, J. On efficient Hessian computation using the edge pushing algorithm in Julia. United States: N. p., 2018. Web. doi:10.1080/10556788.2018.1480625.
Petra, C. G., Qiang, F., Lubin, M., & Huchette, J. On efficient Hessian computation using the edge pushing algorithm in Julia. United States. doi:https://doi.org/10.1080/10556788.2018.1480625
Petra, C. G., Qiang, F., Lubin, M., and Huchette, J. Wed . "On efficient Hessian computation using the edge pushing algorithm in Julia". United States. doi:https://doi.org/10.1080/10556788.2018.1480625. https://www.osti.gov/servlets/purl/1474340.
@article{osti_1474340,
title = {On efficient Hessian computation using the edge pushing algorithm in Julia},
author = {Petra, C. G. and Qiang, F. and Lubin, M. and Huchette, J.},
abstractNote = {Evaluating the Hessian matrix of second-order derivatives at a sequence of points can be costly when applying second-order methods for nonlinear optimization. In this work, we discuss our experiences implementing the recently proposed Edge Pushing (EP) method in Julia as an experimental replacement for the current colouring-based methods used by JuMP, an open-source algebraic modelling language. We propose an alternative data structure for sparse Hessians to avoid the use of hash tables and analyse the space and time complexity of EP method. In our benchmarks, we find that EP is very competitive in terms of both preprocessing time and Hessian evaluation time. As a result, we identify cases where EP closes the performance gap between JuMP's previous implementation and the implementation in AMPL, a commercial software package with similar functionality.},
doi = {10.1080/10556788.2018.1480625},
journal = {Optimization Methods and Software},
number = 4-6,
volume = 33,
place = {United States},
year = {2018},
month = {9}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Save / Share:

Works referenced in this record:

ColPack: Software for graph coloring and related problems in scientific computing
journal, September 2013

  • Gebremedhin, Assefaw H.; Nguyen, Duc; Patwary, Md. Mostofa Ali
  • ACM Transactions on Mathematical Software, Vol. 40, Issue 1
  • DOI: 10.1145/2513109.2513110

Evaluating Derivatives
book, January 2008


Efficient Computation of Sparse Hessians Using Coloring and Automatic Differentiation
journal, May 2009

  • Gebremedhin, Assefaw H.; Tarafdar, Arijit; Pothen, Alex
  • INFORMS Journal on Computing, Vol. 21, Issue 2
  • DOI: 10.1287/ijoc.1080.0286

LIBSVM: A library for support vector machines
journal, April 2011

  • Chang, Chih-Chung; Lin, Chih-Jen
  • ACM Transactions on Intelligent Systems and Technology, Vol. 2, Issue 3
  • DOI: 10.1145/1961189.1961199

Partitioned variable metric updates for large structured optimization problems
journal, February 1982

  • Griewank, A.; Toint, Ph. L.
  • Numerische Mathematik, Vol. 39, Issue 1
  • DOI: 10.1007/BF01399316

JuMP: A Modeling Language for Mathematical Optimization
journal, January 2017

  • Dunning, Iain; Huchette, Joey; Lubin, Miles
  • SIAM Review, Vol. 59, Issue 2
  • DOI: 10.1137/15M1020575

Computing the sparsity pattern of Hessians using automatic differentiation
journal, February 2014

  • Gower, Robert Mansel; Mello, Margarida Pinheiro
  • ACM Transactions on Mathematical Software, Vol. 40, Issue 2
  • DOI: 10.1145/2490254

What Color Is Your Jacobian? Graph Coloring for Computing Derivatives
journal, January 2005


A new framework for the computation of Hessians
journal, April 2012


Local convergence analysis for partitioned quasi-Newton updates
journal, October 1982

  • Griewank, A.; Toint, Ph. L.
  • Numerische Mathematik, Vol. 39, Issue 3
  • DOI: 10.1007/BF01407874

New Acyclic and Star Coloring Algorithms with Application to Computing Hessians
journal, January 2007

  • Gebremedhin, Assefaw H.; Tarafdar, Arijit; Manne, Fredrik
  • SIAM Journal on Scientific Computing, Vol. 29, Issue 3
  • DOI: 10.1137/050639879

Capitalizing on live variables: new algorithms for efficient Hessian computation via automatic differentiation
journal, February 2016

  • Wang, Mu; Gebremedhin, Assefaw; Pothen, Alex
  • Mathematical Programming Computation, Vol. 8, Issue 4
  • DOI: 10.1007/s12532-016-0100-3