On efficient Hessian computation using the edge pushing algorithm in Julia
Abstract
Evaluating the Hessian matrix of second-order derivatives at a sequence of points can be costly when applying second-order methods for nonlinear optimization. In this work, we discuss our experiences implementing the recently proposed Edge Pushing (EP) method in Julia as an experimental replacement for the current colouring-based methods used by JuMP, an open-source algebraic modelling language. We propose an alternative data structure for sparse Hessians to avoid the use of hash tables and analyse the space and time complexity of EP method. In our benchmarks, we find that EP is very competitive in terms of both preprocessing time and Hessian evaluation time. As a result, we identify cases where EP closes the performance gap between JuMP's previous implementation and the implementation in AMPL, a commercial software package with similar functionality.
- Authors:
-
- Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
- Argonne National Lab. (ANL), Lemont, IL (United States)
- Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States)
- Publication Date:
- Research Org.:
- Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
- Sponsoring Org.:
- USDOE National Nuclear Security Administration (NNSA)
- OSTI Identifier:
- 1474340
- Report Number(s):
- LLNL-JRNL-720198
Journal ID: ISSN 1055-6788; 865490
- Grant/Contract Number:
- AC52-07NA27344
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Optimization Methods and Software
- Additional Journal Information:
- Journal Volume: 33; Journal Issue: 4-6; Journal ID: ISSN 1055-6788
- Publisher:
- Taylor & Francis
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 97 MATHEMATICS AND COMPUTING; Automatic differentiation; sparse Hessian computation; edge pushing; algebraic modelling language; Julia; JuMP
Citation Formats
Petra, C. G., Qiang, F., Lubin, M., and Huchette, J. On efficient Hessian computation using the edge pushing algorithm in Julia. United States: N. p., 2018.
Web. doi:10.1080/10556788.2018.1480625.
Petra, C. G., Qiang, F., Lubin, M., & Huchette, J. On efficient Hessian computation using the edge pushing algorithm in Julia. United States. https://doi.org/10.1080/10556788.2018.1480625
Petra, C. G., Qiang, F., Lubin, M., and Huchette, J. Wed .
"On efficient Hessian computation using the edge pushing algorithm in Julia". United States. https://doi.org/10.1080/10556788.2018.1480625. https://www.osti.gov/servlets/purl/1474340.
@article{osti_1474340,
title = {On efficient Hessian computation using the edge pushing algorithm in Julia},
author = {Petra, C. G. and Qiang, F. and Lubin, M. and Huchette, J.},
abstractNote = {Evaluating the Hessian matrix of second-order derivatives at a sequence of points can be costly when applying second-order methods for nonlinear optimization. In this work, we discuss our experiences implementing the recently proposed Edge Pushing (EP) method in Julia as an experimental replacement for the current colouring-based methods used by JuMP, an open-source algebraic modelling language. We propose an alternative data structure for sparse Hessians to avoid the use of hash tables and analyse the space and time complexity of EP method. In our benchmarks, we find that EP is very competitive in terms of both preprocessing time and Hessian evaluation time. As a result, we identify cases where EP closes the performance gap between JuMP's previous implementation and the implementation in AMPL, a commercial software package with similar functionality.},
doi = {10.1080/10556788.2018.1480625},
journal = {Optimization Methods and Software},
number = 4-6,
volume = 33,
place = {United States},
year = {Wed Sep 12 00:00:00 EDT 2018},
month = {Wed Sep 12 00:00:00 EDT 2018}
}
Free Publicly Available Full Text
Publisher's Version of Record
Other availability
Search WorldCat to find libraries that may hold this journal
Cited by: 3 works
Citation information provided by
Web of Science
Web of Science
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
ColPack: Software for graph coloring and related problems in scientific computing
journal, September 2013
- Gebremedhin, Assefaw H.; Nguyen, Duc; Patwary, Md. Mostofa Ali
- ACM Transactions on Mathematical Software, Vol. 40, Issue 1
Efficient Computation of Sparse Hessians Using Coloring and Automatic Differentiation
journal, May 2009
- Gebremedhin, Assefaw H.; Tarafdar, Arijit; Pothen, Alex
- INFORMS Journal on Computing, Vol. 21, Issue 2
LIBSVM: A library for support vector machines
journal, April 2011
- Chang, Chih-Chung; Lin, Chih-Jen
- ACM Transactions on Intelligent Systems and Technology, Vol. 2, Issue 3
Partitioned variable metric updates for large structured optimization problems
journal, February 1982
- Griewank, A.; Toint, Ph. L.
- Numerische Mathematik, Vol. 39, Issue 1
JuMP: A Modeling Language for Mathematical Optimization
journal, January 2017
- Dunning, Iain; Huchette, Joey; Lubin, Miles
- SIAM Review, Vol. 59, Issue 2
Computing the sparsity pattern of Hessians using automatic differentiation
journal, February 2014
- Gower, Robert Mansel; Mello, Margarida Pinheiro
- ACM Transactions on Mathematical Software, Vol. 40, Issue 2
What Color Is Your Jacobian? Graph Coloring for Computing Derivatives
journal, January 2005
- Gebremedhin, Assefaw Hadish; Manne, Fredrik; Pothen, Alex
- SIAM Review, Vol. 47, Issue 4
A new framework for the computation of Hessians
journal, April 2012
- Gower, R. M.; Mello, M. P.
- Optimization Methods and Software, Vol. 27, Issue 2
Local convergence analysis for partitioned quasi-Newton updates
journal, October 1982
- Griewank, A.; Toint, Ph. L.
- Numerische Mathematik, Vol. 39, Issue 3
New Acyclic and Star Coloring Algorithms with Application to Computing Hessians
journal, January 2007
- Gebremedhin, Assefaw H.; Tarafdar, Arijit; Manne, Fredrik
- SIAM Journal on Scientific Computing, Vol. 29, Issue 3
Capitalizing on live variables: new algorithms for efficient Hessian computation via automatic differentiation
journal, February 2016
- Wang, Mu; Gebremedhin, Assefaw; Pothen, Alex
- Mathematical Programming Computation, Vol. 8, Issue 4