DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

This content will become publicly available on December 31, 2022

Title: Optimization and supervised machine learning methods for fitting numerical physics models without derivatives

Abstract

Here, we address the calibration of a computationally expensive nuclear physics model for which derivative information with respect to the fit parameters is not readily available. Of particular interest is the performance of optimization-based training algorithms when dozens, rather than millions or more, of training data are available and when the expense of the model places limitations on the number of concurrent model evaluations that can be performed. As a case study, we consider the Fayans energy density functional model, which has characteristics similar to many model fitting and calibration problems in nuclear physics. We analyze hyperparameter tuning considerations and variability associated with stochastic optimization algorithms and illustrate considerations for tuning in different computational settings.

Authors:
ORCiD logo [1]; ORCiD logo [2]; ORCiD logo [3]; ORCiD logo [2]; ORCiD logo [4]; ORCiD logo [2]
  1. Argonne National Lab. (ANL), Argonne, IL (United States); Univ. of Texas, Austin, TX (United States)
  2. Argonne National Lab. (ANL), Argonne, IL (United States)
  3. Michigan State Univ., East Lansing, MI (United States)
  4. Univ. Erlanger-Nurnberg (Germany)
Publication Date:
Research Org.:
Argonne National Lab. (ANL), Argonne, IL (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Nuclear Physics (NP); USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
OSTI Identifier:
1765468
Grant/Contract Number:  
AC02-06CH11357; SC0013365; SC0018083
Resource Type:
Accepted Manuscript
Journal Name:
Journal of Physics. G, Nuclear and Particle Physics
Additional Journal Information:
Journal Volume: 48; Journal Issue: 2; Journal ID: ISSN 0954-3899
Publisher:
IOP Publishing
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; Density functional theory; Machine learning in physics; Model Calibration; Numerical optimization

Citation Formats

Bollapragada, Raghu, Menickelly, Matt, Nazarewicz, Witold, O’Neal, Jared, Reinhard, Paul-Gerhard, and Wild, Stefan M. Optimization and supervised machine learning methods for fitting numerical physics models without derivatives. United States: N. p., 2021. Web. doi:10.1088/1361-6471/abd009.
Bollapragada, Raghu, Menickelly, Matt, Nazarewicz, Witold, O’Neal, Jared, Reinhard, Paul-Gerhard, & Wild, Stefan M. Optimization and supervised machine learning methods for fitting numerical physics models without derivatives. United States. https://doi.org/10.1088/1361-6471/abd009
Bollapragada, Raghu, Menickelly, Matt, Nazarewicz, Witold, O’Neal, Jared, Reinhard, Paul-Gerhard, and Wild, Stefan M. Fri . "Optimization and supervised machine learning methods for fitting numerical physics models without derivatives". United States. https://doi.org/10.1088/1361-6471/abd009.
@article{osti_1765468,
title = {Optimization and supervised machine learning methods for fitting numerical physics models without derivatives},
author = {Bollapragada, Raghu and Menickelly, Matt and Nazarewicz, Witold and O’Neal, Jared and Reinhard, Paul-Gerhard and Wild, Stefan M.},
abstractNote = {Here, we address the calibration of a computationally expensive nuclear physics model for which derivative information with respect to the fit parameters is not readily available. Of particular interest is the performance of optimization-based training algorithms when dozens, rather than millions or more, of training data are available and when the expense of the model places limitations on the number of concurrent model evaluations that can be performed. As a case study, we consider the Fayans energy density functional model, which has characteristics similar to many model fitting and calibration problems in nuclear physics. We analyze hyperparameter tuning considerations and variability associated with stochastic optimization algorithms and illustrate considerations for tuning in different computational settings.},
doi = {10.1088/1361-6471/abd009},
journal = {Journal of Physics. G, Nuclear and Particle Physics},
number = 2,
volume = 48,
place = {United States},
year = {2021},
month = {12}
}

Journal Article:
Free Publicly Available Full Text
This content will become publicly available on December 31, 2022
Publisher's Version of Record

Save / Share:

Works referenced in this record:

From Calcium to Cadmium: Testing the Pairing Functional through Charge Radii Measurements of Cd 100 130
journal, September 2018


Estimating Derivatives of Noisy Simulations
journal, April 2012

  • Moré, Jorge J.; Wild, Stefan M.
  • ACM Transactions on Mathematical Software, Vol. 38, Issue 3
  • DOI: 10.1145/2168773.2168777

Local chiral potentials with Δ -intermediate states and the structure of light nuclei
journal, November 2016


The importance of better models in stochastic optimization
journal, October 2019

  • Asi, Hilal; Duchi, John C.
  • Proceedings of the National Academy of Sciences, Vol. 116, Issue 46
  • DOI: 10.1073/pnas.1908018116

Do you trust derivatives or differences?
journal, September 2014


Evaluating Derivatives
book, January 2008


Chapter 40: POUNDERS in TAO: Solving Derivative-Free Nonlinear Least-Squares Problems with POUNDERS
book, April 2017

  • Wild, Stefan M.; Terlaky, Tamás; Anjos, Miguel F.
  • Advances and Trends in Optimization with Engineering Applications
  • DOI: 10.1137/1.9781611974683.ch40

Laser Spectroscopy of Neutron-Rich Tin Isotopes: A Discontinuity in Charge Radii across the N = 82 Shell Closure
journal, May 2019


Optimization Methods for Large-Scale Machine Learning
journal, January 2018

  • Bottou, Léon; Curtis, Frank E.; Nocedal, Jorge
  • SIAM Review, Vol. 60, Issue 2
  • DOI: 10.1137/16m1080173

Toward a global description of nuclear charge radii: Exploring the Fayans energy density functional
journal, June 2017


Derivative-Free and Blackbox Optimization
book, January 2017

  • Audet, Charles; Hare, Warren
  • Springer Series in Operations Research and Financial Engineering
  • DOI: 10.1007/978-3-319-68913-5

Towards a universal nuclear density functional
journal, August 1998

  • Fayans, S. A.
  • Journal of Experimental and Theoretical Physics Letters, Vol. 68, Issue 3
  • DOI: 10.1134/1.567841

Statistical uncertainties of a chiral interaction at next-to-next-to leading order
journal, February 2015

  • Ekström, A.; Carlsson, B. D.; Wendt, K. A.
  • Journal of Physics G: Nuclear and Particle Physics, Vol. 42, Issue 3
  • DOI: 10.1088/0954-3899/42/3/034003

Beyond the charge radius: The information content of the fourth radial moment
journal, February 2020


Optimized Chiral Nucleon-Nucleon Interaction at Next-to-Next-to-Leading Order
journal, May 2013


The Skyrme—Hartree—Fock Model of the Nuclear Ground State
book, January 1991


The Open Images Dataset V4: Unified Image Classification, Object Detection, and Visual Relationship Detection at Scale
journal, March 2020

  • Kuznetsova, Alina; Rom, Hassan; Alldrin, Neil
  • International Journal of Computer Vision, Vol. 128, Issue 7
  • DOI: 10.1007/s11263-020-01316-z

Stochastic Estimation of the Maximum of a Regression Function
journal, September 1952


Simulation-Based Optimization with Stochastic Approximation Using Common Random Numbers
journal, November 1999

  • Kleinman, Nathan L.; Spall, James C.; Naiman, Daniel Q.
  • Management Science, Vol. 45, Issue 11
  • DOI: 10.1287/mnsc.45.11.1570

Proton superfluidity and charge radii in proton-rich calcium isotopes
journal, February 2019


Introduction to Derivative-Free Optimization
book, January 2009


Stochastic First- and Zeroth-Order Methods for Nonconvex Stochastic Programming
journal, January 2013

  • Ghadimi, Saeed; Lan, Guanghui
  • SIAM Journal on Optimization, Vol. 23, Issue 4
  • DOI: 10.1137/120880811

Convergence Properties of the Nelder--Mead Simplex Method in Low Dimensions
journal, January 1998

  • Lagarias, Jeffrey C.; Reeds, James A.; Wright, Margaret H.
  • SIAM Journal on Optimization, Vol. 9, Issue 1
  • DOI: 10.1137/s1052623496303470

Self-consistent mean-field models for nuclear structure
journal, January 2003

  • Bender, Michael; Heenen, Paul-Henri; Reinhard, Paul-Gerhard
  • Reviews of Modern Physics, Vol. 75, Issue 1
  • DOI: 10.1103/revmodphys.75.121

Random Gradient-Free Minimization of Convex Functions
journal, November 2015

  • Nesterov, Yurii; Spokoiny, Vladimir
  • Foundations of Computational Mathematics, Vol. 17, Issue 2
  • DOI: 10.1007/s10208-015-9296-2

Nuclear energy density optimization: Large deformations
journal, February 2012


A Simplex Method for Function Minimization
journal, January 1965


Spurious finite-size instabilities in nuclear energy density functionals: Spin channel
journal, August 2015


A Bayesian approach for parameter estimation and prediction using a computationally intensive model
journal, February 2015

  • Higdon, Dave; McDonnell, Jordan D.; Schunck, Nicolas
  • Journal of Physics G: Nuclear and Particle Physics, Vol. 42, Issue 3
  • DOI: 10.1088/0954-3899/42/3/034009

Derivative-free optimization methods
journal, May 2019


Estimating Computational Noise
journal, January 2011

  • Moré, Jorge J.; Wild, Stefan M.
  • SIAM Journal on Scientific Computing, Vol. 33, Issue 3
  • DOI: 10.1137/100786125

Nuclear energy density optimization
journal, August 2010


Nuclear energy density optimization: Shell structure
journal, May 2014


Measurement and microscopic description of odd–even staggering of charge radii of exotic copper isotopes
journal, April 2020


Uncertainty Quantification for Nuclear Density Functional Theory and Information Content of New Measurements
journal, March 2015


Bayesian optimization in ab initio nuclear physics
journal, July 2019

  • Ekström, A.; Forssén, C.; Dimitrakakis, C.
  • Journal of Physics G: Nuclear and Particle Physics, Vol. 46, Issue 9
  • DOI: 10.1088/1361-6471/ab2b14

Error estimates of theoretical models: a guide
journal, May 2014

  • Dobaczewski, J.; Nazarewicz, W.; Reinhard, P-G
  • Journal of Physics G: Nuclear and Particle Physics, Vol. 41, Issue 7
  • DOI: 10.1088/0954-3899/41/7/074001

A Stochastic Approximation Method
journal, September 1951

  • Robbins, Herbert; Monro, Sutton
  • The Annals of Mathematical Statistics, Vol. 22, Issue 3
  • DOI: 10.1214/aoms/1177729586