skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Optimization and supervised machine learning methods for fitting numerical physics models without derivatives

Abstract

Here, we address the calibration of a computationally expensive nuclear physics model for which derivative information with respect to the fit parameters is not readily available. Of particular interest is the performance of optimization-based training algorithms when dozens, rather than millions or more, of training data are available and when the expense of the model places limitations on the number of concurrent model evaluations that can be performed. As a case study, we consider the Fayans energy density functional model, which has characteristics similar to many model fitting and calibration problems in nuclear physics. We analyze hyperparameter tuning considerations and variability associated with stochastic optimization algorithms and illustrate considerations for tuning in different computational settings.

Authors:
ORCiD logo [1]; ORCiD logo [2]; ORCiD logo [3]; ORCiD logo [2]; ORCiD logo [4]; ORCiD logo [2]
  1. Argonne National Lab. (ANL), Argonne, IL (United States); Univ. of Texas, Austin, TX (United States)
  2. Argonne National Lab. (ANL), Argonne, IL (United States)
  3. Michigan State Univ., East Lansing, MI (United States)
  4. Univ. Erlanger-Nurnberg (Germany)
Publication Date:
Research Org.:
Argonne National Lab. (ANL), Argonne, IL (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Nuclear Physics (NP); USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
OSTI Identifier:
1765468
Grant/Contract Number:  
AC02-06CH11357; SC0013365; SC0018083
Resource Type:
Journal Article: Accepted Manuscript
Journal Name:
Journal of Physics. G, Nuclear and Particle Physics
Additional Journal Information:
Journal Volume: 48; Journal Issue: 2; Journal ID: ISSN 0954-3899
Publisher:
IOP Publishing
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; Density functional theory; Machine learning in physics; Model Calibration; Numerical optimization

Citation Formats

Bollapragada, Raghu, Menickelly, Matt, Nazarewicz, Witold, O’Neal, Jared, Reinhard, Paul-Gerhard, and Wild, Stefan M. Optimization and supervised machine learning methods for fitting numerical physics models without derivatives. United States: N. p., 2021. Web. doi:10.1088/1361-6471/abd009.
Bollapragada, Raghu, Menickelly, Matt, Nazarewicz, Witold, O’Neal, Jared, Reinhard, Paul-Gerhard, & Wild, Stefan M. Optimization and supervised machine learning methods for fitting numerical physics models without derivatives. United States. https://doi.org/10.1088/1361-6471/abd009
Bollapragada, Raghu, Menickelly, Matt, Nazarewicz, Witold, O’Neal, Jared, Reinhard, Paul-Gerhard, and Wild, Stefan M. 2021. "Optimization and supervised machine learning methods for fitting numerical physics models without derivatives". United States. https://doi.org/10.1088/1361-6471/abd009.
@article{osti_1765468,
title = {Optimization and supervised machine learning methods for fitting numerical physics models without derivatives},
author = {Bollapragada, Raghu and Menickelly, Matt and Nazarewicz, Witold and O’Neal, Jared and Reinhard, Paul-Gerhard and Wild, Stefan M.},
abstractNote = {Here, we address the calibration of a computationally expensive nuclear physics model for which derivative information with respect to the fit parameters is not readily available. Of particular interest is the performance of optimization-based training algorithms when dozens, rather than millions or more, of training data are available and when the expense of the model places limitations on the number of concurrent model evaluations that can be performed. As a case study, we consider the Fayans energy density functional model, which has characteristics similar to many model fitting and calibration problems in nuclear physics. We analyze hyperparameter tuning considerations and variability associated with stochastic optimization algorithms and illustrate considerations for tuning in different computational settings.},
doi = {10.1088/1361-6471/abd009},
url = {https://www.osti.gov/biblio/1765468}, journal = {Journal of Physics. G, Nuclear and Particle Physics},
issn = {0954-3899},
number = 2,
volume = 48,
place = {United States},
year = {2021},
month = {12}
}

Journal Article:
Free Publicly Available Full Text
This content will become publicly available on December 31, 2022
Publisher's Version of Record

Save / Share:

Works referenced in this record:

From Calcium to Cadmium: Testing the Pairing Functional through Charge Radii Measurements of Cd 100 130
journal, September 2018


Estimating Derivatives of Noisy Simulations
journal, April 2012


Local chiral potentials with Δ -intermediate states and the structure of light nuclei
journal, November 2016


The importance of better models in stochastic optimization
journal, October 2019


Do you trust derivatives or differences?
journal, September 2014


Evaluating Derivatives
book, January 2008


Chapter 40: POUNDERS in TAO: Solving Derivative-Free Nonlinear Least-Squares Problems with POUNDERS
book, April 2017


Laser Spectroscopy of Neutron-Rich Tin Isotopes: A Discontinuity in Charge Radii across the N = 82 Shell Closure
journal, May 2019


Optimization Methods for Large-Scale Machine Learning
journal, January 2018


Derivative-Free and Blackbox Optimization
book, January 2017


Towards a universal nuclear density functional
journal, August 1998


Statistical uncertainties of a chiral interaction at next-to-next-to leading order
journal, February 2015


Beyond the charge radius: The information content of the fourth radial moment
journal, February 2020


Optimized Chiral Nucleon-Nucleon Interaction at Next-to-Next-to-Leading Order
journal, May 2013


The Skyrme—Hartree—Fock Model of the Nuclear Ground State
book, January 1991


The Open Images Dataset V4: Unified Image Classification, Object Detection, and Visual Relationship Detection at Scale
journal, March 2020


Stochastic Estimation of the Maximum of a Regression Function
journal, September 1952


Simulation-Based Optimization with Stochastic Approximation Using Common Random Numbers
journal, November 1999


Proton superfluidity and charge radii in proton-rich calcium isotopes
journal, February 2019


Introduction to Derivative-Free Optimization
book, January 2009


Stochastic First- and Zeroth-Order Methods for Nonconvex Stochastic Programming
journal, January 2013


Convergence Properties of the Nelder--Mead Simplex Method in Low Dimensions
journal, January 1998


Self-consistent mean-field models for nuclear structure
journal, January 2003


Random Gradient-Free Minimization of Convex Functions
journal, November 2015


Nuclear energy density optimization: Large deformations
journal, February 2012


A Simplex Method for Function Minimization
journal, January 1965


Spurious finite-size instabilities in nuclear energy density functionals: Spin channel
journal, August 2015


A Bayesian approach for parameter estimation and prediction using a computationally intensive model
journal, February 2015


Derivative-free optimization methods
journal, May 2019


Estimating Computational Noise
journal, January 2011


Nuclear energy density optimization
journal, August 2010


Nuclear energy density optimization: Shell structure
journal, May 2014


Measurement and microscopic description of odd–even staggering of charge radii of exotic copper isotopes
journal, April 2020


Uncertainty Quantification for Nuclear Density Functional Theory and Information Content of New Measurements
journal, March 2015


Bayesian optimization in ab initio nuclear physics
journal, July 2019


Error estimates of theoretical models: a guide
journal, May 2014


A Stochastic Approximation Method
journal, September 1951