Optimization and supervised machine learning methods for fitting numerical physics models without derivatives

Bollapragada, Raghu; Menickelly, Matt; Nazarewicz, Witold; O’Neal, Jared; Reinhard, Paul-Gerhard; Wild, Stefan M.

doi:10.1088/1361-6471/abd009

Title: Optimization and supervised machine learning methods for fitting numerical physics models without derivatives

Journal Article · 31 December 2021 · Journal of Physics. G, Nuclear and Particle Physics

DOI: https://doi.org/10.1088/1361-6471/abd009 · OSTI ID:1765468

^[1];

^[2];

^[3];

^[2];

^[4];

^[2]

Argonne National Lab. (ANL), Argonne, IL (United States); Univ. of Texas, Austin, TX (United States)
Argonne National Lab. (ANL), Argonne, IL (United States)
Michigan State Univ., East Lansing, MI (United States)
Univ. Erlanger-Nurnberg (Germany)

Here, we address the calibration of a computationally expensive nuclear physics model for which derivative information with respect to the fit parameters is not readily available. Of particular interest is the performance of optimization-based training algorithms when dozens, rather than millions or more, of training data are available and when the expense of the model places limitations on the number of concurrent model evaluations that can be performed. As a case study, we consider the Fayans energy density functional model, which has characteristics similar to many model fitting and calibration problems in nuclear physics. We analyze hyperparameter tuning considerations and variability associated with stochastic optimization algorithms and illustrate considerations for tuning in different computational settings.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Argonne National Laboratory (ANL), Argonne, IL (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Nuclear Physics (NP); USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)

Grant/Contract Number:: AC02-06CH11357; SC0013365; SC0018083

OSTI ID:: 1765468

Journal Information:: Journal of Physics. G, Nuclear and Particle Physics, Vol. 48, Issue 2; ISSN 0954-3899

Publisher:: IOP PublishingCopyright Statement

Country of Publication:: United States

Language:: English

References (30)

Estimating Derivatives of Noisy Simulations Moré, Jorge J.; Wild, Stefan M. ACM Transactions on Mathematical Software, Vol. 38, Issue 3 https://doi.org/10.1145/2168773.2168777	journal	April 2012
The importance of better models in stochastic optimization Asi, Hilal; Duchi, John C. Proceedings of the National Academy of Sciences, Vol. 116, Issue 46 https://doi.org/10.1073/pnas.1908018116	journal	October 2019
Do you trust derivatives or differences? Moré, Jorge J.; Wild, Stefan M. Journal of Computational Physics, Vol. 273 https://doi.org/10.1016/j.jcp.2014.04.056	journal	September 2014
Evaluating Derivatives Griewank, Andreas; Walther, Andrea https://doi.org/10.1137/1.9780898717761	book	January 2008
Chapter 40: POUNDERS in TAO: Solving Derivative-Free Nonlinear Least-Squares Problems with POUNDERS Wild, Stefan M.; Terlaky, Tamás; Anjos, Miguel F. Advances and Trends in Optimization with Engineering Applications https://doi.org/10.1137/1.9781611974683.ch40	book	April 2017
Laser Spectroscopy of Neutron-Rich Tin Isotopes: A Discontinuity in Charge Radii across the N = 82 Shell Closure Gorges, C.; Rodríguez, L. V.; Balabanski, D. L. Physical Review Letters, Vol. 122, Issue 19 https://doi.org/10.1103/PhysRevLett.122.192502	journal	May 2019
Energy Density Functional Methods for Atomic Nuclei Schunck, Nicolas https://doi.org/10.1088/2053-2563/aae0ed	book	January 2019
Derivative-Free and Blackbox Optimization Audet, Charles; Hare, Warren Springer Series in Operations Research and Financial Engineering https://doi.org/10.1007/978-3-319-68913-5	book	January 2017
Towards a universal nuclear density functional Fayans, S. A. Journal of Experimental and Theoretical Physics Letters, Vol. 68, Issue 3 https://doi.org/10.1134/1.567841	journal	August 1998
Statistical uncertainties of a chiral interaction at next-to-next-to leading order Ekström, A.; Carlsson, B. D.; Wendt, K. A. Journal of Physics G: Nuclear and Particle Physics, Vol. 42, Issue 3 https://doi.org/10.1088/0954-3899/42/3/034003	journal	February 2015
Beyond the charge radius: The information content of the fourth radial moment Reinhard, P. -G.; Nazarewicz, W.; Garcia Ruiz, R. F. Physical Review C, Vol. 101, Issue 2 https://doi.org/10.1103/physrevc.101.021301	journal	February 2020
The Skyrme—Hartree—Fock Model of the Nuclear Ground State Reinhard, P. -G.; Langanke, K.; Maruhn, Joachim A. Computational Nuclear Physics 1 https://doi.org/10.1007/978-3-642-76356-4_2	book	January 1991
The Open Images Dataset V4: Unified Image Classification, Object Detection, and Visual Relationship Detection at Scale Kuznetsova, Alina; Rom, Hassan; Alldrin, Neil International Journal of Computer Vision, Vol. 128, Issue 7 https://doi.org/10.1007/s11263-020-01316-z	journal	March 2020
Stochastic Estimation of the Maximum of a Regression Function Kiefer, J.; Wolfowitz, J. The Annals of Mathematical Statistics, Vol. 23, Issue 3 https://doi.org/10.1214/aoms/1177729392	journal	September 1952
Simulation-Based Optimization with Stochastic Approximation Using Common Random Numbers Kleinman, Nathan L.; Spall, James C.; Naiman, Daniel Q. Management Science, Vol. 45, Issue 11 https://doi.org/10.1287/mnsc.45.11.1570	journal	November 1999
Proton superfluidity and charge radii in proton-rich calcium isotopes Miller, A. J.; Minamisono, K.; Klose, A. Nature Physics, Vol. 15, Issue 5 https://doi.org/10.1038/s41567-019-0416-9	journal	February 2019
Introduction to Derivative-Free Optimization Conn, Andrew R.; Scheinberg, Katya; Vicente, Luis N. https://doi.org/10.1137/1.9780898718768	book	January 2009
Stochastic First- and Zeroth-Order Methods for Nonconvex Stochastic Programming Ghadimi, Saeed; Lan, Guanghui SIAM Journal on Optimization, Vol. 23, Issue 4 https://doi.org/10.1137/120880811	journal	January 2013
Random Gradient-Free Minimization of Convex Functions Nesterov, Yurii; Spokoiny, Vladimir Foundations of Computational Mathematics, Vol. 17, Issue 2 https://doi.org/10.1007/s10208-015-9296-2	journal	November 2015
A Simplex Method for Function Minimization Nelder, J. A.; Mead, R. The Computer Journal, Vol. 7, Issue 4 https://doi.org/10.1093/comjnl/7.4.308	journal	January 1965
Spurious finite-size instabilities in nuclear energy density functionals: Spin channel Pastore, A.; Tarpanov, D.; Davesne, D. Physical Review C, Vol. 92, Issue 2 https://doi.org/10.1103/physrevc.92.024305	journal	August 2015
A Bayesian approach for parameter estimation and prediction using a computationally intensive model Higdon, Dave; McDonnell, Jordan D.; Schunck, Nicolas Journal of Physics G: Nuclear and Particle Physics, Vol. 42, Issue 3 https://doi.org/10.1088/0954-3899/42/3/034009	journal	February 2015
Derivative-free optimization methods Larson, Jeffrey; Menickelly, Matt; Wild, Stefan M. Acta Numerica, Vol. 28 https://doi.org/10.1017/s0962492919000060	journal	May 2019
Estimating Computational Noise Moré, Jorge J.; Wild, Stefan M. SIAM Journal on Scientific Computing, Vol. 33, Issue 3 https://doi.org/10.1137/100786125	journal	January 2011
Measurement and microscopic description of odd–even staggering of charge radii of exotic copper isotopes de Groote, R. P.; Billowes, J.; Binnersley, C. L. Nature Physics, Vol. 16, Issue 6 https://doi.org/10.1038/s41567-020-0868-y	journal	April 2020
Uncertainty Quantification for Nuclear Density Functional Theory and Information Content of New Measurements McDonnell, J. D.; Schunck, N.; Higdon, D. Physical Review Letters, Vol. 114, Issue 12 https://doi.org/10.1103/PhysRevLett.114.122501	journal	March 2015
Bayesian optimization in ab initio nuclear physics Ekström, A.; Forssén, C.; Dimitrakakis, C. Journal of Physics G: Nuclear and Particle Physics, Vol. 46, Issue 9 https://doi.org/10.1088/1361-6471/ab2b14	journal	July 2019
Error estimates of theoretical models: a guide Dobaczewski, J.; Nazarewicz, W.; Reinhard, P-G Journal of Physics G: Nuclear and Particle Physics, Vol. 41, Issue 7 https://doi.org/10.1088/0954-3899/41/7/074001	journal	May 2014
A Stochastic Approximation Method Robbins, Herbert; Monro, Sutton The Annals of Mathematical Statistics, Vol. 22, Issue 3 https://doi.org/10.1214/aoms/1177729586	journal	September 1951
Derivative-free optimization for parameter estimation in computational nuclear physics Wild, Stefan M.; Sarich, Jason; Schunck, Nicolas Journal of Physics G: Nuclear and Particle Physics, Vol. 42, Issue 3 https://doi.org/10.1088/0954-3899/42/3/034031	journal	February 2015

Similar Records

Scalable deep learning for watershed model calibration

Journal Article · 2022 · Frontiers in Earth Science · OSTI ID:1900374

Mudunuru, Maruti K.; Son, Kyongho; Jiang, Peishi; +2 more

Machine Learning Assisted Safety Modeling and Analysis of Advanced Reactors

Technical Report · 2021 · OSTI ID:1838607

Liu, Yang; Hu, Rui; Dai, Dezhi; +2 more

Deep learning to estimate permeability using geophysical data

Journal Article · 2022 · Advances in Water Resources · OSTI ID:1883390

Mudunuru, Maruti K.; Cromwell, Erol L. D.; Wang, Hongsheng; +1 more

Related Subjects

97 MATHEMATICS AND COMPUTING
Density functional theory
Machine learning in physics
Model Calibration
Numerical optimization

Title: Optimization and supervised machine learning methods for fitting numerical physics models without derivatives

Citation Formats

References (30)

Similar Records

Related Subjects