Optimization and supervised machine learning methods for fitting numerical physics models without derivatives

Bollapragada, Raghu; Menickelly, Matt; Nazarewicz, Witold; O’Neal, Jared; Reinhard, Paul-Gerhard; Wild, Stefan M.

doi:10.1088/1361-6471/abd009

Title: Optimization and supervised machine learning methods for fitting numerical physics models without derivatives

Journal Article · 31 December 2021 · Journal of Physics. G, Nuclear and Particle Physics

DOI: https://doi.org/10.1088/1361-6471/abd009 · OSTI ID:1765468

^[1];

^[2];

^[3];

^[2];

^[4];

^[2]

Argonne National Lab. (ANL), Argonne, IL (United States); Univ. of Texas, Austin, TX (United States)
Argonne National Lab. (ANL), Argonne, IL (United States)
Michigan State Univ., East Lansing, MI (United States)
Univ. Erlanger-Nurnberg (Germany)

Here, we address the calibration of a computationally expensive nuclear physics model for which derivative information with respect to the fit parameters is not readily available. Of particular interest is the performance of optimization-based training algorithms when dozens, rather than millions or more, of training data are available and when the expense of the model places limitations on the number of concurrent model evaluations that can be performed. As a case study, we consider the Fayans energy density functional model, which has characteristics similar to many model fitting and calibration problems in nuclear physics. We analyze hyperparameter tuning considerations and variability associated with stochastic optimization algorithms and illustrate considerations for tuning in different computational settings.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Argonne National Laboratory (ANL), Argonne, IL (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR); USDOE Office of Science (SC), Nuclear Physics (NP)

Grant/Contract Number:: AC02-06CH11357; SC0013365; SC0018083

OSTI ID:: 1765468

Journal Information:: Journal of Physics. G, Nuclear and Particle Physics, Journal Name: Journal of Physics. G, Nuclear and Particle Physics Journal Issue: 2 Vol. 48; ISSN 0954-3899

Publisher:: IOP PublishingCopyright Statement

Country of Publication:: United States

Language:: English

References (45)

Derivative-Free and Blackbox Optimization Audet, Charles; Hare, Warren Springer Series in Operations Research and Financial Engineering https://doi.org/10.1007/978-3-319-68913-5	book	January 2017
The Skyrme—Hartree—Fock Model of the Nuclear Ground State Reinhard, P. -G.; Langanke, K.; Maruhn, Joachim A. Computational Nuclear Physics 1 https://doi.org/10.1007/978-3-642-76356-4_2	book	January 1991
Random Gradient-Free Minimization of Convex Functions Nesterov, Yurii; Spokoiny, Vladimir Foundations of Computational Mathematics, Vol. 17, Issue 2 https://doi.org/10.1007/s10208-015-9296-2	journal	November 2015
The Open Images Dataset V4: Unified Image Classification, Object Detection, and Visual Relationship Detection at Scale Kuznetsova, Alina; Rom, Hassan; Alldrin, Neil International Journal of Computer Vision, Vol. 128, Issue 7 https://doi.org/10.1007/s11263-020-01316-z	journal	March 2020
Do you trust derivatives or differences? Moré, Jorge J.; Wild, Stefan M. Journal of Computational Physics, Vol. 273 https://doi.org/10.1016/j.jcp.2014.04.056	journal	September 2014
Derivative-free optimization methods Larson, Jeffrey; Menickelly, Matt; Wild, Stefan M. Acta Numerica, Vol. 28 https://doi.org/10.1017/s0962492919000060	journal	May 2019
Proton superfluidity and charge radii in proton-rich calcium isotopes Miller, A. J.; Minamisono, K.; Klose, A. Nature Physics, Vol. 15, Issue 5 https://doi.org/10.1038/s41567-019-0416-9	journal	February 2019
Measurement and microscopic description of odd–even staggering of charge radii of exotic copper isotopes de Groote, R. P.; Billowes, J.; Binnersley, C. L. Nature Physics, Vol. 16, Issue 6 https://doi.org/10.1038/s41567-020-0868-y	journal	April 2020
The importance of better models in stochastic optimization Asi, Hilal; Duchi, John C. Proceedings of the National Academy of Sciences, Vol. 116, Issue 46 https://doi.org/10.1073/pnas.1908018116	journal	October 2019
Error estimates of theoretical models: a guide Dobaczewski, J.; Nazarewicz, W.; Reinhard, P-G Journal of Physics G: Nuclear and Particle Physics, Vol. 41, Issue 7 https://doi.org/10.1088/0954-3899/41/7/074001	journal	May 2014
Statistical uncertainties of a chiral interaction at next-to-next-to leading order Ekström, A.; Carlsson, B. D.; Wendt, K. A. Journal of Physics G: Nuclear and Particle Physics, Vol. 42, Issue 3 https://doi.org/10.1088/0954-3899/42/3/034003	journal	February 2015
A Bayesian approach for parameter estimation and prediction using a computationally intensive model Higdon, Dave; McDonnell, Jordan D.; Schunck, Nicolas Journal of Physics G: Nuclear and Particle Physics, Vol. 42, Issue 3 https://doi.org/10.1088/0954-3899/42/3/034009	journal	February 2015
Derivative-free optimization for parameter estimation in computational nuclear physics Wild, Stefan M.; Sarich, Jason; Schunck, Nicolas Journal of Physics G: Nuclear and Particle Physics, Vol. 42, Issue 3 https://doi.org/10.1088/0954-3899/42/3/034031	journal	February 2015
Bayesian optimization in ab initio nuclear physics Ekström, A.; Forssén, C.; Dimitrakakis, C. Journal of Physics G: Nuclear and Particle Physics, Vol. 46, Issue 9 https://doi.org/10.1088/1361-6471/ab2b14	journal	July 2019
Energy Density Functional Methods for Atomic Nuclei Schunck, Nicolas https://doi.org/10.1088/2053-2563/aae0ed	book	January 2019
A Simplex Method for Function Minimization Nelder, J. A.; Mead, R. The Computer Journal, Vol. 7, Issue 4 https://doi.org/10.1093/comjnl/7.4.308	journal	January 1965
Uncertainty Quantification for Nuclear Density Functional Theory and Information Content of New Measurements McDonnell, J. D.; Schunck, N.; Higdon, D. Physical Review Letters, Vol. 114, Issue 12 https://doi.org/10.1103/PhysRevLett.114.122501	journal	March 2015
Laser Spectroscopy of Neutron-Rich Tin Isotopes: A Discontinuity in Charge Radii across the N = 82 Shell Closure Gorges, C.; Rodríguez, L. V.; Balabanski, D. L. Physical Review Letters, Vol. 122, Issue 19 https://doi.org/10.1103/PhysRevLett.122.192502	journal	May 2019
Beyond the charge radius: The information content of the fourth radial moment Reinhard, P. -G.; Nazarewicz, W.; Garcia Ruiz, R. F. Physical Review C, Vol. 101, Issue 2 https://doi.org/10.1103/physrevc.101.021301	journal	February 2020
Nuclear energy density optimization Kortelainen, M.; Lesinski, T.; Moré, J. Physical Review C, Vol. 82, Issue 2 https://doi.org/10.1103/physrevc.82.024313	journal	August 2010
Nuclear energy density optimization: Large deformations Kortelainen, M.; McDonnell, J.; Nazarewicz, W. Physical Review C, Vol. 85, Issue 2 https://doi.org/10.1103/physrevc.85.024304	journal	February 2012
Spurious finite-size instabilities in nuclear energy density functionals Hellemans, V.; Pastore, A.; Duguet, T. Physical Review C, Vol. 88, Issue 6 https://doi.org/10.1103/physrevc.88.064323	journal	December 2013
Nuclear energy density optimization: Shell structure Kortelainen, M.; McDonnell, J.; Nazarewicz, W. Physical Review C, Vol. 89, Issue 5 https://doi.org/10.1103/physrevc.89.054314	journal	May 2014
Spurious finite-size instabilities in nuclear energy density functionals: Spin channel Pastore, A.; Tarpanov, D.; Davesne, D. Physical Review C, Vol. 92, Issue 2 https://doi.org/10.1103/physrevc.92.024305	journal	August 2015
Local chiral potentials with Δ -intermediate states and the structure of light nuclei Piarulli, M.; Girlanda, L.; Schiavilla, R. Physical Review C, Vol. 94, Issue 5 https://doi.org/10.1103/physrevc.94.054007	journal	November 2016
Toward a global description of nuclear charge radii: Exploring the Fayans energy density functional Reinhard, P. -G.; Nazarewicz, W. Physical Review C, Vol. 95, Issue 6 https://doi.org/10.1103/physrevc.95.064328	journal	June 2017
Optimized Chiral Nucleon-Nucleon Interaction at Next-to-Next-to-Leading Order Ekström, A.; Baardsen, G.; Forssén, C. Physical Review Letters, Vol. 110, Issue 19 https://doi.org/10.1103/physrevlett.110.192502	journal	May 2013
From Calcium to Cadmium: Testing the Pairing Functional through Charge Radii Measurements of Cd 100 − 130 Hammen, M.; Nörtershäuser, W.; Balabanski, D. L. Physical Review Letters, Vol. 121, Issue 10 https://doi.org/10.1103/physrevlett.121.102501	journal	September 2018
Self-organized criticality and the Barkhausen effect Cote, P. J.; Meisel, L. V. Physical Review Letters, Vol. 67, Issue 10 https://doi.org/10.1103/physrevlett.67.1334	journal	September 1991
Self-consistent mean-field models for nuclear structure Bender, Michael; Heenen, Paul-Henri; Reinhard, Paul-Gerhard Reviews of Modern Physics, Vol. 75, Issue 1 https://doi.org/10.1103/revmodphys.75.121	journal	January 2003
Optimal Rates for Zero-Order Convex Optimization: The Power of Two Function Evaluations Duchi, John C.; Jordan, Michael I.; Wainwright, Martin J. IEEE Transactions on Information Theory, Vol. 61, Issue 5 https://doi.org/10.1109/tit.2015.2409256	journal	May 2015
Towards a universal nuclear density functional Fayans, S. A. Journal of Experimental and Theoretical Physics Letters, Vol. 68, Issue 3 https://doi.org/10.1134/1.567841	journal	August 1998
Stochastic online optimization. Single-point and multi-point non-linear multi-armed bandits. Convex and strongly-convex case Gasnikov, A. V.; Krymova, E. A.; Lagunovskaya, A. A. Automation and Remote Control, Vol. 78, Issue 2 https://doi.org/10.1134/s0005117917020035	journal	February 2017
Evaluating Derivatives Griewank, Andreas; Walther, Andrea https://doi.org/10.1137/1.9780898717761	book	January 2008
Introduction to Derivative-Free Optimization Conn, Andrew R.; Scheinberg, Katya; Vicente, Luis N. https://doi.org/10.1137/1.9780898718768	book	January 2009
Chapter 40: POUNDERS in TAO: Solving Derivative-Free Nonlinear Least-Squares Problems with POUNDERS Wild, Stefan M.; Terlaky, Tamás; Anjos, Miguel F. Advances and Trends in Optimization with Engineering Applications https://doi.org/10.1137/1.9781611974683.ch40	book	April 2017
Estimating Computational Noise Moré, Jorge J.; Wild, Stefan M. SIAM Journal on Scientific Computing, Vol. 33, Issue 3 https://doi.org/10.1137/100786125	journal	January 2011
Stochastic First- and Zeroth-Order Methods for Nonconvex Stochastic Programming Ghadimi, Saeed; Lan, Guanghui SIAM Journal on Optimization, Vol. 23, Issue 4 https://doi.org/10.1137/120880811	journal	January 2013
Optimization Methods for Large-Scale Machine Learning Bottou, Léon; Curtis, Frank E.; Nocedal, Jorge SIAM Review, Vol. 60, Issue 2 https://doi.org/10.1137/16m1080173	journal	January 2018
Budget-Dependent Convergence Rate of Stochastic Approximation L'Ecuyer, Pierre; Yin, George SIAM Journal on Optimization, Vol. 8, Issue 1 https://doi.org/10.1137/s1052623495270723	journal	February 1998
Convergence Properties of the Nelder--Mead Simplex Method in Low Dimensions Lagarias, Jeffrey C.; Reeds, James A.; Wright, Margaret H. SIAM Journal on Optimization, Vol. 9, Issue 1 https://doi.org/10.1137/s1052623496303470	journal	January 1998
Estimating Derivatives of Noisy Simulations Moré, Jorge J.; Wild, Stefan M. ACM Transactions on Mathematical Software, Vol. 38, Issue 3 https://doi.org/10.1145/2168773.2168777	journal	April 2012
Stochastic Estimation of the Maximum of a Regression Function Kiefer, J.; Wolfowitz, J. The Annals of Mathematical Statistics, Vol. 23, Issue 3 https://doi.org/10.1214/aoms/1177729392	journal	September 1952
A Stochastic Approximation Method Robbins, Herbert; Monro, Sutton The Annals of Mathematical Statistics, Vol. 22, Issue 3 https://doi.org/10.1214/aoms/1177729586	journal	September 1951
Simulation-Based Optimization with Stochastic Approximation Using Common Random Numbers Kleinman, Nathan L.; Spall, James C.; Naiman, Daniel Q. Management Science, Vol. 45, Issue 11 https://doi.org/10.1287/mnsc.45.11.1570	journal	November 1999

Similar Records

A Software Framework for Comparing Training Approaches for Spiking Neuromorphic Systems

Conference · 2021 · OSTI ID:1823363

Schuman, Catherine; Plank, James; Parsa, Maryam; +3 more

Optimizing training trajectories in variational autoencoders via latent Bayesian optimization approach^*

Journal Article · 2023 · Machine Learning: Science and Technology · OSTI ID:1923196

Biswas, Arpan; Vasudevan, Rama; Ziatdinov, Maxim; +1 more

A machine learning approach for particle accelerator errant beam prediction using spatial phase deviation

Journal Article · 2024 · Nuclear Instruments and Methods in Physics Research. Section A, Accelerators, Spectrometers, Detectors and Associated Equipment · OSTI ID:2448194

Yucesan, Yigit A.; Blokland, Willem; Ramuhalli, Pradeep; +4 more

Related Subjects

97 MATHEMATICS AND COMPUTING
Density functional theory
Machine learning in physics
Model Calibration
Numerical optimization

Title: Optimization and supervised machine learning methods for fitting numerical physics models without derivatives

Citation Formats

References (45)

Similar Records

Related Subjects