Improved information criteria for Bayesian model averaging in lattice field theory

Neil, Ethan T.; Sitison, Jacob W.

doi:10.1103/physrevd.109.014510

Improved information criteria for Bayesian model averaging in lattice field theory

Journal Article · Mon Jan 29 00:00:00 EST 2024 · Physical Review. D.

DOI:https://doi.org/10.1103/physrevd.109.014510· OSTI ID:2283673

^[1]; ^[2]

Univ. of Colorado, Boulder, CO (United States); Department of Physics, University of Colorado, Boulder
Univ. of Colorado, Boulder, CO (United States)

Bayesian model averaging is a practical method for dealing with uncertainty due to model specification. Use of this technique requires the estimation of model probability weights. Here, we revisit the derivation of estimators for these model weights. Use of the Kullback-Leibler divergence as a starting point leads naturally to a number of alternative information criteria suitable for Bayesian model weight estimation. We explore three such criteria, known to the statistics literature before, in detail: a Bayesian analog of the Akaike information criterion which we call the BAIC, the Bayesian predictive information criterion, and the posterior predictive information criterion (PPIC). We compare the use of these information criteria in numerical analysis problems common in lattice field theory calculations. We find that the PPIC has the most appealing theoretical properties and can give the best performance in terms of model-averaging uncertainty, particularly in the presence of noisy data, while the BAIC is a simple and reliable alternative.

View Accepted Manuscript (DOE)

Research Organization:: Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). National Energy Research Scientific Computing Center (NERSC); Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF); Univ. of Colorado, Boulder, CO (United States)

Sponsoring Organization:: USDOE Office of Science (SC), High Energy Physics (HEP)

Grant/Contract Number:: AC02-05CH11231; AC05-00OR22725; SC0010005

OSTI ID:: 2283673

Alternate ID(s):: OSTI ID: 2294139

Journal Information:: Physical Review. D., Journal Name: Physical Review. D. Journal Issue: 1 Vol. 109; ISSN 2470-0010

Publisher:: American Physical Society (APS)Copyright Statement

Country of Publication:: United States

Language:: English

References (71)

Bayesian model averaging in astrophysics: a review Parkinson, David; Liddle, Andrew R. Statistical Analysis and Data Mining, Vol. 6, Issue 1 https://doi.org/10.1002/sam.11179	journal	January 2013
Asymptotics beyond All Orders in a Model of Crystal Growth Kruskal, Martin D.; Segur, Harvey Studies in Applied Mathematics, Vol. 85, Issue 2 https://doi.org/10.1002/sapm1991852129	journal	August 1991
Bayesian Model Selection and Model Averaging Wasserman, Larry Journal of Mathematical Psychology, Vol. 44, Issue 1 https://doi.org/10.1006/jmps.1999.1278	journal	March 2000
Information Criteria and Statistical Modeling Konishi, Sadanori; Kitagawa, Genshiro Springer Series in Statistics https://doi.org/10.1007/978-0-387-71887-3	book	October 2007
Asymptotics, Superasymptotics, Hyperasymptotics... Berry, Michael Asymptotics beyond All Orders https://doi.org/10.1007/978-1-4757-0435-8_1	book	January 1991
Sur les intégrales irrégulières: Des équations linéaires Poincaré, H. Acta Mathematica, Vol. 8, Issue 0 https://doi.org/10.1007/BF02417092	journal	January 1886
Sur les fonctions convexes et les inégalités entre les valeurs moyennes Jensen, J. L. W. V. Acta Mathematica, Vol. 30, Issue 0 https://doi.org/10.1007/BF02418571	journal	January 1906
Asymptotic mean efficiency of a selection of regression variables Shibata, Ritei Annals of the Institute of Statistical Mathematics, Vol. 35, Issue 3 https://doi.org/10.1007/BF02480998	journal	December 1983
Relative scales of the GUT and twin sectors in an F-theory model Clemens, C. Herbert; Raby, Stuart Journal of High Energy Physics, Vol. 2020, Issue 4 https://doi.org/10.1007/JHEP04(2020)004	journal	April 2020
Understanding predictive information criteria for Bayesian models Gelman, Andrew; Hwang, Jessica; Vehtari, Aki Statistics and Computing, Vol. 24, Issue 6 https://doi.org/10.1007/s11222-013-9416-2	journal	August 2013
The Jeffreys–Lindley paradox and discovery criteria in high energy physics Cousins, Robert D. Synthese, Vol. 194, Issue 2 https://doi.org/10.1007/s11229-014-0525-z	journal	July 2014
A new algorithm for adaptive multidimensional integration Peter Lepage, G. Journal of Computational Physics, Vol. 27, Issue 2 https://doi.org/10.1016/0021-9991(78)90004-9	journal	May 1978
A well-conditioned estimator for large-dimensional covariance matrices Ledoit, Olivier; Wolf, Michael Journal of Multivariate Analysis, Vol. 88, Issue 2 https://doi.org/10.1016/S0047-259X(03)00096-4	journal	February 2004
Constrained curve fitting Lepage, G. P.; Clark, B.; Davies, C. T. H. Nuclear Physics B - Proceedings Supplements, Vol. 106-107 https://doi.org/10.1016/S0920-5632(01)01638-3	journal	March 2002
Bayesian methods for parameter estimation in effective field theories Schindler, M. R.; Phillips, D. R. Annals of Physics, Vol. 324, Issue 3 https://doi.org/10.1016/j.aop.2008.09.003	journal	March 2009
Erratum to “Bayesian methods for parameter estimation in effective field theories” [Ann. Phys. 324 (2009) 682–708] Schindler, M. R.; Phillips, D. R. Annals of Physics, Vol. 324, Issue 9 https://doi.org/10.1016/j.aop.2009.05.007	journal	September 2009
Adaptive multidimensional integration: vegas enhanced Lepage, G. Peter Journal of Computational Physics, Vol. 439 https://doi.org/10.1016/j.jcp.2021.110386	journal	August 2021
A high-bias, low-variance introduction to Machine Learning for physicists Mehta, Pankaj; Bukov, Marin; Wang, Ching-Hao Physics Reports, Vol. 810 https://doi.org/10.1016/j.physrep.2019.03.001	journal	May 2019
Hadamard Expansions and Hyperasymptotic Evaluation: An Extension of the Method of Steepest Descents Paris, R. B. https://doi.org/10.1017/CBO9780511753626	book	January 2011
The Devil's Invention: Asymptotic, Superasymptotic and Hyperasymptotic Series Boyd, John P. Acta Applicandae Mathematicae, Vol. 56, Issue 1, p. 1-98 https://doi.org/10.1023/A:1006145903624	journal	March 1999
Model selection and psychological theory: A discussion of the differences between the Akaike information criterion (AIC) and the Bayesian information criterion (BIC). Vrieze, Scott I. Psychological Methods, Vol. 17, Issue 2 https://doi.org/10.1037/a0027127	journal	January 2012
A per-cent-level determination of the nucleon axial coupling from quantum chromodynamics Chang, C. C.; Nicholson, A. N.; Rinaldi, E. Nature, Vol. 558, Issue 7708 https://doi.org/10.1038/s41586-018-0161-8	journal	May 2018
Leading hadronic contribution to the muon magnetic moment from lattice QCD Borsanyi, Sz.; Fodor, Z.; Guenther, J. N. Nature, Vol. 593, Issue 7857 https://doi.org/10.1038/s41586-021-03418-1	journal	April 2021
Why your model parameter confidences might be too optimistic. Unbiased estimation of the inverse covariance matrix Hartlap, J.; Simon, P.; Schneider, P. Astronomy & Astrophysics, Vol. 464, Issue 1 https://doi.org/10.1051/0004-6361:20066170	journal	December 2006
Accurate Approximations for Posterior Moments and Marginal Densities Tierney, Luke; Kadane, Joseph B. Journal of the American Statistical Association, Vol. 81, Issue 393 https://doi.org/10.1080/01621459.1986.10478240	journal	March 1986
Model Selection and Accounting for Model Uncertainty in Graphical Models Using Occam's Window Madigan, David; Raftery, Adrian E. Journal of the American Statistical Association, Vol. 89, Issue 428 https://doi.org/10.1080/01621459.1994.10476894	journal	December 1994
Bayes Factors Kass, Robert E.; Raftery, Adrian E. Journal of the American Statistical Association, Vol. 90, Issue 430 https://doi.org/10.1080/01621459.1995.10476572	journal	June 1995
Further analysts of the data by akaike' s information criterion and the finite corrections: Further analysts of the data by akaike' s Sugiura, Nariaki Communications in Statistics - Theory and Methods, Vol. 7, Issue 1 https://doi.org/10.1080/03610927808827599	journal	January 1978
Information criteria for the predictive evaluation of bayesian models Genshiro, Kitagawa Communications in Statistics - Theory and Methods, Vol. 26, Issue 9 https://doi.org/10.1080/03610929708832043	journal	January 1997
Bayesian inference in physics: case studies Dose, V. Reports on Progress in Physics, Vol. 66, Issue 9 https://doi.org/10.1088/0034-4885/66/9/202	journal	August 2003
Get on the BAND Wagon: a Bayesian framework for quantifying model uncertainties in nuclear dynamics Phillips, D. R.; Furnstahl, R. J.; Heinz, U. Journal of Physics G: Nuclear and Particle Physics, Vol. 48, Issue 7 https://doi.org/10.1088/1361-6471/abf1df	journal	May 2021
Does Bayesian model averaging improve polynomial extrapolations? Two toy problems as tests Connell, M. A.; Billig, I.; Phillips, D. R. Journal of Physics G: Nuclear and Particle Physics, Vol. 48, Issue 10 https://doi.org/10.1088/1361-6471/ac215a	journal	September 2021
A Statistical Paradox Lindley, D. V. Biometrika, Vol. 44, Issue 1-2 https://doi.org/10.1093/biomet/44.1-2.187	journal	January 1957
Regression and time series model selection in small samples Hurvich, Clifford M.; Tsai, Chih-Ling Biometrika, Vol. 76, Issue 2 https://doi.org/10.1093/biomet/76.2.297	journal	January 1989
Generalised information criteria in model selection Konishi, S. Biometrika, Vol. 83, Issue 4 https://doi.org/10.1093/biomet/83.4.875	journal	December 1996
Bayesian predictive information criterion for the evaluation of hierarchical Bayesian and empirical Bayes models Ando, T. Biometrika, Vol. 94, Issue 2 https://doi.org/10.1093/biomet/asm017	journal	February 2007
Theory of Probability Jeffreys, Harold https://doi.org/10.1093/oso/9780198503682.001.0001	book	August 1998
Hyperasymptotics for Integrals with Saddles Berry, M. V.; Howls, C. J. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 434, Issue 1892 https://doi.org/10.1098/rspa.1991.0119	journal	September 1991
Hyperasymptotics for multidimensional integrals, exact remainder terms and the global connection problem Howls, C. J. Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences, Vol. 453, Issue 1966 https://doi.org/10.1098/rspa.1997.0122	journal	November 1997
On optimal truncation of divergent series solutions of nonlinear differential systems Costin, Ovidiu; Kruskal, Martin D. Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences, Vol. 455, Issue 1985 https://doi.org/10.1098/rspa.1999.0387	journal	May 1999
F K / F π from Möbius domain-wall fermions solved on gradient-flowed HISQ ensembles Miller, Nolan; Monge-Camacho, Henry; Chang, Chia Cheng Physical Review D, Vol. 102, Issue 3 https://doi.org/10.1103/PhysRevD.102.034507	journal	August 2020
Charged multihadron systems in lattice QCD + QED Beane, S. R.; Detmold, W.; Horsley, R. Physical Review D, Vol. 103, Issue 5 https://doi.org/10.1103/PhysRevD.103.054504	journal	March 2021
Bayesian model averaging for analysis of lattice field theory results Jay, William I.; Neil, Ethan T. Physical Review D, Vol. 103, Issue 11 https://doi.org/10.1103/PhysRevD.103.114502	journal	June 2021
D -meson semileptonic decays to pseudoscalars from four-flavor lattice QCD Bazavov, Alexei; DeTar, Carleton; El-Khadra, Aida X. Physical Review D, Vol. 107, Issue 9 https://doi.org/10.1103/PhysRevD.107.094516	journal	May 2023
Update: Accurate determinations of α s from realistic lattice QCD Davies, C. T. H.; Hornbostel, K.; Kendall, I. D. Physical Review D, Vol. 78, Issue 11 https://doi.org/10.1103/PhysRevD.78.114507	journal	December 2008
Sign problems, noise, and chiral symmetry breaking in a QCD-like theory Grabowska, Dorota; Kaplan, David B.; Nicholson, Amy N. Physical Review D, Vol. 87, Issue 1 https://doi.org/10.1103/PhysRevD.87.014504	journal	January 2013
Lattice QCD at the physical point meets S U ( 2 ) chiral perturbation theory Dürr, Stephan; Fodor, Zoltán; Hoelbling, Christian Physical Review D, Vol. 90, Issue 11 https://doi.org/10.1103/PhysRevD.90.114504	journal	December 2014
B ( s ) 0 -mixing matrix elements from lattice QCD for the Standard Model and beyond Bazavov, A.; Bernard, C.; Bouchard, C. M. Physical Review D, Vol. 93, Issue 11 https://doi.org/10.1103/PhysRevD.93.113016	journal	June 2016
Statistics of baryon correlation functions in lattice QCD Wagman, Michael L.; Savage, Martin J. Physical Review D, Vol. 96, Issue 11 https://doi.org/10.1103/PhysRevD.96.114508	journal	December 2017
Nonperturbative investigations of SU(3) gauge theory with eight dynamical flavors Appelquist, T.; Brower, R. C.; Fleming, G. T. Physical Review D, Vol. 99, Issue 1 https://doi.org/10.1103/PhysRevD.99.014509	journal	January 2019
Lattice QCD determination of neutron-antineutron matrix elements with physical quark masses Rinaldi, Enrico; Syritsyn, Sergey; Wagman, Michael L. Physical Review D, Vol. 99, Issue 7 https://doi.org/10.1103/PhysRevD.99.074510	journal	April 2019
Model averaging approaches to data subset selection Neil, Ethan T.; Sitison, Jacob W. Physical Review E, Vol. 108, Issue 4 https://doi.org/10.1103/PhysRevE.108.045308	journal	October 2023
A new look at the statistical model identification Akaike, H. IEEE Transactions on Automatic Control, Vol. 19, Issue 6 https://doi.org/10.1109/TAC.1974.1100705	journal	December 1974
Bayesian measures of model complexity and fit Spiegelhalter, David J.; Best, Nicola G.; Carlin, Bradley P. Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 64, Issue 4 https://doi.org/10.1111/1467-9868.00353	journal	October 2002
Bayesian Model Averaging: A Systematic Review and Conceptual Classification: BMA: A Systematic Review Fragoso, Tiago M.; Bertoli, Wesley; Louzada, Francisco International Statistical Review, Vol. 86, Issue 1 https://doi.org/10.1111/insr.12243	journal	December 2017
Philosophy and the practice of Bayesian statistics: Philosophy and the practice of Bayesian statistics Gelman, Andrew; Shalizi, Cosma Rohilla British Journal of Mathematical and Statistical Psychology, Vol. 66, Issue 1 https://doi.org/10.1111/j.2044-8317.2011.02037.x	journal	February 2012
An Asymptotic Equivalence of Choice of Model by Cross-Validation and Akaike's Criterion Stone, M. Journal of the Royal Statistical Society: Series B (Methodological), Vol. 39, Issue 1 https://doi.org/10.1111/j.2517-6161.1977.tb01603.x	journal	September 1977
Ab initio calculation of the neutron-proton mass difference Borsanyi, S.; Durr, S.; Fodor, Z. Science, Vol. 347, Issue 6229 https://doi.org/10.1126/science.1257050	journal	March 2015
Applied Analysis Hunter, John K.; Nachtergaele, Bruno https://doi.org/10.1142/4319	journal	February 2001
Discussion: Performance of Bayesian Model Averaging Raftery, Adrian E.; Zheng, Yingye Journal of the American Statistical Association, Vol. 98, Issue 464 https://doi.org/10.1198/016214503000000891	journal	December 2003
Nonlinear shrinkage estimation of large-dimensional covariance matrices Ledoit, Olivier; Wolf, Michael The Annals of Statistics, Vol. 40, Issue 2 https://doi.org/10.1214/12-AOS989	journal	April 2012
The Expected Coverage to the Left of the $i$th Order Statistic for Arbitary Distributions Margolin, Barry H.; Mosteller, Frederick The Annals of Mathematical Statistics, Vol. 40, Issue 2 https://doi.org/10.1214/aoms/1177697732	journal	April 1969
On Information and Sufficiency Kullback, S.; Leibler, R. A. The Annals of Mathematical Statistics, Vol. 22, Issue 1 https://doi.org/10.1214/aoms/1177729694	journal	March 1951
Estimating the Dimension of a Model Schwarz, Gideon The Annals of Statistics, Vol. 6, Issue 2 https://doi.org/10.1214/aos/1176344136	journal	March 1978
Asymptotic Theory of Nonlinear Least Squares Estimation Wu, Chien-Fu The Annals of Statistics, Vol. 9, Issue 3 https://doi.org/10.1214/aos/1176345455	journal	May 1981
Bayesian Model Averaging and Bayesian Predictive Information Criterion for Model Selection Ando, Tomohiro JOURNAL OF THE JAPAN STATISTICAL SOCIETY, Vol. 38, Issue 2 https://doi.org/10.14490/jjss.38.243	journal	January 2008
Maximum Likelihood Estimation of Misspecified Models White, Halbert Econometrica, Vol. 50, Issue 1 https://doi.org/10.2307/1912526	journal	January 1982
Bayesian Methods in Practice: Experiences in the Pharmaceutical Industry Racine, A.; Grieve, A. P.; Fluhler, H. Applied Statistics, Vol. 35, Issue 2 https://doi.org/10.2307/2347264	journal	January 1986
Model Selection for Extended Quasi-Likelihood Models in Small Samples Hurvich, Clifford M.; Tsai, Chih-Ling Biometrics, Vol. 51, Issue 3 https://doi.org/10.2307/2533006	journal	September 1995
Higher asymptotics of Laplace's approximation Kirwin, William D. Asymptotic Analysis, Vol. 70, Issue 3-4 https://doi.org/10.3233/ASY-2010-1016	journal	January 2010
Information-Corrected Estimation: A Generalization Error Reducing Parameter Estimation Method Dixon, Matthew; Ward, Tyler Entropy, Vol. 23, Issue 11 https://doi.org/10.3390/e23111419	journal	October 2021

Similar Records

MMA, A Computer Code for Multi-Model Analysis

Technical Report · Mon Aug 20 00:00:00 EDT 2007 · OSTI ID:920086

Effects of error covariance structure on estimation of model averaging weights and predictive performance

Journal Article · Tue Jul 23 00:00:00 EDT 2013 · Water Resources Research, 49(9):6029-6047 · OSTI ID:1132712

Model averaging approaches to data subset selection

Journal Article · Sun Oct 22 20:00:00 EDT 2023 · Physical Review. E · OSTI ID:2223083

Related Subjects

97 MATHEMATICS AND COMPUTING

Improved information criteria for Bayesian model averaging in lattice field theory

Citation Formats

References (71)

Similar Records

Related Subjects