Evaluating marginal likelihood with thermodynamic integration method and comparison with several other numerical methods
Evaluating marginal likelihood is the most critical and computationally expensive task, when conducting Bayesian model averaging to quantify parametric and model uncertainties. The evaluation is commonly done by using Laplace approximations to evaluate semianalytical expressions of the marginal likelihood or by using Monte Carlo (MC) methods to evaluate arithmetic or harmonic mean of a joint likelihood function. This study introduces a new MC method, i.e., thermodynamic integration, which has not been attempted in environmental modeling. Instead of using samples only from prior parameter space (as in arithmetic mean evaluation) or posterior parameter space (as in harmonic mean evaluation), the thermodynamic integration method uses samples generated gradually from the prior to posterior parameter space. This is done through a path sampling that conducts Markov chain Monte Carlo simulation with different power coefficient values applied to the joint likelihood function. The thermodynamic integration method is evaluated using three analytical functions by comparing the method with two variants of the Laplace approximation method and three MC methods, including the nested sampling method that is recently introduced into environmental modeling. The thermodynamic integration method outperforms the other methods in terms of their accuracy, convergence, and consistency. The thermodynamic integration method is also applied tomore »
 Authors:

^{[1]};
^{[2]};
^{[2]};
^{[2]};
^{[3]};
^{[4]};
^{[5]}
 Hefei Univ. of Technology, Hefei (China); Florida State Univ., Tallahassee, FL (United States)
 Florida State Univ., Tallahassee, FL (United States)
 Nanjing Univ., Nanjing (China)
 Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
 Hefei Univ. of Technology, Hefei (China)
 Publication Date:
 Grant/Contract Number:
 AC0500OR22725; SC0008272
 Type:
 Accepted Manuscript
 Journal Name:
 Water Resources Research
 Additional Journal Information:
 Journal Volume: 52; Journal Issue: 2; Journal ID: ISSN 00431397
 Publisher:
 American Geophysical Union (AGU)
 Research Org:
 Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
 Sponsoring Org:
 USDOE
 Country of Publication:
 United States
 Language:
 English
 Subject:
 97 MATHEMATICS AND COMPUTING; Bayesian model averaging; arithmetic mean; harmonic mean; nested sampling; model uncertainty; Markov chain Monte Carlo
 OSTI Identifier:
 1326496
 Alternate Identifier(s):
 OSTI ID: 1402380
Liu, Peigui, Elshall, Ahmed S., Ye, Ming, Beerli, Peter, Zeng, Xiankui, Lu, Dan, and Tao, Yuezan. Evaluating marginal likelihood with thermodynamic integration method and comparison with several other numerical methods. United States: N. p.,
Web. doi:10.1002/2014WR016718.
Liu, Peigui, Elshall, Ahmed S., Ye, Ming, Beerli, Peter, Zeng, Xiankui, Lu, Dan, & Tao, Yuezan. Evaluating marginal likelihood with thermodynamic integration method and comparison with several other numerical methods. United States. doi:10.1002/2014WR016718.
Liu, Peigui, Elshall, Ahmed S., Ye, Ming, Beerli, Peter, Zeng, Xiankui, Lu, Dan, and Tao, Yuezan. 2016.
"Evaluating marginal likelihood with thermodynamic integration method and comparison with several other numerical methods". United States.
doi:10.1002/2014WR016718. https://www.osti.gov/servlets/purl/1326496.
@article{osti_1326496,
title = {Evaluating marginal likelihood with thermodynamic integration method and comparison with several other numerical methods},
author = {Liu, Peigui and Elshall, Ahmed S. and Ye, Ming and Beerli, Peter and Zeng, Xiankui and Lu, Dan and Tao, Yuezan},
abstractNote = {Evaluating marginal likelihood is the most critical and computationally expensive task, when conducting Bayesian model averaging to quantify parametric and model uncertainties. The evaluation is commonly done by using Laplace approximations to evaluate semianalytical expressions of the marginal likelihood or by using Monte Carlo (MC) methods to evaluate arithmetic or harmonic mean of a joint likelihood function. This study introduces a new MC method, i.e., thermodynamic integration, which has not been attempted in environmental modeling. Instead of using samples only from prior parameter space (as in arithmetic mean evaluation) or posterior parameter space (as in harmonic mean evaluation), the thermodynamic integration method uses samples generated gradually from the prior to posterior parameter space. This is done through a path sampling that conducts Markov chain Monte Carlo simulation with different power coefficient values applied to the joint likelihood function. The thermodynamic integration method is evaluated using three analytical functions by comparing the method with two variants of the Laplace approximation method and three MC methods, including the nested sampling method that is recently introduced into environmental modeling. The thermodynamic integration method outperforms the other methods in terms of their accuracy, convergence, and consistency. The thermodynamic integration method is also applied to a synthetic case of groundwater modeling with four alternative models. The application shows that model probabilities obtained using the thermodynamic integration method improves predictive performance of Bayesian model averaging. As a result, the thermodynamic integration method is mathematically rigorous, and its MC implementation is computationally general for a wide range of environmental problems.},
doi = {10.1002/2014WR016718},
journal = {Water Resources Research},
number = 2,
volume = 52,
place = {United States},
year = {2016},
month = {2}
}