Accelerated DimensionIndependent Adaptive Metropolis
This work describes improvements from algorithmic and architectural means to blackbox Bayesian inference over highdimensional parameter spaces. The wellknown adaptive Metropolis (AM) algorithm [33] is extended herein to scale asymptotically uniformly with respect to the underlying parameter dimension for Gaussian targets, by respecting the variance of the target. The resulting algorithm, referred to as the dimensionindependent adaptive Metropolis (DIAM) algorithm, also shows improved performance with respect to adaptive Metropolis on nonGaussian targets. This algorithm is further improved, and the possibility of probing highdimensional (with dimension d 1000) targets is enabled, via GPUaccelerated numerical libraries and periodically synchronized concurrent chains (justi ed a posteriori). Asymptotically in dimension, this GPU implementation exhibits a factor of four improvement versus a competitive CPUbased Intel MKL parallel version alone. Strong scaling to concurrent chains is exhibited, through a combination of longer time per sample batch (weak scaling) and yet fewer necessary samples to convergence. The algorithm performance is illustrated on several Gaussian and nonGaussian target examples, in which the dimension may be in excess of one thousand.
 Authors:

^{[1]};
^{[2]};
^{[3]};
^{[4]}
 King Abdullah University of Science and Technology (KAUST), Thuwal (Saudi Arabia)
 Columbia Univ., New York, NY (United States)
 Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
 King Abdullah University of Science and Technology (KAUST), Thuwal, Kingdom of Saudi Arabia
 Publication Date:
 Grant/Contract Number:
 AC0500OR22725
 Type:
 Accepted Manuscript
 Journal Name:
 SIAM Journal on Scientific Computing
 Additional Journal Information:
 Journal Volume: 38; Journal Issue: 5; Journal ID: ISSN 10648275
 Publisher:
 SIAM
 Research Org:
 Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
 Sponsoring Org:
 USDOE
 Country of Publication:
 United States
 Language:
 English
 Subject:
 97 MATHEMATICS AND COMPUTING; Markov chain Monte Carlo; big data; Bayesian inference; adaptive Metropolis; MetropolisHastings; BLAS; GPU acceleration; High performance computing
 OSTI Identifier:
 1346642
Chen, Yuxin, Keyes, David E., Law, Kody J., and Ltaief, Hatem. Accelerated DimensionIndependent Adaptive Metropolis. United States: N. p.,
Web. doi:10.1137/15M1026432.
Chen, Yuxin, Keyes, David E., Law, Kody J., & Ltaief, Hatem. Accelerated DimensionIndependent Adaptive Metropolis. United States. doi:10.1137/15M1026432.
Chen, Yuxin, Keyes, David E., Law, Kody J., and Ltaief, Hatem. 2016.
"Accelerated DimensionIndependent Adaptive Metropolis". United States.
doi:10.1137/15M1026432. https://www.osti.gov/servlets/purl/1346642.
@article{osti_1346642,
title = {Accelerated DimensionIndependent Adaptive Metropolis},
author = {Chen, Yuxin and Keyes, David E. and Law, Kody J. and Ltaief, Hatem},
abstractNote = {This work describes improvements from algorithmic and architectural means to blackbox Bayesian inference over highdimensional parameter spaces. The wellknown adaptive Metropolis (AM) algorithm [33] is extended herein to scale asymptotically uniformly with respect to the underlying parameter dimension for Gaussian targets, by respecting the variance of the target. The resulting algorithm, referred to as the dimensionindependent adaptive Metropolis (DIAM) algorithm, also shows improved performance with respect to adaptive Metropolis on nonGaussian targets. This algorithm is further improved, and the possibility of probing highdimensional (with dimension d 1000) targets is enabled, via GPUaccelerated numerical libraries and periodically synchronized concurrent chains (justi ed a posteriori). Asymptotically in dimension, this GPU implementation exhibits a factor of four improvement versus a competitive CPUbased Intel MKL parallel version alone. Strong scaling to concurrent chains is exhibited, through a combination of longer time per sample batch (weak scaling) and yet fewer necessary samples to convergence. The algorithm performance is illustrated on several Gaussian and nonGaussian target examples, in which the dimension may be in excess of one thousand.},
doi = {10.1137/15M1026432},
journal = {SIAM Journal on Scientific Computing},
number = 5,
volume = 38,
place = {United States},
year = {2016},
month = {10}
}