Accelerated Dimension-Independent Adaptive Metropolis

Chen, Yuxin; Keyes, David E.; Law, Kody J.; Ltaief, Hatem

doi:10.1137/15M1026432

Title: Accelerated Dimension-Independent Adaptive Metropolis

Full Record
Other Related Research

Abstract

This work describes improvements from algorithmic and architectural means to black-box Bayesian inference over high-dimensional parameter spaces. The well-known adaptive Metropolis (AM) algorithm [33] is extended herein to scale asymptotically uniformly with respect to the underlying parameter dimension for Gaussian targets, by respecting the variance of the target. The resulting algorithm, referred to as the dimension-independent adaptive Metropolis (DIAM) algorithm, also shows improved performance with respect to adaptive Metropolis on non-Gaussian targets. This algorithm is further improved, and the possibility of probing high-dimensional (with dimension d 1000) targets is enabled, via GPU-accelerated numerical libraries and periodically synchronized concurrent chains (justi ed a posteriori). Asymptotically in dimension, this GPU implementation exhibits a factor of four improvement versus a competitive CPU-based Intel MKL parallel version alone. Strong scaling to concurrent chains is exhibited, through a combination of longer time per sample batch (weak scaling) and yet fewer necessary samples to convergence. The algorithm performance is illustrated on several Gaussian and non-Gaussian target examples, in which the dimension may be in excess of one thousand.

Authors:

Chen, Yuxin ^[1]; Keyes, David E. ^[2]; Law, Kody J. ^[3]; Ltaief, Hatem ^[4]

King Abdullah University of Science and Technology (KAUST), Thuwal (Saudi Arabia)
Columbia Univ., New York, NY (United States)
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
King Abdullah University of Science and Technology (KAUST), Thuwal, Kingdom of Saudi Arabia

Publication Date:: Thu Oct 27 00:00:00 EDT 2016

Research Org.:: Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)

Sponsoring Org.:: USDOE

OSTI Identifier:: 1346642

Grant/Contract Number:: AC05-00OR22725

Resource Type:: Accepted Manuscript

Journal Name:: SIAM Journal on Scientific Computing

Additional Journal Information:: Journal Volume: 38; Journal Issue: 5; Journal ID: ISSN 1064-8275

Publisher:: SIAM

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING; Markov chain Monte Carlo; big data; Bayesian inference; adaptive Metropolis; Metropolis-Hastings; BLAS; GPU acceleration; High performance computing

Citation Formats


                    Chen, Yuxin, Keyes, David E., Law, Kody J., and Ltaief, Hatem. Accelerated Dimension-Independent Adaptive Metropolis.  United States: N. p., 2016. 
Web.  doi:10.1137/15M1026432.

Copy to clipboard


                    Chen, Yuxin, Keyes, David E., Law, Kody J., & Ltaief, Hatem. Accelerated Dimension-Independent Adaptive Metropolis.  United States.  https://doi.org/10.1137/15M1026432

Copy to clipboard


                    Chen, Yuxin, Keyes, David E., Law, Kody J., and Ltaief, Hatem. Thu .  
"Accelerated Dimension-Independent Adaptive Metropolis".  United States.  https://doi.org/10.1137/15M1026432.  https://www.osti.gov/servlets/purl/1346642.

Copy to clipboard


                    
@article{osti_1346642,

  title        = {Accelerated Dimension-Independent Adaptive Metropolis},

  author       = {Chen, Yuxin and Keyes, David E. and Law, Kody J. and Ltaief, Hatem},

  abstractNote = {This work describes improvements from algorithmic and architectural means to black-box Bayesian inference over high-dimensional parameter spaces. The well-known adaptive Metropolis (AM) algorithm [33] is extended herein to scale asymptotically uniformly with respect to the underlying parameter dimension for Gaussian targets, by respecting the variance of the target. The resulting algorithm, referred to as the dimension-independent adaptive Metropolis (DIAM) algorithm, also shows improved performance with respect to adaptive Metropolis on non-Gaussian targets. This algorithm is further improved, and the possibility of probing high-dimensional (with dimension d 1000) targets is enabled, via GPU-accelerated numerical libraries and periodically synchronized concurrent chains (justi ed a posteriori). Asymptotically in dimension, this GPU implementation exhibits a factor of four improvement versus a competitive CPU-based Intel MKL parallel version alone. Strong scaling to concurrent chains is exhibited, through a combination of longer time per sample batch (weak scaling) and yet fewer necessary samples to convergence. The algorithm performance is illustrated on several Gaussian and non-Gaussian target examples, in which the dimension may be in excess of one thousand.},

  doi          = {10.1137/15M1026432},

  journal      = {SIAM Journal on Scientific Computing},

  number       = 5,

  volume       = 38,

  place        = {United States},

  year         = {Thu Oct 27 00:00:00 EDT 2016},

  month        = {Thu Oct 27 00:00:00 EDT 2016}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1137/15M1026432

Other availability

Search WorldCat to find libraries that may hold this journal

Citation Metrics:

Cited by: 12 works

Citation information provided by
Web of Science

Save / Share:

Export Metadata

Save to My Library

Similar Records in DOE PAGES and OSTI.GOV collections:

The Metropolis Monte Carlo method with CUDA enabled Graphic Processing Units

Journal Article Hall, Clifford ; School of Physics, Astronomy, and Computational Sciences, George Mason University, 4400 University Dr., Fairfax, VA 22030 ; Ji, Weixiao ; ... - Journal of Computational Physics

We present a CPU–GPU system for runtime acceleration of large molecular simulations using GPU computation and memory swaps. The memory architecture of the GPU can be used both as container for simulation data stored on the graphics card and as floating-point code target, providing an effective means for the manipulation of atomistic or molecular data on the GPU. To fully take advantage of this mechanism, efficient GPU realizations of algorithms used to perform atomistic and molecular simulations are essential. Our system implements a versatile molecular engine, including inter-molecule interactions and orientational variables for performing the Metropolis Monte Carlo (MMC) algorithm,more »« less
https://doi.org/10.1016/J.JCP.2013.11.012
Tensor Contraction and Operation Minimization forExtreme Scale Computational Chemistry

Technical Report Sabin, Gerald ; Sadayappan, P.

During Phase 1, RNET and U. Utah accomplished four main tasks to demonstrate the technical and practical feasibility of the ES-SciLA tensor library. Evaluated the SpMM runtime performance using the KokkosKernels framework, i.e., the multi-vector 2D Tensor KokkosSparse :: spmv interface. KokkosKernels represents a high productivity library that is an important part of the DOE exascale platform. It provides shared memory performance portability across a range of hardware architectures and is a target for planned future integration is ES-SciLA. The results include matrices from DARPA’s SDNN Graph Challenge, a Transformer model [8], structural and power law matrices from the Suitemore »« less
Dimension-independent likelihood-informed MCMC

Journal Article Cui, Tiangang ; Law, Kody J.H., E-mail: kody.law@kaust.edu.sa ; Marzouk, Youssef M., E-mail: ymarz@mit.edu - Journal of Computational Physics

Many Bayesian inference problems require exploring the posterior distribution of high-dimensional parameters that represent the discretization of an underlying function. This work introduces a family of Markov chain Monte Carlo (MCMC) samplers that can adapt to the particular structure of a posterior distribution over functions. Two distinct lines of research intersect in the methods developed here. First, we introduce a general class of operator-weighted proposal distributions that are well defined on function space, such that the performance of the resulting MCMC samplers is independent of the discretization of the function. Second, by exploiting local Hessian information and any associated low-dimensionalmore »« less
https://doi.org/10.1016/J.JCP.2015.10.008
Dimension-independent likelihood-informed MCMC

Journal Article Cui, Tiangang ; Law, Kody J. H. ; Marzouk, Youssef M. - Journal of Computational Physics

Many Bayesian inference problems require exploring the posterior distribution of highdimensional parameters that represent the discretization of an underlying function. Our work introduces a family of Markov chain Monte Carlo (MCMC) samplers that can adapt to the particular structure of a posterior distribution over functions. There are two distinct lines of research that intersect in the methods we develop here. First, we introduce a general class of operator-weighted proposal distributions that are well defined on function space, such that the performance of the resulting MCMC samplers is independent of the discretization of the function. Second, by exploiting local Hessian informationmore »« less
Cited by 100
https://doi.org/10.1016/j.jcp.2015.10.008

Full Text Available
Closeout Report for DE-SC0018121

Technical Report Detmold, William

In order to add more capabilities to Halide, we have designed a new framework called Tiramisu and integrated this framework into Halide. Since Tiramisu enables Halide to target heterogeneous architectures, our development efforts have been refocused on Tiramisu. Most high-performance computer systems today are complex and increasingly heterogeneous; they may have CPUs, GPUs and FPGAs. Achieving best performance requires taking full advantage of all these different architectures. To address this issue, we have designed Tiramisu, an optimization framework that enables Halide (and other DSLs) to target heterogeneous architectures. Tiramisu is an optimization framework that takes as input a high level,more »« less
https://doi.org/10.2172/1971643

Full Text Available

Similar Records