Ensemble preconditioning for Markov chain Monte Carlo simulation
Abstract
We describe parallel Markov chain Monte Carlo methods that propagate a collective ensemble of paths, with local covariance information calculated from neighbouring replicas. Here, the use of collective dynamics eliminates multiplicative noise and stabilizes the dynamics, thus providing a practical approach to difficult anisotropic sampling problems in high dimensions. Numerical experiments with model problems demonstrate that dramatic potential speedups, compared to various alternative schemes, are attainable.
- Authors:
- Publication Date:
- Research Org.:
- Univ. of Chicago, IL (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
- OSTI Identifier:
- 1344938
- Alternate Identifier(s):
- OSTI ID: 1502101
- Grant/Contract Number:
- SC0014205
- Resource Type:
- Published Article
- Journal Name:
- Statistics and Computing
- Additional Journal Information:
- Journal Name: Statistics and Computing Journal Volume: 28 Journal Issue: 2; Journal ID: ISSN 0960-3174
- Publisher:
- Springer Science + Business Media
- Country of Publication:
- Germany
- Language:
- English
- Subject:
- 97 MATHEMATICS AND COMPUTING; Stochastic sampling; Markov chain Monte Carlo; MCMC; Computational statistics; Machine learning; BFGS; Langevin methods; Brownian dynamics
Citation Formats
Leimkuhler, Benedict, Matthews, Charles, and Weare, Jonathan. Ensemble preconditioning for Markov chain Monte Carlo simulation. Germany: N. p., 2017.
Web. doi:10.1007/s11222-017-9730-1.
Leimkuhler, Benedict, Matthews, Charles, & Weare, Jonathan. Ensemble preconditioning for Markov chain Monte Carlo simulation. Germany. https://doi.org/10.1007/s11222-017-9730-1
Leimkuhler, Benedict, Matthews, Charles, and Weare, Jonathan. Mon .
"Ensemble preconditioning for Markov chain Monte Carlo simulation". Germany. https://doi.org/10.1007/s11222-017-9730-1.
@article{osti_1344938,
title = {Ensemble preconditioning for Markov chain Monte Carlo simulation},
author = {Leimkuhler, Benedict and Matthews, Charles and Weare, Jonathan},
abstractNote = {We describe parallel Markov chain Monte Carlo methods that propagate a collective ensemble of paths, with local covariance information calculated from neighbouring replicas. Here, the use of collective dynamics eliminates multiplicative noise and stabilizes the dynamics, thus providing a practical approach to difficult anisotropic sampling problems in high dimensions. Numerical experiments with model problems demonstrate that dramatic potential speedups, compared to various alternative schemes, are attainable.},
doi = {10.1007/s11222-017-9730-1},
journal = {Statistics and Computing},
number = 2,
volume = 28,
place = {Germany},
year = {Mon Feb 27 00:00:00 EST 2017},
month = {Mon Feb 27 00:00:00 EST 2017}
}
Free Publicly Available Full Text
Publisher's Version of Record
https://doi.org/10.1007/s11222-017-9730-1
https://doi.org/10.1007/s11222-017-9730-1
Other availability
Cited by: 21 works
Citation information provided by
Web of Science
Web of Science
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
Ensemble samplers with affine invariance
journal, January 2010
- Goodman, Jonathan; Weare, Jonathan
- Communications in Applied Mathematics and Computational Science, Vol. 5, Issue 1
Population Monte Carlo
journal, December 2004
- Cappé, O.; Guillin, A.; Marin, J. M.
- Journal of Computational and Graphical Statistics, Vol. 13, Issue 4
Free energy methods for Bayesian inference: efficient exploration of univariate Gaussian mixture posteriors
journal, June 2011
- Chopin, Nicolas; Lelièvre, Tony; Stoltz, Gabriel
- Statistics and Computing, Vol. 22, Issue 4
Scaling limits for the transient phase of local Metropolis-Hastings algorithms
journal, April 2005
- Christensen, Ole F.; Roberts, Gareth O.; Rosenthal, Jeffrey S.
- Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 67, Issue 2
Hybrid Monte Carlo
journal, September 1987
- Duane, Simon; Kennedy, A. D.; Pendleton, Brian J.
- Physics Letters B, Vol. 195, Issue 2
Accelerating diffusions
journal, May 2005
- Hwang, Chii-Ruey; Hwang-Ma, Shu-Yin; Sheu, Shuenn-Jyi
- The Annals of Applied Probability, Vol. 15, Issue 2
Philatelic Mixtures and Multimodal Densities
journal, December 1988
- Izenman, Alan J.; Sommer, Charles J.
- Journal of the American Statistical Association, Vol. 83, Issue 404
A general purpose sampling algorithm for continuous distributions (the t-walk)
journal, June 2010
- Christen, J. Andrés; Fox, Colin
- Bayesian Analysis, Vol. 5, Issue 2
On population-based simulation for static inference
journal, July 2007
- Jasra, Ajay; Stephens, David A.; Holmes, Christopher C.
- Statistics and Computing, Vol. 17, Issue 3
A Stochastic Newton MCMC Method for Large-Scale Statistical Inverse Problems with Application to Seismic Inversion
journal, January 2012
- Martin, James; Wilcox, Lucas C.; Burstedde, Carsten
- SIAM Journal on Scientific Computing, Vol. 34, Issue 3
Population Monte Carlo algorithms.
journal, January 2001
- Iba, Yukito
- Transactions of the Japanese Society for Artificial Intelligence, Vol. 16
Geometric convergence and central limit theorems for multidimensional Hastings and Metropolis algorithms
journal, March 1996
- Roberts, G.
- Biometrika, Vol. 83, Issue 1
Irreversible Langevin samplers and variance reduction: a large deviations approach
journal, May 2015
- Rey-Bellet, Luc; Spiliopoulos, Konstantinos
- Nonlinearity, Vol. 28, Issue 7
emcee : The MCMC Hammer
journal, March 2013
- Foreman-Mackey, Daniel; Hogg, David W.; Lang, Dustin
- Publications of the Astronomical Society of the Pacific, Vol. 125, Issue 925
Multigrid Monte Carlo method. Conceptual foundations
journal, September 1989
- Goodman, Jonathan; Sokal, Alan D.
- Physical Review D, Vol. 40, Issue 6
Theoretical guarantees for approximate sampling from smooth and log-concave densities
journal, April 2016
- Dalalyan, Arnak S.
- Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 79, Issue 3
Brownian dynamics as smart Monte Carlo simulation
journal, November 1978
- Rossky, P. J.; Doll, J. D.; Friedman, H. L.
- The Journal of Chemical Physics, Vol. 69, Issue 10
Monte Carlo Calculation of the Average Extension of Molecular Chains
journal, February 1955
- Rosenbluth, Marshall N.; Rosenbluth, Arianna W.
- The Journal of Chemical Physics, Vol. 23, Issue 2
Metropolis Integration Schemes for Self-Adjoint Diffusions
journal, January 2014
- Bou-Rabee, Nawaf; Donev, Aleksandar; Vanden-Eijnden, Eric
- Multiscale Modeling & Simulation, Vol. 12, Issue 2
Stochastic Processes and Applications
book, January 2014
- Pavliotis, Grigorios A.
- Texts in Applied Mathematics
Variance Reduction Using Nonreversible Langevin Samplers
journal, March 2016
- Duncan, A. B.; Lelièvre, T.; Pavliotis, G. A.
- Journal of Statistical Physics, Vol. 163, Issue 3
An Adaptive Metropolis Algorithm
journal, April 2001
- Haario, Heikki; Saksman, Eero; Tamminen, Johanna
- Bernoulli, Vol. 7, Issue 2
The computation of averages from equilibrium and nonequilibrium Langevin molecular dynamics
journal, January 2015
- Leimkuhler, Benedict; Matthews, Charles; Stoltz, Gabriel
- IMA Journal of Numerical Analysis
Riemann manifold Langevin and Hamiltonian Monte Carlo methods: Riemann Manifold Langevin and Hamiltonian Monte Carlo Methods
journal, March 2011
- Girolami, Mark; Calderhead, Ben
- Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 73, Issue 2
Improved Diffusion Monte Carlo
journal, June 2014
- Hairer, Martin; Weare, Jonathan
- Communications on Pure and Applied Mathematics, Vol. 67, Issue 12
Faster estimation of Bayesian models in ecology using Hamiltonian Monte Carlo
journal, November 2016
- Monnahan, Cole C.; Thorson, James T.; Branch, Trevor A.
- Methods in Ecology and Evolution, Vol. 8, Issue 3
Works referencing / citing this record:
A new bi‐fidelity model reduction method for Bayesian inverse problems
journal, May 2019
- Ou, Na; Jiang, Lijian; Lin, Guang
- International Journal for Numerical Methods in Engineering, Vol. 119, Issue 10