Multilevel Sequential Monte Carlo with Dimension-Independent Likelihood-Informed Proposals
Abstract
In this article we develop a new sequential Monte Carlo method for multilevel Monte Carlo estimation. In particular, the method can be used to estimate expectations with respect to a target probability distribution over an infinite-dimensional and noncompact space---as produced, for example, by a Bayesian inverse problem with a Gaussian random field prior. Under suitable assumptions the MLSMC method has the optimal $$\mathcal{O}(\varepsilon^{-2})$$ bound on the cost to obtain a mean-square error of $$\mathcal{O}(\varepsilon^2)$$. The algorithm is accelerated by dimension-independent likelihood-informed proposals [T. Cui, K. J. Law, and Y. M. Marzouk, (2016), J. Comput. Phys., 304, pp. 109--137] designed for Gaussian priors, leveraging a novel variation which uses empirical covariance information in lieu of Hessian information, hence eliminating the requirement for gradient evaluations. The efficiency of the algorithm is illustrated on two examples: (i) inversion of noisy pressure measurements in a PDE model of Darcy flow to recover the posterior distribution of the permeability field and (ii) inversion of noisy measurements of the solution of an SDE to recover the posterior path measure.
- Authors:
-
- University College London (United Kingdom)
- National Univ. of Singapore (Singapore)
- Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
- Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States)
- Publication Date:
- Research Org.:
- Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
- Sponsoring Org.:
- USDOE Laboratory Directed Research and Development (LDRD) Program; USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
- OSTI Identifier:
- 1862172
- Grant/Contract Number:
- AC05-00OR22725; 32112580
- Resource Type:
- Accepted Manuscript
- Journal Name:
- SIAM/ASA Journal on Uncertainty Quantification
- Additional Journal Information:
- Journal Volume: 6; Journal Issue: 2; Journal ID: ISSN 2166-2525
- Publisher:
- Society for Industrial and Applied Mathematics (SIAM)
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 97 MATHEMATICS AND COMPUTING; multilevel Monte Carlo; sequential Monte Carlo; Bayesian inverse problem; uncertainty quantification
Citation Formats
Beskos, Alexandros, Jasra, Ajay, Law, Kody, Marzouk, Youssef, and Zhou, Yan. Multilevel Sequential Monte Carlo with Dimension-Independent Likelihood-Informed Proposals. United States: N. p., 2017.
Web. doi:10.1137/17m1120993.
Beskos, Alexandros, Jasra, Ajay, Law, Kody, Marzouk, Youssef, & Zhou, Yan. Multilevel Sequential Monte Carlo with Dimension-Independent Likelihood-Informed Proposals. United States. https://doi.org/10.1137/17m1120993
Beskos, Alexandros, Jasra, Ajay, Law, Kody, Marzouk, Youssef, and Zhou, Yan. Tue .
"Multilevel Sequential Monte Carlo with Dimension-Independent Likelihood-Informed Proposals". United States. https://doi.org/10.1137/17m1120993. https://www.osti.gov/servlets/purl/1862172.
@article{osti_1862172,
title = {Multilevel Sequential Monte Carlo with Dimension-Independent Likelihood-Informed Proposals},
author = {Beskos, Alexandros and Jasra, Ajay and Law, Kody and Marzouk, Youssef and Zhou, Yan},
abstractNote = {In this article we develop a new sequential Monte Carlo method for multilevel Monte Carlo estimation. In particular, the method can be used to estimate expectations with respect to a target probability distribution over an infinite-dimensional and noncompact space---as produced, for example, by a Bayesian inverse problem with a Gaussian random field prior. Under suitable assumptions the MLSMC method has the optimal $\mathcal{O}(\varepsilon^{-2})$ bound on the cost to obtain a mean-square error of $\mathcal{O}(\varepsilon^2)$. The algorithm is accelerated by dimension-independent likelihood-informed proposals [T. Cui, K. J. Law, and Y. M. Marzouk, (2016), J. Comput. Phys., 304, pp. 109--137] designed for Gaussian priors, leveraging a novel variation which uses empirical covariance information in lieu of Hessian information, hence eliminating the requirement for gradient evaluations. The efficiency of the algorithm is illustrated on two examples: (i) inversion of noisy pressure measurements in a PDE model of Darcy flow to recover the posterior distribution of the permeability field and (ii) inversion of noisy measurements of the solution of an SDE to recover the posterior path measure.},
doi = {10.1137/17m1120993},
journal = {SIAM/ASA Journal on Uncertainty Quantification},
number = 2,
volume = 6,
place = {United States},
year = {Tue Mar 14 00:00:00 EDT 2017},
month = {Tue Mar 14 00:00:00 EDT 2017}
}
Web of Science
Works referenced in this record:
Dimension-independent likelihood-informed MCMC
journal, January 2016
- Cui, Tiangang; Law, Kody J. H.; Marzouk, Youssef M.
- Journal of Computational Physics, Vol. 304
Rates of convergence of the Hastings and Metropolis algorithms
journal, February 1996
- Mengersen, K. L.; Tweedie, R. L.
- The Annals of Statistics, Vol. 24, Issue 1
Multilevel sequential Monte Carlo: Mean square error bounds under verifiable conditions
journal, December 2016
- Del Moral, Pierre; Jasra, Ajay; Law, Kody J. H.
- Stochastic Analysis and Applications, Vol. 35, Issue 3
Multilevel ensemble Kalman filtering
journal, January 2016
- Hoel, Håkon; Law, Kody J. H.; Tempone, Raul
- SIAM Journal on Numerical Analysis, Vol. 54, Issue 3
Complexity analysis of accelerated MCMC methods for Bayesian inversion
journal, July 2013
- Hoang, Viet Ha; Schwab, Christoph; Stuart, Andrew M.
- Inverse Problems, Vol. 29, Issue 8
Weighted Subspace Distance and Its Applications to Object Recognition and Retrieval With Image Sets
journal, March 2009
- Li, Fei; Dai, Qionghai; Xu, Wenli
- IEEE Signal Processing Letters, Vol. 16, Issue 3
Multilevel Ensemble Transform Particle Filtering
journal, January 2016
- Gregory, A.; Cotter, C. J.; Reich, S.
- SIAM Journal on Scientific Computing, Vol. 38, Issue 3
Multilevel sequential Monte Carlo samplers
journal, May 2017
- Beskos, Alexandros; Jasra, Ajay; Law, Kody
- Stochastic Processes and their Applications, Vol. 127, Issue 5
Accelerated Dimension-Independent Adaptive Metropolis
journal, January 2016
- Chen, Yuxin; Keyes, David; Law, Kody J. H.
- SIAM Journal on Scientific Computing, Vol. 38, Issue 5
Multilevel Monte Carlo Path Simulation
journal, June 2008
- Giles, Michael B.
- Operations Research, Vol. 56, Issue 3
Monte Carlo Complexity of Global Solution of Integral Equations
journal, June 1998
- Heinrich, S.
- Journal of Complexity, Vol. 14, Issue 2
Convergence analysis of multilevel Monte Carlo variance estimators and application for random obstacle problems
journal, December 2014
- Bierig, Claudio; Chernov, Alexey
- Numerische Mathematik, Vol. 130, Issue 4
Geometric MCMC for infinite-dimensional inverse problems
journal, April 2017
- Beskos, Alexandros; Girolami, Mark; Lan, Shiwei
- Journal of Computational Physics, Vol. 335
Likelihood-informed dimension reduction for nonlinear inverse problems
journal, October 2014
- Cui, T.; Martin, J.; Marzouk, Y. M.
- Inverse Problems, Vol. 30, Issue 11
On a Generalization of the Preconditioned Crank–Nicolson Metropolis Algorithm
journal, November 2016
- Rudolf, Daniel; Sprungk, Björn
- Foundations of Computational Mathematics, Vol. 18, Issue 2
Unbiased Estimation with Square Root Convergence for SDE Models
journal, October 2015
- Rhee, Chang-Han; Glynn, Peter W.
- Operations Research, Vol. 63, Issue 5
Forward and Inverse Uncertainty Quantification Using Multilevel Monte Carlo Algorithms for an Elliptic Nonlocal Equation
journal, January 2016
- Jasra, Ajay; Law, Kody J. H.; Zhou, Yan
- International Journal for Uncertainty Quantification, Vol. 6, Issue 6
Inference for Lévy-Driven Stochastic Volatility Models via Adaptive Sequential Monte Carlo: Lévy-driven stochastic volatility
journal, December 2010
- Jasra, Ajay; Stephens, David A.; Doucet, Arnaud
- Scandinavian Journal of Statistics, Vol. 38, Issue 1
Sequential Monte Carlo samplers
journal, June 2006
- Del Moral, Pierre; Doucet, Arnaud; Jasra, Ajay
- Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 68, Issue 3
Optimal Low-rank Approximations of Bayesian Linear Inverse Problems
journal, January 2015
- Spantini, Alessio; Solonen, Antti; Cui, Tiangang
- SIAM Journal on Scientific Computing, Vol. 37, Issue 6
Sequential Monte Carlo methods for Bayesian elliptic inverse problems
journal, June 2015
- Beskos, Alexandros; Jasra, Ajay; Muzaffer, Ege A.
- Statistics and Computing, Vol. 25, Issue 4
Multilevel sequential Monte Carlo samplers
journal, May 2017
- Beskos, Alexandros; Jasra, Ajay; Law, Kody
- Stochastic Processes and their Applications, Vol. 127, Issue 5
Multilevel Sequential Monte Carlo Samplers for Normalizing Constants
journal, July 2017
- Moral, Pierre Del; Jasra, Ajay; Law, Kody J. H.
- ACM Transactions on Modeling and Computer Simulation, Vol. 27, Issue 3
On the stability of sequential Monte Carlo methods in high dimensions
journal, August 2014
- Beskos, Alexandros; Crisan, Dan; Jasra, Ajay
- The Annals of Applied Probability, Vol. 24, Issue 4
Rates of convergence of the Hastings and Metropolis algorithms
journal, February 1996
- Mengersen, K. L.; Tweedie, R. L.
- The Annals of Statistics, Vol. 24, Issue 1
Uncertainty quantification and weak approximation of an elliptic inverse problem
preprint, January 2011
- Dashti, Masoumeh; Stuart, Andrew M.
- arXiv
Optimal Shrinkage of Eigenvalues in the Spiked Covariance Model
preprint, January 2013
- Donoho, David L.; Gavish, Matan; Johnstone, Iain M.
- arXiv
Geometric MCMC for Infinite-Dimensional Inverse Problems
text, January 2016
- Beskos, Alexandros; Girolami, Mark; Lan, Shiwei
- arXiv
Works referencing / citing this record:
Transform-based particle filtering for elliptic Bayesian inverse problems
journal, October 2019
- Ruchi, S.; Dubinkina, S.; Iglesias, M. A.
- Inverse Problems, Vol. 35, Issue 11
Transform-based particle filtering for elliptic Bayesian inverse problems
text, January 2019
- Ruchi, Sangeetika; Dubinkina, Svetlana; Iglesias, Marco
- arXiv
Multilevel Sequential2 Monte Carlo for Bayesian inverse problems
journal, September 2018
- Latz, Jonas; Papaioannou, Iason; Ullmann, Elisabeth
- Journal of Computational Physics, Vol. 368
Markov chain simulation for multilevel Monte Carlo
journal, January 2021
- Jasra, Ajay; Law, Kody J. H.; Xu, Yaxian
- Foundations of Data Science, Vol. 3, Issue 1
Data-Free Likelihood-Informed Dimension Reduction of Bayesian Inverse Problems
text, January 2021
- Cui, Tiangang; Zahm, Olivier
- arXiv