Sequential ensemble transform for Bayesian inverse problems

Myers, Aaron; Thiéry, Alexandre H.; Wang, Kainan; Bui-Thanh, Tan

doi:10.1016/j.jcp.2020.110055

Sequential ensemble transform for Bayesian inverse problems

Journal Article · Mon Dec 07 00:00:00 EST 2020 · Journal of Computational Physics

DOI:https://doi.org/10.1016/j.jcp.2020.110055· OSTI ID:1852513

Myers, Aaron ^[1]; Thiéry, Alexandre H. ^[2]; Wang, Kainan ^[3]; ^[4]

Univ. of Texas, Austin, TX (United States). Inst. for Computational Engineering & Sciences; National Univ. of Singapore (Singapore)
National Univ. of Singapore (Singapore). Probability and Applied Statistics
Sanchez Oil & Gas, Houston, TX (United States)
Univ. of Texas, Austin, TX (United States). Inst. for Computational Engineering & Sciences; Univ. of Texas, Austin, TX (United States). Dept. of Aerospace Engineering & Engineering Mechanics

In this work, we present the Sequential Ensemble Transform (SET) method, an approach for generating approximate samples from a Bayesian posterior distribution. The method explores the posterior distribution by solving a sequence of discrete optimal transport problems to produce a series of transport plans which map prior samples to posterior samples. We prove that the sequence of Dirac mixture distributions produced by the SET method converges weakly to the true posterior as the sample size approaches infinity. Furthermore, our numerical results indicate that, when compared to standard Sequential Monte Carlo (SMC) methods, the SET approach is more robust to the choice of Markov mutation kernels and requires less computational efforts to reach a similar accuracy when used to explore complex posterior distributions. Finally, we describe adaptive schemes that allow to completely automate the use of the SET method.

View Accepted Manuscript (DOE)

Research Organization:: Univ. of Texas, Austin, TX (United States)

Sponsoring Organization:: Defense Threat Reduction Agency (DTRA); National Science Foundation (NSF); USDOE; USDOE Office of Science (SC)

Grant/Contract Number:: SC0018147

OSTI ID:: 1852513

Alternate ID(s):: OSTI ID: 1775922
OSTI ID: 23203496

Journal Information:: Journal of Computational Physics, Journal Name: Journal of Computational Physics Journal Issue: C Vol. 427; ISSN 0021-9991

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

References (45)

Bayesian inference with optimal maps El Moselhy, Tarek A.; Marzouk, Youssef M. Journal of Computational Physics, Vol. 231, Issue 23 https://doi.org/10.1016/j.jcp.2012.07.022	journal	October 2012
Equation of state calculations by fast computing machines Metropolis, Nicholas; Rosenbluth, Arianna W.; Rosenbluth, Marshall N. https://doi.org/10.2172/4390578	report	March 1953
Polar factorization and monotone rearrangement of vector-valued functions Brenier, Yann Communications on Pure and Applied Mathematics, Vol. 44, Issue 4 https://doi.org/10.1002/cpa.3160440402	journal	June 1991
Hessian-based model reduction for large-scale systems with initial-condition inputs Bashir, O.; Willcox, K.; Ghattas, O. International Journal for Numerical Methods in Engineering, Vol. 73, Issue 6 https://doi.org/10.1002/nme.2100	journal	February 2008
Forest resampling for distributed sequential Monte Carlo Lee, Anthony; Whiteley, Nick Statistical Analysis and Data Mining: The ASA Data Science Journal, Vol. 9, Issue 4 https://doi.org/10.1002/sam.11280	journal	July 2015
On the Monge mass transfer problem Trudinger, Neil S.; Wang, Xu-Jia Calculus of Variations and Partial Differential Equations, Vol. 13, Issue 1 https://doi.org/10.1007/PL00009922	journal	August 2001
Adaptive importance sampling in general mixture classes Cappé, Olivier; Douc, Randal; Guillin, Arnaud Statistics and Computing, Vol. 18, Issue 4 https://doi.org/10.1007/s11222-008-9059-x	journal	April 2008
An adaptive sequential Monte Carlo method for approximate Bayesian computation Del Moral, Pierre; Doucet, Arnaud; Jasra, Ajay Statistics and Computing, Vol. 22, Issue 5 https://doi.org/10.1007/s11222-011-9271-y	journal	August 2011
Sequential Monte Carlo on large binary sampling spaces Schäfer, Christian; Chopin, Nicolas Statistics and Computing, Vol. 23, Issue 2 https://doi.org/10.1007/s11222-011-9299-z	journal	November 2011
On parallel implementation of sequential Monte Carlo methods: the island particle model Vergé, Christelle; Dubarry, Cyrille; Del Moral, Pierre Statistics and Computing, Vol. 25, Issue 2 https://doi.org/10.1007/s11222-013-9429-x	journal	November 2013
Sequential Monte Carlo methods for Bayesian elliptic inverse problems Beskos, Alexandros; Jasra, Ajay; Muzaffer, Ege A. Statistics and Computing, Vol. 25, Issue 4 https://doi.org/10.1007/s11222-015-9556-7	journal	June 2015
Existence and uniqueness of optimal transport maps Cavalletti, Fabio; Huesmann, Martin Annales de l'Institut Henri Poincaré C, Analyse non linéaire, Vol. 32, Issue 6 https://doi.org/10.1016/j.anihpc.2014.09.006	journal	November 2015
A Bayesian approach to joint tracking and identification of geometric shapes in video sequences Minvielle, Pierre; Doucet, Arnaud; Marrs, Alan Image and Vision Computing, Vol. 28, Issue 1 https://doi.org/10.1016/j.imavis.2009.05.002	journal	January 2010
Dimension-independent likelihood-informed MCMC Cui, Tiangang; Law, Kody J. H.; Marzouk, Youssef M. Journal of Computational Physics, Vol. 304 https://doi.org/10.1016/j.jcp.2015.10.008	journal	January 2016
Inverse problems: A Bayesian perspective Stuart, A. M. Acta Numerica, Vol. 19 https://doi.org/10.1017/S0962492910000061	journal	May 2010
Monte Carlo sampling of solutions to inverse problems Mosegaard, Klaus; Tarantola, Albert Journal of Geophysical Research: Solid Earth, Vol. 100, Issue B7 https://doi.org/10.1029/94JB03097	journal	July 1995
Equation of State Calculations by Fast Computing Machines Metropolis, Nicholas; Rosenbluth, Arianna W.; Rosenbluth, Marshall N. The Journal of Chemical Physics, Vol. 21, Issue 6 https://doi.org/10.1063/1.1699114	journal	June 1953
Adaptive importance sampling in monte carlo integration Oh, Man-Suk; Berger, James O. Journal of Statistical Computation and Simulation, Vol. 41, Issue 3-4 https://doi.org/10.1080/00949659208810398	journal	July 1992
Toward Automatic Model Comparison: An Adaptive Sequential Monte Carlo Approach Zhou, Yan; Johansen, Adam M.; Aston, John A. D. Journal of Computational and Graphical Statistics, Vol. 25, Issue 3 https://doi.org/10.1080/10618600.2015.1060885	journal	July 2016
Constructing optimal maps for Monge’s transport problem as a limit of strictly convex costs Caffarelli, Luis A.; Feldman, Mikhail; McCann, Robert J. Journal of the American Mathematical Society, Vol. 15, Issue 1 https://doi.org/10.1090/S0894-0347-01-00376-9	journal	July 2001
Monte Carlo sampling methods using Markov chains and their applications Hastings, W. K. Biometrika, Vol. 57, Issue 1 https://doi.org/10.1093/biomet/57.1.97	journal	April 1970
A sequential particle filter method for static models Chopin, N. Biometrika, Vol. 89, Issue 3 https://doi.org/10.1093/biomet/89.3.539	journal	August 2002
Variance estimation in the particle filter Lee, A.; Whiteley, N. Biometrika, Vol. 105, Issue 3 https://doi.org/10.1093/biomet/asy028	journal	June 2018
A survey of convergence results on particle filtering methods for practitioners Crisan, D.; Doucet, A. IEEE Transactions on Signal Processing, Vol. 50, Issue 3 https://doi.org/10.1109/78.984773	journal	March 2002
Efficient Sequential Monte-Carlo Samplers for Bayesian Inference Nguyen, Thi Le Thu; Septier, Francois; Peters, Gareth W. IEEE Transactions on Signal Processing, Vol. 64, Issue 5 https://doi.org/10.1109/TSP.2015.2504342	journal	March 2016
Inference for Lévy-Driven Stochastic Volatility Models via Adaptive Sequential Monte Carlo: Lévy-driven stochastic volatility Jasra, Ajay; Stephens, David A.; Doucet, Arnaud Scandinavian Journal of Statistics, Vol. 38, Issue 1 https://doi.org/10.1111/j.1467-9469.2010.00723.x	journal	December 2010
Adaptive Multiple Importance Sampling: Adaptive multiple importance sampling Cornuet, Jean-Marie; Marin, Jean-Michel; Mira, Antonietta Scandinavian Journal of Statistics, Vol. 39, Issue 4 https://doi.org/10.1111/j.1467-9469.2011.00756.x	journal	February 2012
Sequential Monte Carlo samplers Del Moral, Pierre; Doucet, Arnaud; Jasra, Ajay Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 68, Issue 3 https://doi.org/10.1111/j.1467-9868.2006.00553.x	journal	June 2006
An explicit link between Gaussian fields and Gaussian Markov random fields: the stochastic partial differential equation approach: Link between Gaussian Fields and Gaussian Markov Random Fields Lindgren, Finn; Rue, Håvard; Lindström, Johan Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 73, Issue 4 https://doi.org/10.1111/j.1467-9868.2011.00777.x	journal	August 2011
Fast Algorithms for Bayesian Uncertainty Quantification in Large-Scale Linear Inverse Problems Based on Low-Rank Partial Hessian Approximations Flath, H. P.; Wilcox, L. C.; Akçelik, V. SIAM Journal on Scientific Computing, Vol. 33, Issue 1 https://doi.org/10.1137/090780717	journal	January 2011
A Computational Framework for Infinite-Dimensional Bayesian Inverse Problems Part I: The Linearized Case, with Application to Global Seismic Inversion Bui-Thanh, Tan; Ghattas, Omar; Martin, James SIAM Journal on Scientific Computing, Vol. 35, Issue 6 https://doi.org/10.1137/12089586X	journal	January 2013
A Nonparametric Ensemble Transform Method for Bayesian Inference Reich, Sebastian SIAM Journal on Scientific Computing, Vol. 35, Issue 4 https://doi.org/10.1137/130907367	journal	January 2013
Sequential Monte Carlo Methods for High-Dimensional Inverse Problems: A Case Study for the Navier--Stokes Equations Kantas, Nikolas; Beskos, Alexandros; Jasra, Ajay SIAM/ASA Journal on Uncertainty Quantification, Vol. 2, Issue 1 https://doi.org/10.1137/130930364	journal	January 2014
Multilevel Ensemble Transform Particle Filtering Gregory, A.; Cotter, C. J.; Reich, S. SIAM Journal on Scientific Computing, Vol. 38, Issue 3 https://doi.org/10.1137/15M1038232	journal	January 2016
A Hybrid Ensemble Transform Particle Filter for Nonlinear and Spatially Extended Dynamical Systems Chustagulprom, Nawinda; Reich, Sebastian; Reinhardt, Maria SIAM/ASA Journal on Uncertainty Quantification, Vol. 4, Issue 1 https://doi.org/10.1137/15M1040967	journal	January 2016
A general theory of particle filters in hidden Markov models and some applications Chan, Hock Peng; Lai, Tze Leung The Annals of Statistics, Vol. 41, Issue 6 https://doi.org/10.1214/13-AOS1172	journal	December 2013
MCMC Methods for Functions: Modifying Old Algorithms to Make Them Faster Cotter, S. L.; Roberts, G. O.; Stuart, A. M. Statistical Science, Vol. 28, Issue 3 https://doi.org/10.1214/13-STS421	journal	August 2013
On Stochastic Limit and Order Relationships Mann, H. B.; Wald, A. The Annals of Mathematical Statistics, Vol. 14, Issue 3 https://doi.org/10.1214/aoms/1177731415	journal	September 1943
Existence and uniqueness of monotone measure-preserving maps McCann, Robert J. Duke Mathematical Journal, Vol. 80, Issue 2 https://doi.org/10.1215/S0012-7094-95-08013-2	journal	November 1995
Error Bounds and Normalising Constants for Sequential Monte Carlo Samplers in High Dimensions Beskos, Alexandros; Crisan, Dan O.; Jasra, Ajay Advances in Applied Probability, Vol. 46, Issue 1 https://doi.org/10.1239/aap/1396360114	journal	March 2014
Computational Optimal Transport: With Applications to Data Science Peyré, Gabriel; Cuturi, Marco Foundations and Trends® in Machine Learning, Vol. 11, Issue 5-6 https://doi.org/10.1561/2200000073	journal	January 2019
On the role of interaction in sequential Monte Carlo algorithms Whiteley, Nick; Lee, Anthony; Heine, Kari Bernoulli, Vol. 22, Issue 1 https://doi.org/10.3150/14-BEJ666	journal	February 2016
Nonasymptotic analysis of adaptive and annealed Feynman–Kac particle models Giraud, François; Del Moral, Pierre Bernoulli, Vol. 23, Issue 1 https://doi.org/10.3150/14-BEJ680	journal	February 2017
Numerically stable online estimation of variance in particle filters Olsson, Jimmy; Douc, Randal Bernoulli, Vol. 25, Issue 2 https://doi.org/10.3150/18-BEJ1028	journal	May 2019
Toward Automatic Model Comparison: An Adaptive Sequential Monte Carlo Approach Zhou, Yan; Johansen, Adam M.; Aston, John A. D. Taylor & Francis https://doi.org/10.6084/m9.figshare.3543491	dataset	January 2016

Similar Records

Sequential Monte Carlo for Cut-Bayesian Posterior Computation

Technical Report · Wed Jan 31 23:00:00 EST 2024 · OSTI ID:2323517

Bayesian inference of Stochastic reaction networks using Multifidelity Sequential Tempered Markov Chain Monte Carlo

Journal Article · Sun May 31 20:00:00 EDT 2020 · International Journal for Uncertainty Quantification · OSTI ID:1670752

Sequential Monte-Carlo Framework for Dynamic Data-Driven Event Reconstruction for Atmospheric Release

Conference · Mon Jul 17 00:00:00 EDT 2006 · OSTI ID:893974

Related Subjects

71 CLASSICAL AND QUANTUM MECHANICS
GENERAL PHYSICS
97 MATHEMATICS AND COMPUTING
Bayesian inverse problems
Computer Science
Monte-Carlo methods
Optimal transport
Particle methods
Physics

Sequential ensemble transform for Bayesian inverse problems

Citation Formats

References (45)

Similar Records

Related Subjects