Likelihood-informed dimension reduction for nonlinear inverse problems

Cui, T.; Martin, J.; Marzouk, Y. M.; Solonen, A.; Spantini, A.

doi:10.1088/0266-5611/30/11/114015

Title: Likelihood-informed dimension reduction for nonlinear inverse problems

Journal Article · Tue Oct 28 00:00:00 EDT 2014 · Inverse Problems

DOI:https://doi.org/10.1088/0266-5611/30/11/114015· OSTI ID:1557619

Cui, T. ^[1]; Martin, J. ^[2]; Marzouk, Y. M. ^[1]; Solonen, A. ^[3]; Spantini, A. ^[1]

Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States). Dept. of Aeronautics and Astronautics
Univ. of Texas, Austin, TX (United States). Inst. for Computational Engineering and Sciences
Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States). Dept. of Aeronautics and Astronautics; Lappeenranta Univ. of Technology, Lappeenranta (Finland)

The intrinsic dimensionality of an inverse problem is affected by prior information, the accuracy and number of observations, and the smoothing properties of the forward operator. From a Bayesian perspective, changes from the prior to the posterior may, in many problems, be confined to a relatively low-dimensional subspace of the parameter space. We present a dimension reduction approach that defines and identifies such a subspace, called the 'likelihood-informed subspace' (LIS), by characterizing the relative influences of the prior and the likelihood over the support of the posterior distribution. This identification enables new and more efficient computational methods for Bayesian inference with nonlinear forward models and Gaussian priors. In particular, we approximate the posterior distribution as the product of a lower-dimensional posterior defined on the LIS and the prior distribution marginalized onto the complementary subspace. Markov chain Monte Carlo sampling can then proceed in lower dimensions, with significant gains in computational efficiency. We also introduce a Rao–Blackwellization strategy that de-randomizes Monte Carlo estimates of posterior expectations for additional variance reduction. We demonstrate the efficiency of our methods using two numerical examples: inference of permeability in a groundwater system governed by an elliptic PDE, and an atmospheric remote sensing problem based on Global Ozone Monitoring System (GOMOS) observations.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)

Grant/Contract Number:: SC0003908

OSTI ID:: 1557619

Journal Information:: Inverse Problems, Vol. 30, Issue 11; ISSN 0266-5611

Publisher:: IOPscienceCopyright Statement

Country of Publication:: United States

Language:: English

Citation Metrics:

Cited by: 78 works

Citation information provided by
Web of Science

References (29)

Inverse Problem Theory and Methods for Model Parameter Estimation Tarantola, Albert https://doi.org/10.1137/1.9780898717921	book	January 2005
Inverse problems: A Bayesian perspective Stuart, A. M. Acta Numerica, Vol. 19 https://doi.org/10.1017/S0962492910000061	journal	May 2010
Handbook of Markov Chain Monte Carlo Brooks, Steve; Gelman, Andrew; Jones, Galin Handbooks of Modern Statistical Methods https://doi.org/10.1201/b10905	book	May 2011
Weak convergence and optimal scaling of random walk Metropolis algorithms Roberts, G. O.; Gelman, A.; Gilks, W. R. The Annals of Applied Probability, Vol. 7, Issue 1 https://doi.org/10.1214/aoap/1034625254	journal	February 1997
Editorial Jones, M. C.; Young, G. A. Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 60, Issue 1 https://doi.org/10.1111/rssb.1998.60.issue-1	journal	January 1998
Optimal scaling for various Metropolis-Hastings algorithms Roberts, Gareth O.; Rosenthal, Jeffrey S. Statistical Science, Vol. 16, Issue 4 https://doi.org/10.1214/ss/1015346320	journal	November 2001
Diffusion limits of the random walk Metropolis algorithm in high dimensions Mattingly, Jonathan C.; Pillai, Natesh S.; Stuart, Andrew M. The Annals of Applied Probability, Vol. 22, Issue 3 https://doi.org/10.1214/10-AAP754	journal	June 2012
Optimal scaling and diffusion limits for the Langevin algorithm in high dimensions Pillai, Natesh S.; Stuart, Andrew M.; Thiéry, Alexandre H. The Annals of Applied Probability, Vol. 22, Issue 6 https://doi.org/10.1214/11-AAP828	journal	December 2012
Mcmc Methods for Diffusion Bridges Beskos, Alexandros; Roberts, Gareth; Stuart, Andrew Stochastics and Dynamics, Vol. 08, Issue 03 https://doi.org/10.1142/S0219493708002378	journal	September 2008
MCMC Methods for Functions: Modifying Old Algorithms to Make Them Faster Cotter, S. L.; Roberts, G. O.; Stuart, A. M. Statistical Science, Vol. 28, Issue 3 https://doi.org/10.1214/13-STS421	journal	August 2013
Fast Algorithms for Bayesian Uncertainty Quantification in Large-Scale Linear Inverse Problems Based on Low-Rank Partial Hessian Approximations Flath, H. P.; Wilcox, L. C.; Akçelik, V. SIAM Journal on Scientific Computing, Vol. 33, Issue 1 https://doi.org/10.1137/090780717	journal	January 2011
Dimensionality reduction and polynomial chaos acceleration of Bayesian inference in inverse problems Marzouk, Youssef M.; Najm, Habib N. Journal of Computational Physics, Vol. 228, Issue 6 https://doi.org/10.1016/j.jcp.2008.11.024	journal	April 2009
A Stochastic Newton MCMC Method for Large-Scale Statistical Inverse Problems with Application to Seismic Inversion Martin, James; Wilcox, Lucas C.; Burstedde, Carsten SIAM Journal on Scientific Computing, Vol. 34, Issue 3 https://doi.org/10.1137/110845598	journal	January 2012
A Metric for Covariance Matrices Förstner, Wolfgang; Moonen, Boudewijn Geodesy-The Challenge of the 3rd Millennium https://doi.org/10.1007/978-3-662-05296-9_31	book	January 2003
Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions Halko, N.; Martinsson, P. G.; Tropp, J. A. SIAM Review, Vol. 53, Issue 2 https://doi.org/10.1137/090771806	journal	January 2011
Randomized algorithms for the low-rank approximation of matrices Liberty, E.; Woolfe, F.; Martinsson, P. -G. Proceedings of the National Academy of Sciences, Vol. 104, Issue 51 https://doi.org/10.1073/pnas.0709640104	journal	December 2007
An Adaptive Metropolis Algorithm Haario, Heikki; Saksman, Eero; Tamminen, Johanna Bernoulli, Vol. 7, Issue 2 https://doi.org/10.2307/3318737	journal	April 2001
On the ergodicity properties of some adaptive MCMC algorithms Andrieu, Christophe; Moulines, Éric The Annals of Applied Probability, Vol. 16, Issue 3 https://doi.org/10.1214/105051606000000286	journal	August 2006
An Adaptive Version for the Metropolis Adjusted Langevin Algorithm with a Truncated Drift Atchadé, Yves F. Methodology and Computing in Applied Probability, Vol. 8, Issue 2 https://doi.org/10.1007/s11009-006-8550-0	journal	June 2006
Coupling and Ergodicity of Adaptive Markov Chain Monte Carlo Algorithms Roberts, Gareth O.; Rosenthal, Jeffrey S. Journal of Applied Probability, Vol. 44, Issue 2 https://doi.org/10.1239/jap/1183667414	journal	June 2007
Examples of Adaptive MCMC Roberts, Gareth O.; Rosenthal, Jeffrey S. Journal of Computational and Graphical Statistics, Vol. 18, Issue 2 https://doi.org/10.1198/jcgs.2009.06134	journal	January 2009
Original Articles Lauritzen, S.; Witten, D.; Yao, Q. Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 73, Issue 2 https://doi.org/10.1111/rssb.2011.73.issue-2	journal	January 2011
Rao-Blackwellisation of sampling schemes Casella, G. Biometrika, Vol. 83, Issue 1 https://doi.org/10.1093/biomet/83.1.81	journal	March 1996
Weighted Subspace Distance and Its Applications to Object Recognition and Retrieval With Image Sets Li, Fei; Dai, Qionghai; Xu, Wenli IEEE Signal Processing Letters, Vol. 16, Issue 3 https://doi.org/10.1109/LSP.2008.2010819	journal	March 2009
Markov chain Monte Carlo methods for high dimensional inversion in remote sensing Haario, H.; Laine, M.; Lehtinen, M. Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 66, Issue 3 https://doi.org/10.1111/j.1467-9868.2004.02053.x	journal	August 2004
Optimal Low-rank Approximations of Bayesian Linear Inverse Problems Spantini, Alessio; Solonen, Antti; Cui, Tiangang SIAM Journal on Scientific Computing, Vol. 37, Issue 6 https://doi.org/10.1137/140977308	journal	January 2015
Discretization-invariant Bayesian inversion and Besov space priors Saksman, Matti Lassas. Eero; Siltanen, Samuli arXiv https://doi.org/10.48550/arxiv.0901.4220	preprint	January 2009
Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions Halko, Nathan; Martinsson, Per-Gunnar; Tropp, Joel A. arXiv https://doi.org/10.48550/arxiv.0909.4061	text	January 2009
On the ergodicity properties of some adaptive MCMC algorithms Andrieu, Christophe; Moulines, Éric arXiv https://doi.org/10.48550/arxiv.math/0610317	text	January 2006

Cited By (14)

Retrieval of atmospheric CH ₄ profiles from Fourier transform infrared data using dimension reduction and MCMC: RETRIEVAL OF FTIR CH ₄ PROFILES Tukiainen, S.; Railo, J.; Laine, M. Journal of Geophysical Research: Atmospheres, Vol. 121, Issue 17 https://doi.org/10.1002/2015jd024657	journal	September 2016
Randomized Truncated SVD Levenberg‐Marquardt Approach to Geothermal Natural State and History Matching Bjarkason, Elvar K.; Maclaren, Oliver J.; O'Sullivan, John P. Water Resources Research, Vol. 54, Issue 3 https://doi.org/10.1002/2017wr021870	journal	March 2018
Inverse problems: From regularization to Bayesian inference Calvetti, D.; Somersalo, E. WIREs Computational Statistics, Vol. 10, Issue 3 https://doi.org/10.1002/wics.1427	journal	January 2018
Sampling via Measure Transport: An Introduction Marzouk, Youssef; Moselhy, Tarek; Parno, Matthew Handbook of Uncertainty Quantification https://doi.org/10.1007/978-3-319-11259-6_23-1	book	January 2016
Spatial Localization for Nonlinear Dynamical Stochastic Models for Excitable Media Chen, Nan; Majda, Andrew J.; Tong, Xin T. Chinese Annals of Mathematics, Series B, Vol. 40, Issue 6 https://doi.org/10.1007/s11401-019-0166-0	journal	November 2019
Scaling limits in computational Bayesian inversion Schillings, Claudia; Schwab, Christoph ESAIM: Mathematical Modelling and Numerical Analysis, Vol. 50, Issue 6 https://doi.org/10.1051/m2an/2016005	journal	October 2016
Accelerated MCMC for Satellite-Based Measurements of Atmospheric CO2 Lamminpää, Otto; Hobbs, Jonathan; Brynjarsdóttir, Jenný Remote Sensing, Vol. 11, Issue 17 https://doi.org/10.3390/rs11172061	journal	September 2019
Top-down constraints on global N₂O emissions at optimal resolution: application of a new dimension reduction technique Wells, Kelley C.; Millet, Dylan B.; Bousserez, Nicolas Atmospheric Chemistry and Physics, Vol. 18, Issue 2 https://doi.org/10.5194/acp-18-735-2018	journal	January 2018
Sampling via Measure Transport: An Introduction Marzouk, Youssef; Moselhy, Tarek; Parno, Matthew Handbook of Uncertainty Quantification https://doi.org/10.1007/978-3-319-12385-1_23	book	June 2017
Scaling Limits in Computational Bayesian Inversion Schillings, Claudia; Schwab, Christoph ETH Zurich https://doi.org/10.3929/ethz-a-010386179	text	January 2014
Randomized Truncated SVD Levenberg-Marquardt Approach to Geothermal Natural State and History Matching Bjarkason, Elvar K.; Maclaren, Oliver J.; O'Sullivan, John P. arXiv https://doi.org/10.48550/arxiv.1710.01031	text	January 2017
Spatial localization for nonlinear dynamical stochastic models for excitable media Chen, Nan; Majda, Andrew J.; Tong, Xin T. arXiv https://doi.org/10.48550/arxiv.1901.07318	preprint	January 2019
Fast Kalman Filter using Hierarchical-matrices and low-rank perturbative approach Saibaba, Arvind K.; Miller, Eric; Kitanidis, Peter K. arXiv https://doi.org/10.48550/arxiv.1405.2276	text	January 2014
A randomized maximum a posterior method for posterior sampling of high dimensional nonlinear Bayesian inverse problems Wang, Kainan; Bui-Thanh, Tan; Ghattas, Omar arXiv https://doi.org/10.48550/arxiv.1602.03658	preprint	January 2016

Similar Records

Scalable posterior approximations for large-scale Bayesian inverse problems via likelihood-informed parameter and state reduction

Journal Article · Tue Mar 29 00:00:00 EDT 2016 · Journal of Computational Physics · OSTI ID:1557619

Cui, Tiangang; Marzouk, Youssef; Willcox, Karen

Dimension-independent likelihood-informed MCMC

Journal Article · Fri Jan 01 00:00:00 EST 2016 · Journal of Computational Physics · OSTI ID:1557619

Cui, Tiangang

Dimension-independent likelihood-informed MCMC

Journal Article · Thu Oct 08 00:00:00 EDT 2015 · Journal of Computational Physics · OSTI ID:1557619

Cui, Tiangang; Law, Kody J. H.; Marzouk, Youssef M.

Related Subjects

42 ENGINEERING

Title: Likelihood-informed dimension reduction for nonlinear inverse problems

Citation Formats

References (29)

Cited By (14)

Similar Records

Related Subjects