Likelihood-informed dimension reduction for nonlinear inverse problems
Abstract
The intrinsic dimensionality of an inverse problem is affected by prior information, the accuracy and number of observations, and the smoothing properties of the forward operator. From a Bayesian perspective, changes from the prior to the posterior may, in many problems, be confined to a relatively low-dimensional subspace of the parameter space. We present a dimension reduction approach that defines and identifies such a subspace, called the 'likelihood-informed subspace' (LIS), by characterizing the relative influences of the prior and the likelihood over the support of the posterior distribution. This identification enables new and more efficient computational methods for Bayesian inference with nonlinear forward models and Gaussian priors. In particular, we approximate the posterior distribution as the product of a lower-dimensional posterior defined on the LIS and the prior distribution marginalized onto the complementary subspace. Markov chain Monte Carlo sampling can then proceed in lower dimensions, with significant gains in computational efficiency. We also introduce a Rao–Blackwellization strategy that de-randomizes Monte Carlo estimates of posterior expectations for additional variance reduction. We demonstrate the efficiency of our methods using two numerical examples: inference of permeability in a groundwater system governed by an elliptic PDE, and an atmospheric remote sensing problem based onmore »
- Authors:
-
- Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States). Dept. of Aeronautics and Astronautics
- Univ. of Texas, Austin, TX (United States). Inst. for Computational Engineering and Sciences
- Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States). Dept. of Aeronautics and Astronautics; Lappeenranta Univ. of Technology, Lappeenranta (Finland)
- Publication Date:
- Research Org.:
- Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
- OSTI Identifier:
- 1557619
- Grant/Contract Number:
- SC0003908
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Inverse Problems
- Additional Journal Information:
- Journal Volume: 30; Journal Issue: 11; Journal ID: ISSN 0266-5611
- Publisher:
- IOPscience
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 42 ENGINEERING
Citation Formats
Cui, T., Martin, J., Marzouk, Y. M., Solonen, A., and Spantini, A. Likelihood-informed dimension reduction for nonlinear inverse problems. United States: N. p., 2014.
Web. doi:10.1088/0266-5611/30/11/114015.
Cui, T., Martin, J., Marzouk, Y. M., Solonen, A., & Spantini, A. Likelihood-informed dimension reduction for nonlinear inverse problems. United States. https://doi.org/10.1088/0266-5611/30/11/114015
Cui, T., Martin, J., Marzouk, Y. M., Solonen, A., and Spantini, A. Tue .
"Likelihood-informed dimension reduction for nonlinear inverse problems". United States. https://doi.org/10.1088/0266-5611/30/11/114015. https://www.osti.gov/servlets/purl/1557619.
@article{osti_1557619,
title = {Likelihood-informed dimension reduction for nonlinear inverse problems},
author = {Cui, T. and Martin, J. and Marzouk, Y. M. and Solonen, A. and Spantini, A.},
abstractNote = {The intrinsic dimensionality of an inverse problem is affected by prior information, the accuracy and number of observations, and the smoothing properties of the forward operator. From a Bayesian perspective, changes from the prior to the posterior may, in many problems, be confined to a relatively low-dimensional subspace of the parameter space. We present a dimension reduction approach that defines and identifies such a subspace, called the 'likelihood-informed subspace' (LIS), by characterizing the relative influences of the prior and the likelihood over the support of the posterior distribution. This identification enables new and more efficient computational methods for Bayesian inference with nonlinear forward models and Gaussian priors. In particular, we approximate the posterior distribution as the product of a lower-dimensional posterior defined on the LIS and the prior distribution marginalized onto the complementary subspace. Markov chain Monte Carlo sampling can then proceed in lower dimensions, with significant gains in computational efficiency. We also introduce a Rao–Blackwellization strategy that de-randomizes Monte Carlo estimates of posterior expectations for additional variance reduction. We demonstrate the efficiency of our methods using two numerical examples: inference of permeability in a groundwater system governed by an elliptic PDE, and an atmospheric remote sensing problem based on Global Ozone Monitoring System (GOMOS) observations.},
doi = {10.1088/0266-5611/30/11/114015},
journal = {Inverse Problems},
number = 11,
volume = 30,
place = {United States},
year = {Tue Oct 28 00:00:00 EDT 2014},
month = {Tue Oct 28 00:00:00 EDT 2014}
}
Free Publicly Available Full Text
Publisher's Version of Record
Other availability
Search WorldCat to find libraries that may hold this journal
Cited by: 78 works
Citation information provided by
Web of Science
Web of Science
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
Inverse Problem Theory and Methods for Model Parameter Estimation
book, January 2005
- Tarantola, Albert
Handbook of Markov Chain Monte Carlo
book, May 2011
- Brooks, Steve; Gelman, Andrew; Jones, Galin
- Handbooks of Modern Statistical Methods
Weak convergence and optimal scaling of random walk Metropolis algorithms
journal, February 1997
- Roberts, G. O.; Gelman, A.; Gilks, W. R.
- The Annals of Applied Probability, Vol. 7, Issue 1
Editorial
journal, January 1998
- Jones, M. C.; Young, G. A.
- Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 60, Issue 1
Optimal scaling for various Metropolis-Hastings algorithms
journal, November 2001
- Roberts, Gareth O.; Rosenthal, Jeffrey S.
- Statistical Science, Vol. 16, Issue 4
Diffusion limits of the random walk Metropolis algorithm in high dimensions
journal, June 2012
- Mattingly, Jonathan C.; Pillai, Natesh S.; Stuart, Andrew M.
- The Annals of Applied Probability, Vol. 22, Issue 3
Optimal scaling and diffusion limits for the Langevin algorithm in high dimensions
journal, December 2012
- Pillai, Natesh S.; Stuart, Andrew M.; Thiéry, Alexandre H.
- The Annals of Applied Probability, Vol. 22, Issue 6
Mcmc Methods for Diffusion Bridges
journal, September 2008
- Beskos, Alexandros; Roberts, Gareth; Stuart, Andrew
- Stochastics and Dynamics, Vol. 08, Issue 03
MCMC Methods for Functions: Modifying Old Algorithms to Make Them Faster
journal, August 2013
- Cotter, S. L.; Roberts, G. O.; Stuart, A. M.
- Statistical Science, Vol. 28, Issue 3
Fast Algorithms for Bayesian Uncertainty Quantification in Large-Scale Linear Inverse Problems Based on Low-Rank Partial Hessian Approximations
journal, January 2011
- Flath, H. P.; Wilcox, L. C.; Akçelik, V.
- SIAM Journal on Scientific Computing, Vol. 33, Issue 1
Dimensionality reduction and polynomial chaos acceleration of Bayesian inference in inverse problems
journal, April 2009
- Marzouk, Youssef M.; Najm, Habib N.
- Journal of Computational Physics, Vol. 228, Issue 6
A Stochastic Newton MCMC Method for Large-Scale Statistical Inverse Problems with Application to Seismic Inversion
journal, January 2012
- Martin, James; Wilcox, Lucas C.; Burstedde, Carsten
- SIAM Journal on Scientific Computing, Vol. 34, Issue 3
A Metric for Covariance Matrices
book, January 2003
- Förstner, Wolfgang; Moonen, Boudewijn
- Geodesy-The Challenge of the 3rd Millennium
Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions
journal, January 2011
- Halko, N.; Martinsson, P. G.; Tropp, J. A.
- SIAM Review, Vol. 53, Issue 2
Randomized algorithms for the low-rank approximation of matrices
journal, December 2007
- Liberty, E.; Woolfe, F.; Martinsson, P. -G.
- Proceedings of the National Academy of Sciences, Vol. 104, Issue 51
An Adaptive Metropolis Algorithm
journal, April 2001
- Haario, Heikki; Saksman, Eero; Tamminen, Johanna
- Bernoulli, Vol. 7, Issue 2
On the ergodicity properties of some adaptive MCMC algorithms
journal, August 2006
- Andrieu, Christophe; Moulines, Éric
- The Annals of Applied Probability, Vol. 16, Issue 3
An Adaptive Version for the Metropolis Adjusted Langevin Algorithm with a Truncated Drift
journal, June 2006
- Atchadé, Yves F.
- Methodology and Computing in Applied Probability, Vol. 8, Issue 2
Coupling and Ergodicity of Adaptive Markov Chain Monte Carlo Algorithms
journal, June 2007
- Roberts, Gareth O.; Rosenthal, Jeffrey S.
- Journal of Applied Probability, Vol. 44, Issue 2
Examples of Adaptive MCMC
journal, January 2009
- Roberts, Gareth O.; Rosenthal, Jeffrey S.
- Journal of Computational and Graphical Statistics, Vol. 18, Issue 2
Original Articles
journal, January 2011
- Lauritzen, S.; Witten, D.; Yao, Q.
- Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 73, Issue 2
Rao-Blackwellisation of sampling schemes
journal, March 1996
- Casella, G.
- Biometrika, Vol. 83, Issue 1
Weighted Subspace Distance and Its Applications to Object Recognition and Retrieval With Image Sets
journal, March 2009
- Li, Fei; Dai, Qionghai; Xu, Wenli
- IEEE Signal Processing Letters, Vol. 16, Issue 3
Randomized algorithms for the low-rank approximation of matrices
journal, December 2007
- Liberty, E.; Woolfe, F.; Martinsson, P. -G.
- Proceedings of the National Academy of Sciences, Vol. 104, Issue 51
Markov chain Monte Carlo methods for high dimensional inversion in remote sensing
journal, August 2004
- Haario, H.; Laine, M.; Lehtinen, M.
- Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 66, Issue 3
Optimal Low-rank Approximations of Bayesian Linear Inverse Problems
journal, January 2015
- Spantini, Alessio; Solonen, Antti; Cui, Tiangang
- SIAM Journal on Scientific Computing, Vol. 37, Issue 6
Optimal scaling for various Metropolis-Hastings algorithms
journal, November 2001
- Roberts, Gareth O.; Rosenthal, Jeffrey S.
- Statistical Science, Vol. 16, Issue 4
An Adaptive Metropolis Algorithm
journal, April 2001
- Haario, Heikki; Saksman, Eero; Tamminen, Johanna
- Bernoulli, Vol. 7, Issue 2
Discretization-invariant Bayesian inversion and Besov space priors
preprint, January 2009
- Saksman, Matti Lassas. Eero; Siltanen, Samuli
- arXiv
Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions
text, January 2009
- Halko, Nathan; Martinsson, Per-Gunnar; Tropp, Joel A.
- arXiv
On the ergodicity properties of some adaptive MCMC algorithms
text, January 2006
- Andrieu, Christophe; Moulines, Éric
- arXiv
Works referencing / citing this record:
Retrieval of atmospheric CH 4 profiles from Fourier transform infrared data using dimension reduction and MCMC: RETRIEVAL OF FTIR CH 4 PROFILES
journal, September 2016
- Tukiainen, S.; Railo, J.; Laine, M.
- Journal of Geophysical Research: Atmospheres, Vol. 121, Issue 17
Randomized Truncated SVD Levenberg‐Marquardt Approach to Geothermal Natural State and History Matching
journal, March 2018
- Bjarkason, Elvar K.; Maclaren, Oliver J.; O'Sullivan, John P.
- Water Resources Research, Vol. 54, Issue 3
Inverse problems: From regularization to Bayesian inference
journal, January 2018
- Calvetti, D.; Somersalo, E.
- WIREs Computational Statistics, Vol. 10, Issue 3
Sampling via Measure Transport: An Introduction
book, January 2016
- Marzouk, Youssef; Moselhy, Tarek; Parno, Matthew
- Handbook of Uncertainty Quantification
Spatial Localization for Nonlinear Dynamical Stochastic Models for Excitable Media
journal, November 2019
- Chen, Nan; Majda, Andrew J.; Tong, Xin T.
- Chinese Annals of Mathematics, Series B, Vol. 40, Issue 6
Scaling limits in computational Bayesian inversion
journal, October 2016
- Schillings, Claudia; Schwab, Christoph
- ESAIM: Mathematical Modelling and Numerical Analysis, Vol. 50, Issue 6
Accelerated MCMC for Satellite-Based Measurements of Atmospheric CO2
journal, September 2019
- Lamminpää, Otto; Hobbs, Jonathan; Brynjarsdóttir, Jenný
- Remote Sensing, Vol. 11, Issue 17
Top-down constraints on global N2O emissions at optimal resolution: application of a new dimension reduction technique
journal, January 2018
- Wells, Kelley C.; Millet, Dylan B.; Bousserez, Nicolas
- Atmospheric Chemistry and Physics, Vol. 18, Issue 2
Sampling via Measure Transport: An Introduction
book, June 2017
- Marzouk, Youssef; Moselhy, Tarek; Parno, Matthew
- Handbook of Uncertainty Quantification
Scaling Limits in Computational Bayesian Inversion
text, January 2014
- Schillings, Claudia; Schwab, Christoph
- ETH Zurich
Randomized Truncated SVD Levenberg-Marquardt Approach to Geothermal Natural State and History Matching
text, January 2017
- Bjarkason, Elvar K.; Maclaren, Oliver J.; O'Sullivan, John P.
- arXiv
Spatial localization for nonlinear dynamical stochastic models for excitable media
preprint, January 2019
- Chen, Nan; Majda, Andrew J.; Tong, Xin T.
- arXiv
Fast Kalman Filter using Hierarchical-matrices and low-rank perturbative approach
text, January 2014
- Saibaba, Arvind K.; Miller, Eric; Kitanidis, Peter K.
- arXiv
A randomized maximum a posterior method for posterior sampling of high dimensional nonlinear Bayesian inverse problems
preprint, January 2016
- Wang, Kainan; Bui-Thanh, Tan; Ghattas, Omar
- arXiv