An adaptive importance sampling algorithm for Bayesian inversion with multimodal distributions

Li, Weixuan; Lin, Guang

doi:10.1016/j.jcp.2015.03.047

Title: An adaptive importance sampling algorithm for Bayesian inversion with multimodal distributions

Journal Article · Sat Mar 21 00:00:00 EDT 2015 · Journal of Computational Physics

DOI:https://doi.org/10.1016/j.jcp.2015.03.047· OSTI ID:1182897

Li, Weixuan ^[1]; Lin, Guang ^[2]

Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
Purdue University, West Lafayette, IN (United States)

Parametric uncertainties are encountered in the simulations of many physical systems, and may be reduced by an inverse modeling procedure that calibrates the simulation results to observations on the real system being simulated. Following Bayes’ rule, a general approach for inverse modeling problems is to sample from the posterior distribution of the uncertain model parameters given the observations. However, the large number of repetitive forward simulations required in the sampling process could pose a prohibitive computational burden. This difficulty is particularly challenging when the posterior is multimodal. We present in this paper an adaptive importance sampling algorithm to tackle these challenges. Two essential ingredients of the algorithm are: 1) a Gaussian mixture (GM) model adaptively constructed as the proposal distribution to approximate the possibly multimodal target posterior, and 2) a mixture of polynomial chaos (PC) expansions, built according to the GM proposal, as a surrogate model to alleviate the computational burden caused by computational-demanding forward model evaluations. In three illustrative examples, the proposed adaptive importance sampling algorithm demonstrates its capabilities of automatically finding a GM proposal with an appropriate number of modes for the specific problem under study, and obtaining a sample accurately and efficiently representing the posterior with limited number of forward simulations.

View Accepted Manuscript (DOE)

View Accepted Manuscript (Publisher)

Cite

Export

Save

Research Organization:: Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)

Grant/Contract Number:: DMS-1115887; AC05-76RL01830

OSTI ID:: 1182897

Alternate ID(s):: OSTI ID: 1367756

Report Number(s):: PNNL-SA-105408; KJ0401000

Journal Information:: Journal of Computational Physics, Vol. 294, Issue C; ISSN 0021-9991

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

Citation Metrics:

Cited by: 15 works

Citation information provided by
Web of Science

References (21)

Identification of multi-modal random variables through mixtures of polynomial chaos expansions Nouy, Anthony Comptes Rendus Mécanique, Vol. 338, Issue 12 https://doi.org/10.1016/j.crme.2010.09.003	journal	December 2010
Inverse problems: A Bayesian perspective Stuart, A. M. Acta Numerica, Vol. 19 https://doi.org/10.1017/S0962492910000061	journal	May 2010
Adaptive independence samplers Keith, Jonathan M.; Kroese, Dirk P.; Sofronov, George Y. Statistics and Computing, Vol. 18, Issue 4 https://doi.org/10.1007/s11222-008-9070-2	journal	May 2008
Adaptive importance sampling in general mixture classes Cappé, Olivier; Douc, Randal; Guillin, Arnaud Statistics and Computing, Vol. 18, Issue 4 https://doi.org/10.1007/s11222-008-9059-x	journal	April 2008
Adaptive Construction of Surrogates for the Bayesian Solution of Inverse Problems Li, Jinglai; Marzouk, Youssef M. SIAM Journal on Scientific Computing, Vol. 36, Issue 3 https://doi.org/10.1137/130938189	journal	January 2014
Bayesian calibration of computer models Kennedy, Marc C.; O'Hagan, Anthony Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 63, Issue 3 https://doi.org/10.1111/1467-9868.00294	journal	August 2001
Combining Field Data and Computer Simulations for Calibration and Prediction Higdon, Dave; Kennedy, Marc; Cavendish, James C. SIAM Journal on Scientific Computing, Vol. 26, Issue 2 https://doi.org/10.1137/S1064827503426693	journal	January 2004
Bayesian Calibration and Uncertainty Analysis for Computationally Expensive Models Using Optimization and Radial Basis Function Approximation Bliznyuk, Nikolay; Ruppert, David; Shoemaker, Christine Journal of Computational and Graphical Statistics, Vol. 17, Issue 2 https://doi.org/10.1198/106186008X320681	journal	June 2008
Multiple-pattern parameter identification and uncertainty analysis approach for water quality modeling Zou, Rui; Lung, Wu-Seng; Wu, Jing Ecological Modelling, Vol. 220, Issue 5 https://doi.org/10.1016/j.ecolmodel.2008.11.021	journal	March 2009
The Homogeneous Chaos Wiener, Norbert American Journal of Mathematics, Vol. 60, Issue 4 https://doi.org/10.2307/2371268	journal	October 1938
Modeling uncertainty in flow simulations via generalized polynomial chaos Xiu, Dongbin; Karniadakis, George Em Journal of Computational Physics, Vol. 187, Issue 1 https://doi.org/10.1016/S0021-9991(03)00092-5	journal	May 2003
Stochastic analysis of unsaturated flow with probabilistic collocation method: PCM MODELING OF UNSATURATED FLOW Li, Weixuan; Lu, Zhiming; Zhang, Dongxiao Water Resources Research, Vol. 45, Issue 8 https://doi.org/10.1029/2008WR007530	journal	August 2009
Characterization of reservoir simulation models using a polynomial chaos-based ensemble Kalman filter: CHARACTERIZATION OF RESERVOIR SIMULATION MODELS Saad, George; Ghanem, Roger Water Resources Research, Vol. 45, Issue 4 https://doi.org/10.1029/2008WR007148	journal	April 2009
Stochastic spectral methods for efficient Bayesian solution of inverse problems Marzouk, Youssef M.; Najm, Habib N.; Rahn, Larry A. Journal of Computational Physics, Vol. 224, Issue 2 https://doi.org/10.1016/j.jcp.2006.10.010	journal	June 2007
An adaptive ANOVA-based PCKF for high-dimensional nonlinear inverse modeling Li, Weixuan; Lin, Guang; Zhang, Dongxiao Journal of Computational Physics, Vol. 258 https://doi.org/10.1016/j.jcp.2013.11.019	journal	February 2014
An adaptive multi-element generalized polynomial chaos method for stochastic differential equations Wan, Xiaoliang; Karniadakis, George Em Journal of Computational Physics, Vol. 209, Issue 2 https://doi.org/10.1016/j.jcp.2005.03.023	journal	November 2005
Annealed importance sampling Neal, Radford M. Statistics and Computing, Vol. 11, Issue 2, p. 125-139 https://doi.org/10.1023/A:1008923215028	journal	April 2001
On the Convergence Properties of the EM Algorithm Wu, C. F. Jeff The Annals of Statistics, Vol. 11, Issue 1 https://doi.org/10.1214/aos/1176346060	journal	March 1983
Errors in History Matching Tavassoli, Z.; Carter, Jonathan N.; King, Peter R. SPE Journal, Vol. 9, Issue 03 https://doi.org/10.2118/86883-PA	journal	September 2004
Our calibrated model has poor predictive value: An example from the petroleum industry Carter, J. N.; Ballester, P. J.; Tavassoli, Z. Reliability Engineering & System Safety, Vol. 91, Issue 10-11 https://doi.org/10.1016/j.ress.2005.11.033	journal	October 2006
Uncertainty quantification for porous media flows Christie, Mike; Demyanov, Vasily; Erbas, Demet Journal of Computational Physics, Vol. 217, Issue 1 https://doi.org/10.1016/j.jcp.2006.01.026	journal	September 2006

Similar Records

An adaptive importance sampling algorithm for Bayesian inversion with multimodal distributions

Journal Article · Sat Aug 01 00:00:00 EDT 2015 · Journal of Computational Physics · OSTI ID:1182897

Li, Weixuan; Lin, Guang

ADAPTIVE ANNEALED IMPORTANCE SAMPLING FOR MULTIMODAL POSTERIOR EXPLORATION AND MODEL SELECTION WITH APPLICATION TO EXTRASOLAR PLANET DETECTION

Journal Article · Tue Jul 01 00:00:00 EDT 2014 · Astrophysical Journal, Supplement Series · OSTI ID:1182897

Liu, Bin

Multi-variance replica exchange SGMCMC for inverse and forward problems via Bayesian PINN

Journal Article · Wed Mar 30 00:00:00 EDT 2022 · Journal of Computational Physics · OSTI ID:1182897

Lin, Guang; Wang, Yating; Zhang, Zecheng

Related Subjects

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS
inverse modeling
uncertainty reduction
adaptive sampling
Gaussian mixture
mixture of polynomial chaos expansions

Title: An adaptive importance sampling algorithm for Bayesian inversion with multimodal distributions

Citation Formats

References (21)

Similar Records

Related Subjects