Data–driven model reduction for the Bayesian solution of inverse problems

Cui, Tiangang; Marzouk, Youssef M.; Willcox, Karen E.

doi:10.1002/nme.4748

Title: Data–driven model reduction for the Bayesian solution of inverse problems

Journal Article · Fri Aug 15 00:00:00 EDT 2014 · International Journal for Numerical Methods in Engineering

DOI:https://doi.org/10.1002/nme.4748· OSTI ID:1557833

Cui, Tiangang ^[1]; Marzouk, Youssef M. ^[1]; Willcox, Karen E. ^[1]

Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States)

One of the prime challenges in the Bayesian solution of inverse problems governed by partial differential equations (PDEs) is the computational cost of repeatedly evaluating numerical PDE models, as required by Markov chain Monte Carlo (MCMC) methods for posterior sampling. This paper presents a data-driven projection-based model reduction technique to reduce this computational cost. The proposed technique has two distinctive features. To begin, the model reduction strategy is tailored to inverse problems: the snapshots used to construct the reduced-order model are computed adaptively from the posterior distribution. Posterior exploration and model reduction are thus pursued simultaneously. Second, to avoid repeated evaluations of the full-scale numerical model as in a standard MCMC method, we couple the full-scale model and the reduced-order model together in the MCMC algorithm. This maintains accurate inference while reducing its overall computational cost. In numerical experiments considering steady-state flow in a porous medium, the data-driven reduced-order model achieves better accuracy than a reduced-order model constructed using the classical approach. It also improves posterior sampling efficiency by several orders of magnitude compared with a standard MCMC method.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)

Grant/Contract Number:: SC0009297; FG02‐08ER2585

OSTI ID:: 1557833

Journal Information:: International Journal for Numerical Methods in Engineering, Vol. 102, Issue 5; ISSN 0029-5981

Publisher:: WileyCopyright Statement

Country of Publication:: United States

Language:: English

Citation Metrics:

Cited by: 103 works

Citation information provided by
Web of Science

References (39)

Inverse Problem Theory and Methods for Model Parameter Estimation Tarantola, Albert https://doi.org/10.1137/1.9780898717921	book	January 2005
Bayesian calibration of a large-scale geothermal reservoir model by a new adaptive delayed acceptance Metropolis Hastings algorithm: ADAPTIVE DELAYED ACCEPTANCE METROPOLIS-HASTINGS ALGORITHM Cui, T.; Fox, C.; O'Sullivan, M. J. Water Resources Research, Vol. 47, Issue 10 https://doi.org/10.1029/2010WR010352	journal	October 2011
Statistical inversion of South Atlantic circulation in an abyssal neutral density layer McKeague, Ian W.; Nicholls, Geoff; Speer, Kevin Journal of Marine Research, Vol. 63, Issue 4 https://doi.org/10.1357/0022240054663240	journal	July 2005
Markov chain Monte Carlo methods for high dimensional inversion in remote sensing Haario, H.; Laine, M.; Lehtinen, M. Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 66, Issue 3 https://doi.org/10.1111/j.1467-9868.2004.02053.x	journal	August 2004
A Stochastic Newton MCMC Method for Large-Scale Statistical Inverse Problems with Application to Seismic Inversion Martin, James; Wilcox, Lucas C.; Burstedde, Carsten SIAM Journal on Scientific Computing, Vol. 34, Issue 3 https://doi.org/10.1137/110845598	journal	January 2012
Stochastic spectral methods for efficient Bayesian solution of inverse problems Marzouk, Youssef M.; Najm, Habib N.; Rahn, Larry A. Journal of Computational Physics, Vol. 224, Issue 2 https://doi.org/10.1016/j.jcp.2006.10.010	journal	June 2007
Dimensionality reduction and polynomial chaos acceleration of Bayesian inference in inverse problems Marzouk, Youssef M.; Najm, Habib N. Journal of Computational Physics, Vol. 228, Issue 6 https://doi.org/10.1016/j.jcp.2008.11.024	journal	April 2009
Predicting Vehicle Crashworthiness: Validation of Computer Models for Functional and Hierarchical Data Bayarri, M. J.; Berger, James O.; Kennedy, Marc C. Journal of the American Statistical Association, Vol. 104, Issue 487 https://doi.org/10.1198/jasa.2009.ap06623	journal	September 2009
Electrical impedance tomography imaging with reduced-order model based on proper orthogonal decomposition Lipponen, Antti; Seppänen, Aku; Kaipio, Jari Journal of Electronic Imaging, Vol. 22, Issue 2 https://doi.org/10.1117/1.JEI.22.2.023008	journal	April 2013
Parameter and State Model Reduction for Large-Scale Statistical Inverse Problems Lieberman, Chad; Willcox, Karen; Ghattas, Omar SIAM Journal on Scientific Computing, Vol. 32, Issue 5 https://doi.org/10.1137/090775622	journal	January 2010
Using Bayesian statistics in the estimation of heat source in radiation Wang, Jingbo; Zabaras, Nicholas International Journal of Heat and Mass Transfer, Vol. 48, Issue 1 https://doi.org/10.1016/j.ijheatmasstransfer.2004.08.009	journal	January 2005
Turbulence and the dynamics of coherent structures. I. Coherent structures Sirovich, Lawrence Quarterly of Applied Mathematics, Vol. 45, Issue 3 https://doi.org/10.1090/qam/910462	journal	January 1987
Adaptive Reduced-Order Controllers for a Thermal Flow System Using Proper Orthogonal Decomposition Ravindran, S. S. SIAM Journal on Scientific Computing, Vol. 23, Issue 6 https://doi.org/10.1137/S1064827500374716	journal	January 2002
Proper orthogonal decomposition for optimality systems Kunisch, Karl; Volkwein, Stefan ESAIM: Mathematical Modelling and Numerical Analysis, Vol. 42, Issue 1 https://doi.org/10.1051/m2an:2007054	journal	January 2008
Inverse problems: A Bayesian perspective Stuart, A. M. Acta Numerica, Vol. 19 https://doi.org/10.1017/S0962492910000061	journal	May 2010
Monte Carlo sampling methods using Markov chains and their applications Hastings, W. K. Biometrika, Vol. 57, Issue 1 https://doi.org/10.1093/biomet/57.1.97	journal	April 1970
Equation of State Calculations by Fast Computing Machines Metropolis, Nicholas; Rosenbluth, Arianna W.; Rosenbluth, Marshall N. The Journal of Chemical Physics, Vol. 21, Issue 6 https://doi.org/10.1063/1.1699114	journal	June 1953
An Adaptive Metropolis Algorithm Haario, Heikki; Saksman, Eero; Tamminen, Johanna Bernoulli, Vol. 7, Issue 2 https://doi.org/10.2307/3318737	journal	April 2001
Coupling and Ergodicity of Adaptive Markov Chain Monte Carlo Algorithms Roberts, Gareth O.; Rosenthal, Jeffrey S. Journal of Applied Probability, Vol. 44, Issue 02 https://doi.org/10.1017/S0021900200003090	journal	June 2007
Riemann manifold Langevin and Hamiltonian Monte Carlo methods: Riemann Manifold Langevin and Hamiltonian Monte Carlo Methods Girolami, Mark; Calderhead, Ben Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 73, Issue 2 https://doi.org/10.1111/j.1467-9868.2010.00765.x	journal	March 2011
Sequential Monte Carlo Methods for Dynamic Systems Liu, Jun S.; Chen, Rong Journal of the American Statistical Association, Vol. 93, Issue 443 https://doi.org/10.1080/01621459.1998.10473765	journal	September 1998
Markov chain Monte Carlo Using an Approximation Christen, J. Andrés; Fox, Colin Journal of Computational and Graphical Statistics, Vol. 14, Issue 4 https://doi.org/10.1198/106186005X76983	journal	December 2005
Missing Point Estimation in Models Described by Proper Orthogonal Decomposition Astrid, Patricia; Weiland, Siep; Willcox, Karen IEEE Transactions on Automatic Control, Vol. 53, Issue 10 https://doi.org/10.1109/TAC.2008.2006102	journal	November 2008
An ‘empirical interpolation’ method: application to efficient reduced-basis discretization of partial differential equations Barrault, Maxime; Maday, Yvon; Nguyen, Ngoc Cuong Comptes Rendus Mathematique, Vol. 339, Issue 9 https://doi.org/10.1016/j.crma.2004.08.006	journal	November 2004
Nonlinear Model Reduction via Discrete Empirical Interpolation Chaturantabut, Saifon; Sorensen, Danny C. SIAM Journal on Scientific Computing, Vol. 32, Issue 5 https://doi.org/10.1137/090766498	journal	January 2010
A training set and multiple bases generation approach for parameterized model reduction based on adaptive grids in parameter space Haasdonk, Bernard; Dihlmann, Markus; Ohlberger, Mario Mathematical and Computer Modelling of Dynamical Systems, Vol. 17, Issue 4 https://doi.org/10.1080/13873954.2011.547674	journal	August 2011
Nonlinear model order reduction based on local reduced-order bases: NONLINEAR MODEL REDUCTION BASED ON LOCAL REDUCED-ORDER BASES Amsallem, David; Zahr, Matthew J.; Farhat, Charbel International Journal for Numerical Methods in Engineering, Vol. 92, Issue 10 https://doi.org/10.1002/nme.4371	journal	June 2012
Parameter multi-domain ‘hp’ empirical interpolation: PARAMETER MULTI-DOMAIN ‘hp’ EMPIRICAL INTERPOLATION Eftang, Jens L.; Stamm, Benjamin International Journal for Numerical Methods in Engineering, Vol. 90, Issue 4 https://doi.org/10.1002/nme.3327	journal	March 2012
Localized Discrete Empirical Interpolation Method Peherstorfer, Benjamin; Butnaru, Daniel; Willcox, Karen SIAM Journal on Scientific Computing, Vol. 36, Issue 1 https://doi.org/10.1137/130924408	journal	January 2014
Efficient model reduction in non-linear dynamics using the Karhunen-Lo�ve expansion and dual-weighted-residual methods Meyer, M.; Matthies, H. G. Computational Mechanics, Vol. 31, Issue 1-2 https://doi.org/10.1007/s00466-002-0404-1	journal	May 2003
Model Reduction for Large-Scale Systems with High-Dimensional Parametric Input Space Bui-Thanh, T.; Willcox, K.; Ghattas, O. SIAM Journal on Scientific Computing, Vol. 30, Issue 6 https://doi.org/10.1137/070694855	journal	January 2008
Reduced basis approximation and a posteriori error estimation for affinely parametrized elliptic coercive partial differential equations Rozza, G.; Huynh, D. B. P.; Patera, A. T. Archives of Computational Methods in Engineering, Vol. 15, Issue 3 https://doi.org/10.1007/BF03024948	journal	September 2007
Coupling and Ergodicity of Adaptive Markov Chain Monte Carlo Algorithms Roberts, Gareth O.; Rosenthal, Jeffrey S. Journal of Applied Probability, Vol. 44, Issue 2 https://doi.org/10.1239/jap/1183667414	journal	June 2007
Sequential Monte Carlo Methods for Dynamic Systems Liu, Jun S.; Chen, Rong Journal of the American Statistical Association, Vol. 93, Issue 443 https://doi.org/10.2307/2669847	journal	September 1998
Coupling and Ergodicity of Adaptive Markov Chain Monte Carlo Algorithms Roberts, Gareth O.; Rosenthal, Jeffrey S. Journal of Applied Probability, Vol. 44, Issue 02 https://doi.org/10.1017/s0021900200117954	journal	June 2007
Missing point estimation in models described by proper orthogonal decomposition Astrid, P.; Weiland, S.; Willcox, K. 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601) https://doi.org/10.1109/cdc.2004.1430301	conference	January 2004
Reduced Basis Approximation and a Posteriori Error Estimation for Affinely Parametrized Elliptic Coercive Partial Differential Equations: Application to Transport and Continuum Mechanics Rozza, G.; Huynh, D. B. P.; Patera, A. T. Archives of Computational Methods in Engineering, Vol. 15, Issue 3 https://doi.org/10.1007/s11831-008-9019-9	journal	May 2008
Stochastic spectral methods for efficient Bayesian solution of inverse problems Marzouk, Youssef M. BAYESIAN INFERENCE AND MAXIMUM ENTROPY METHODS IN SCIENCE AND ENGINEERING: 25th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering, AIP Conference Proceedings https://doi.org/10.1063/1.2149785	conference	January 2005
Efficient Model Reduction in Non-linear Dynamics Using the Karhunen-Loève Expansion and Dual-Weighted-Residual Methods Meyer, Marcus; Matthies, Hermann Institut für Wissenschaftliches Rechnen https://doi.org/10.24355/dbbs.084-200511080100-575	text	January 2003

Cited By (16)

An adaptive and efficient greedy procedure for the optimal training of parametric reduced-order models: ADAPTIVE GREEDY PROCEDURE FOR THE OPTIMAL TRAINING OF PARAMETRIC ROMS Paul-Dubois-Taine, A.; Amsallem, D. International Journal for Numerical Methods in Engineering, Vol. 102, Issue 5 https://doi.org/10.1002/nme.4759	journal	September 2014
A posteriori stochastic correction of reduced models in delayed-acceptance MCMC, with application to multiphase subsurface inverse problems: Stochastic correction of reduced models in delayed-acceptance MCMC Cui, Tiangang; Fox, Colin; O'Sullivan, Michael J. International Journal for Numerical Methods in Engineering, Vol. 118, Issue 10 https://doi.org/10.1002/nme.6028	journal	March 2019
Adaptive multi‐index collocation for uncertainty quantification and sensitivity analysis Jakeman, John D.; Eldred, Michael S.; Geraci, Gianluca International Journal for Numerical Methods in Engineering, Vol. 121, Issue 6 https://doi.org/10.1002/nme.6268	journal	November 2019
The Use of Radial Basis Function Surrogate Models for Sampling Process Acceleration in Bayesian Inversion Domesová, Simona AETA 2018 - Recent Advances in Electrical Engineering and Related Sciences: Theory and Application https://doi.org/10.1007/978-3-030-14907-9_23	book	April 2019
Multilevel model reduction for uncertainty quantification in computational structural dynamics Ezvan, O.; Batou, A.; Soize, C. Computational Mechanics, Vol. 59, Issue 2 https://doi.org/10.1007/s00466-016-1348-1	journal	October 2016
Fast model updating coupling Bayesian inference and PGD model reduction Rubio, Paul-Baptiste; Louf, François; Chamoin, Ludovic Computational Mechanics, Vol. 62, Issue 6 https://doi.org/10.1007/s00466-018-1575-8	journal	April 2018
A transport-based multifidelity preconditioner for Markov chain Monte Carlo Peherstorfer, Benjamin; Marzouk, Youssef Advances in Computational Mathematics https://doi.org/10.1007/s10444-019-09711-y	journal	November 2019
Surrogate accelerated Bayesian inversion for the determination of the thermal diffusivity of a material Rynn, James A. J.; Cotter, Simon L.; Powell, Catherine E. Metrologia, Vol. 56, Issue 1 https://doi.org/10.1088/1681-7575/aaf984	journal	January 2019
Statistical Inference Method for Liner Impedance Eduction with a Shear Grazing Flow Roncen, R.; Méry, F.; Piot, E. AIAA Journal, Vol. 57, Issue 3 https://doi.org/10.2514/1.j057559	journal	March 2019
Adaptivity in Bayesian Inverse Finite Element Problems: Learning and Simultaneous Control of Discretisation and Sampling Errors Kerfriden, Pierre; Kundu, Abhishek; Claus, Susanne Materials, Vol. 12, Issue 4 https://doi.org/10.3390/ma12040642	journal	February 2019
Joint state-parameter estimation of a nonlinear stochastic energy balance model from sparse noisy data Lu, Fei; Weitzel, Nils; Monahan, Adam H. Nonlinear Processes in Geophysics, Vol. 26, Issue 3 https://doi.org/10.5194/npg-26-227-2019	journal	January 2019
Adaptive multi‐index collocation for uncertainty quantification and sensitivity analysis Jakeman, John D.; Eldred, Michael S.; Geraci, Gianluca International Journal for Numerical Methods in Engineering, Vol. 121, Issue 19 https://doi.org/10.1002/nme.6450	journal	August 2020
Bayesian Estimation for Stochastic Gene Expression Using Multifidelity Models Vo, Huy D.; Fox, Zachary; Baetica, Ania The Journal of Physical Chemistry B, Vol. 123, Issue 10 https://doi.org/10.1021/acs.jpcb.8b10946	journal	February 2019
Adaptive multi-index collocation for uncertainty quantification and sensitivity analysis Jakeman, John Davis; Eldred, Michael S.; Geraci, G. https://doi.org/10.2172/1574406	report	November 2019
A transport-based multifidelity preconditioner for Markov chain Monte Carlo Peherstorfer, Benjamin; Marzouk, Youssef arXiv https://doi.org/10.48550/arxiv.1808.09379	preprint	January 2018
Joint state-parameter estimation of a nonlinear stochastic energy balance model from sparse noisy data Lu, Fei; Weitzel, Nils; Monahan, Adam H. arXiv https://doi.org/10.48550/arxiv.1904.05310	text	January 2019