Data–driven model reduction for the Bayesian solution of inverse problems
Abstract
One of the prime challenges in the Bayesian solution of inverse problems governed by partial differential equations (PDEs) is the computational cost of repeatedly evaluating numerical PDE models, as required by Markov chain Monte Carlo (MCMC) methods for posterior sampling. This paper presents a data-driven projection-based model reduction technique to reduce this computational cost. The proposed technique has two distinctive features. To begin, the model reduction strategy is tailored to inverse problems: the snapshots used to construct the reduced-order model are computed adaptively from the posterior distribution. Posterior exploration and model reduction are thus pursued simultaneously. Second, to avoid repeated evaluations of the full-scale numerical model as in a standard MCMC method, we couple the full-scale model and the reduced-order model together in the MCMC algorithm. This maintains accurate inference while reducing its overall computational cost. In numerical experiments considering steady-state flow in a porous medium, the data-driven reduced-order model achieves better accuracy than a reduced-order model constructed using the classical approach. It also improves posterior sampling efficiency by several orders of magnitude compared with a standard MCMC method.
- Authors:
-
- Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States)
- Publication Date:
- Research Org.:
- Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
- OSTI Identifier:
- 1557833
- Grant/Contract Number:
- SC0009297; FG02‐08ER2585
- Resource Type:
- Accepted Manuscript
- Journal Name:
- International Journal for Numerical Methods in Engineering
- Additional Journal Information:
- Journal Volume: 102; Journal Issue: 5; Journal ID: ISSN 0029-5981
- Publisher:
- Wiley
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 97 MATHEMATICS AND COMPUTING; model reduction; inverse problem; adaptive Markov chain Monte Carlo; approximate Bayesian inference
Citation Formats
Cui, Tiangang, Marzouk, Youssef M., and Willcox, Karen E. Data–driven model reduction for the Bayesian solution of inverse problems. United States: N. p., 2014.
Web. doi:10.1002/nme.4748.
Cui, Tiangang, Marzouk, Youssef M., & Willcox, Karen E. Data–driven model reduction for the Bayesian solution of inverse problems. United States. https://doi.org/10.1002/nme.4748
Cui, Tiangang, Marzouk, Youssef M., and Willcox, Karen E. Fri .
"Data–driven model reduction for the Bayesian solution of inverse problems". United States. https://doi.org/10.1002/nme.4748. https://www.osti.gov/servlets/purl/1557833.
@article{osti_1557833,
title = {Data–driven model reduction for the Bayesian solution of inverse problems},
author = {Cui, Tiangang and Marzouk, Youssef M. and Willcox, Karen E.},
abstractNote = {One of the prime challenges in the Bayesian solution of inverse problems governed by partial differential equations (PDEs) is the computational cost of repeatedly evaluating numerical PDE models, as required by Markov chain Monte Carlo (MCMC) methods for posterior sampling. This paper presents a data-driven projection-based model reduction technique to reduce this computational cost. The proposed technique has two distinctive features. To begin, the model reduction strategy is tailored to inverse problems: the snapshots used to construct the reduced-order model are computed adaptively from the posterior distribution. Posterior exploration and model reduction are thus pursued simultaneously. Second, to avoid repeated evaluations of the full-scale numerical model as in a standard MCMC method, we couple the full-scale model and the reduced-order model together in the MCMC algorithm. This maintains accurate inference while reducing its overall computational cost. In numerical experiments considering steady-state flow in a porous medium, the data-driven reduced-order model achieves better accuracy than a reduced-order model constructed using the classical approach. It also improves posterior sampling efficiency by several orders of magnitude compared with a standard MCMC method.},
doi = {10.1002/nme.4748},
journal = {International Journal for Numerical Methods in Engineering},
number = 5,
volume = 102,
place = {United States},
year = {Fri Aug 15 00:00:00 EDT 2014},
month = {Fri Aug 15 00:00:00 EDT 2014}
}
Web of Science
Figures / Tables:
Works referenced in this record:
Inverse Problem Theory and Methods for Model Parameter Estimation
book, January 2005
- Tarantola, Albert
Bayesian calibration of a large-scale geothermal reservoir model by a new adaptive delayed acceptance Metropolis Hastings algorithm: ADAPTIVE DELAYED ACCEPTANCE METROPOLIS-HASTINGS ALGORITHM
journal, October 2011
- Cui, T.; Fox, C.; O'Sullivan, M. J.
- Water Resources Research, Vol. 47, Issue 10
Statistical inversion of South Atlantic circulation in an abyssal neutral density layer
journal, July 2005
- McKeague, Ian W.; Nicholls, Geoff; Speer, Kevin
- Journal of Marine Research, Vol. 63, Issue 4
Markov chain Monte Carlo methods for high dimensional inversion in remote sensing
journal, August 2004
- Haario, H.; Laine, M.; Lehtinen, M.
- Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 66, Issue 3
A Stochastic Newton MCMC Method for Large-Scale Statistical Inverse Problems with Application to Seismic Inversion
journal, January 2012
- Martin, James; Wilcox, Lucas C.; Burstedde, Carsten
- SIAM Journal on Scientific Computing, Vol. 34, Issue 3
Stochastic spectral methods for efficient Bayesian solution of inverse problems
journal, June 2007
- Marzouk, Youssef M.; Najm, Habib N.; Rahn, Larry A.
- Journal of Computational Physics, Vol. 224, Issue 2
Dimensionality reduction and polynomial chaos acceleration of Bayesian inference in inverse problems
journal, April 2009
- Marzouk, Youssef M.; Najm, Habib N.
- Journal of Computational Physics, Vol. 228, Issue 6
Predicting Vehicle Crashworthiness: Validation of Computer Models for Functional and Hierarchical Data
journal, September 2009
- Bayarri, M. J.; Berger, James O.; Kennedy, Marc C.
- Journal of the American Statistical Association, Vol. 104, Issue 487
Electrical impedance tomography imaging with reduced-order model based on proper orthogonal decomposition
journal, April 2013
- Lipponen, Antti; Seppänen, Aku; Kaipio, Jari
- Journal of Electronic Imaging, Vol. 22, Issue 2
Parameter and State Model Reduction for Large-Scale Statistical Inverse Problems
journal, January 2010
- Lieberman, Chad; Willcox, Karen; Ghattas, Omar
- SIAM Journal on Scientific Computing, Vol. 32, Issue 5
Using Bayesian statistics in the estimation of heat source in radiation
journal, January 2005
- Wang, Jingbo; Zabaras, Nicholas
- International Journal of Heat and Mass Transfer, Vol. 48, Issue 1
Turbulence and the dynamics of coherent structures. I. Coherent structures
journal, January 1987
- Sirovich, Lawrence
- Quarterly of Applied Mathematics, Vol. 45, Issue 3
Adaptive Reduced-Order Controllers for a Thermal Flow System Using Proper Orthogonal Decomposition
journal, January 2002
- Ravindran, S. S.
- SIAM Journal on Scientific Computing, Vol. 23, Issue 6
Proper orthogonal decomposition for optimality systems
journal, January 2008
- Kunisch, Karl; Volkwein, Stefan
- ESAIM: Mathematical Modelling and Numerical Analysis, Vol. 42, Issue 1
Monte Carlo sampling methods using Markov chains and their applications
journal, April 1970
- Hastings, W. K.
- Biometrika, Vol. 57, Issue 1
Equation of State Calculations by Fast Computing Machines
journal, June 1953
- Metropolis, Nicholas; Rosenbluth, Arianna W.; Rosenbluth, Marshall N.
- The Journal of Chemical Physics, Vol. 21, Issue 6
An Adaptive Metropolis Algorithm
journal, April 2001
- Haario, Heikki; Saksman, Eero; Tamminen, Johanna
- Bernoulli, Vol. 7, Issue 2
Coupling and Ergodicity of Adaptive Markov Chain Monte Carlo Algorithms
journal, June 2007
- Roberts, Gareth O.; Rosenthal, Jeffrey S.
- Journal of Applied Probability, Vol. 44, Issue 02
Riemann manifold Langevin and Hamiltonian Monte Carlo methods: Riemann Manifold Langevin and Hamiltonian Monte Carlo Methods
journal, March 2011
- Girolami, Mark; Calderhead, Ben
- Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 73, Issue 2
Sequential Monte Carlo Methods for Dynamic Systems
journal, September 1998
- Liu, Jun S.; Chen, Rong
- Journal of the American Statistical Association, Vol. 93, Issue 443
Markov chain Monte Carlo Using an Approximation
journal, December 2005
- Christen, J. Andrés; Fox, Colin
- Journal of Computational and Graphical Statistics, Vol. 14, Issue 4
Missing Point Estimation in Models Described by Proper Orthogonal Decomposition
journal, November 2008
- Astrid, Patricia; Weiland, Siep; Willcox, Karen
- IEEE Transactions on Automatic Control, Vol. 53, Issue 10
An ‘empirical interpolation’ method: application to efficient reduced-basis discretization of partial differential equations
journal, November 2004
- Barrault, Maxime; Maday, Yvon; Nguyen, Ngoc Cuong
- Comptes Rendus Mathematique, Vol. 339, Issue 9
Nonlinear Model Reduction via Discrete Empirical Interpolation
journal, January 2010
- Chaturantabut, Saifon; Sorensen, Danny C.
- SIAM Journal on Scientific Computing, Vol. 32, Issue 5
A training set and multiple bases generation approach for parameterized model reduction based on adaptive grids in parameter space
journal, August 2011
- Haasdonk, Bernard; Dihlmann, Markus; Ohlberger, Mario
- Mathematical and Computer Modelling of Dynamical Systems, Vol. 17, Issue 4
Nonlinear model order reduction based on local reduced-order bases: NONLINEAR MODEL REDUCTION BASED ON LOCAL REDUCED-ORDER BASES
journal, June 2012
- Amsallem, David; Zahr, Matthew J.; Farhat, Charbel
- International Journal for Numerical Methods in Engineering, Vol. 92, Issue 10
Parameter multi-domain ‘hp’ empirical interpolation: PARAMETER MULTI-DOMAIN ‘hp’ EMPIRICAL INTERPOLATION
journal, March 2012
- Eftang, Jens L.; Stamm, Benjamin
- International Journal for Numerical Methods in Engineering, Vol. 90, Issue 4
Localized Discrete Empirical Interpolation Method
journal, January 2014
- Peherstorfer, Benjamin; Butnaru, Daniel; Willcox, Karen
- SIAM Journal on Scientific Computing, Vol. 36, Issue 1
Efficient model reduction in non-linear dynamics using the Karhunen-Lo�ve expansion and dual-weighted-residual methods
journal, May 2003
- Meyer, M.; Matthies, H. G.
- Computational Mechanics, Vol. 31, Issue 1-2
Model Reduction for Large-Scale Systems with High-Dimensional Parametric Input Space
journal, January 2008
- Bui-Thanh, T.; Willcox, K.; Ghattas, O.
- SIAM Journal on Scientific Computing, Vol. 30, Issue 6
Reduced basis approximation and a posteriori error estimation for affinely parametrized elliptic coercive partial differential equations
journal, September 2007
- Rozza, G.; Huynh, D. B. P.; Patera, A. T.
- Archives of Computational Methods in Engineering, Vol. 15, Issue 3
Coupling and Ergodicity of Adaptive Markov Chain Monte Carlo Algorithms
journal, June 2007
- Roberts, Gareth O.; Rosenthal, Jeffrey S.
- Journal of Applied Probability, Vol. 44, Issue 2
Sequential Monte Carlo Methods for Dynamic Systems
journal, September 1998
- Liu, Jun S.; Chen, Rong
- Journal of the American Statistical Association, Vol. 93, Issue 443
Coupling and Ergodicity of Adaptive Markov Chain Monte Carlo Algorithms
journal, June 2007
- Roberts, Gareth O.; Rosenthal, Jeffrey S.
- Journal of Applied Probability, Vol. 44, Issue 02
Missing point estimation in models described by proper orthogonal decomposition
conference, January 2004
- Astrid, P.; Weiland, S.; Willcox, K.
- 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601)
Reduced Basis Approximation and a Posteriori Error Estimation for Affinely Parametrized Elliptic Coercive Partial Differential Equations: Application to Transport and Continuum Mechanics
journal, May 2008
- Rozza, G.; Huynh, D. B. P.; Patera, A. T.
- Archives of Computational Methods in Engineering, Vol. 15, Issue 3
Stochastic spectral methods for efficient Bayesian solution of inverse problems
conference, January 2005
- Marzouk, Youssef M.
- BAYESIAN INFERENCE AND MAXIMUM ENTROPY METHODS IN SCIENCE AND ENGINEERING: 25th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering, AIP Conference Proceedings
Efficient Model Reduction in Non-linear Dynamics Using the Karhunen-Loève Expansion and Dual-Weighted-Residual Methods
text, January 2003
- Meyer, Marcus; Matthies, Hermann
- Institut für Wissenschaftliches Rechnen
Works referencing / citing this record:
An adaptive and efficient greedy procedure for the optimal training of parametric reduced-order models: ADAPTIVE GREEDY PROCEDURE FOR THE OPTIMAL TRAINING OF PARAMETRIC ROMS
journal, September 2014
- Paul-Dubois-Taine, A.; Amsallem, D.
- International Journal for Numerical Methods in Engineering, Vol. 102, Issue 5
A posteriori stochastic correction of reduced models in delayed-acceptance MCMC, with application to multiphase subsurface inverse problems: Stochastic correction of reduced models in delayed-acceptance MCMC
journal, March 2019
- Cui, Tiangang; Fox, Colin; O'Sullivan, Michael J.
- International Journal for Numerical Methods in Engineering, Vol. 118, Issue 10
Adaptive multi‐index collocation for uncertainty quantification and sensitivity analysis
journal, November 2019
- Jakeman, John D.; Eldred, Michael S.; Geraci, Gianluca
- International Journal for Numerical Methods in Engineering, Vol. 121, Issue 6
The Use of Radial Basis Function Surrogate Models for Sampling Process Acceleration in Bayesian Inversion
book, April 2019
- Domesová, Simona
- AETA 2018 - Recent Advances in Electrical Engineering and Related Sciences: Theory and Application
Multilevel model reduction for uncertainty quantification in computational structural dynamics
journal, October 2016
- Ezvan, O.; Batou, A.; Soize, C.
- Computational Mechanics, Vol. 59, Issue 2
Fast model updating coupling Bayesian inference and PGD model reduction
journal, April 2018
- Rubio, Paul-Baptiste; Louf, François; Chamoin, Ludovic
- Computational Mechanics, Vol. 62, Issue 6
A transport-based multifidelity preconditioner for Markov chain Monte Carlo
journal, November 2019
- Peherstorfer, Benjamin; Marzouk, Youssef
- Advances in Computational Mathematics
Surrogate accelerated Bayesian inversion for the determination of the thermal diffusivity of a material
journal, January 2019
- Rynn, James A. J.; Cotter, Simon L.; Powell, Catherine E.
- Metrologia, Vol. 56, Issue 1
Statistical Inference Method for Liner Impedance Eduction with a Shear Grazing Flow
journal, March 2019
- Roncen, R.; Méry, F.; Piot, E.
- AIAA Journal, Vol. 57, Issue 3
Adaptivity in Bayesian Inverse Finite Element Problems: Learning and Simultaneous Control of Discretisation and Sampling Errors
journal, February 2019
- Kerfriden, Pierre; Kundu, Abhishek; Claus, Susanne
- Materials, Vol. 12, Issue 4
Joint state-parameter estimation of a nonlinear stochastic energy balance model from sparse noisy data
journal, January 2019
- Lu, Fei; Weitzel, Nils; Monahan, Adam H.
- Nonlinear Processes in Geophysics, Vol. 26, Issue 3
Adaptive multi‐index collocation for uncertainty quantification and sensitivity analysis
journal, August 2020
- Jakeman, John D.; Eldred, Michael S.; Geraci, Gianluca
- International Journal for Numerical Methods in Engineering, Vol. 121, Issue 19
Bayesian Estimation for Stochastic Gene Expression Using Multifidelity Models
journal, February 2019
- Vo, Huy D.; Fox, Zachary; Baetica, Ania
- The Journal of Physical Chemistry B, Vol. 123, Issue 10
A transport-based multifidelity preconditioner for Markov chain Monte Carlo
preprint, January 2018
- Peherstorfer, Benjamin; Marzouk, Youssef
- arXiv
Joint state-parameter estimation of a nonlinear stochastic energy balance model from sparse noisy data
text, January 2019
- Lu, Fei; Weitzel, Nils; Monahan, Adam H.
- arXiv
Figures / Tables found in this record: