DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Data–driven model reduction for the Bayesian solution of inverse problems

Abstract

One of the prime challenges in the Bayesian solution of inverse problems governed by partial differential equations (PDEs) is the computational cost of repeatedly evaluating numerical PDE models, as required by Markov chain Monte Carlo (MCMC) methods for posterior sampling. This paper presents a data-driven projection-based model reduction technique to reduce this computational cost. The proposed technique has two distinctive features. To begin, the model reduction strategy is tailored to inverse problems: the snapshots used to construct the reduced-order model are computed adaptively from the posterior distribution. Posterior exploration and model reduction are thus pursued simultaneously. Second, to avoid repeated evaluations of the full-scale numerical model as in a standard MCMC method, we couple the full-scale model and the reduced-order model together in the MCMC algorithm. This maintains accurate inference while reducing its overall computational cost. In numerical experiments considering steady-state flow in a porous medium, the data-driven reduced-order model achieves better accuracy than a reduced-order model constructed using the classical approach. It also improves posterior sampling efficiency by several orders of magnitude compared with a standard MCMC method.

Authors:
 [1];  [1];  [1]
  1. Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States)
Publication Date:
Research Org.:
Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
OSTI Identifier:
1557833
Grant/Contract Number:  
SC0009297; FG02‐08ER2585
Resource Type:
Accepted Manuscript
Journal Name:
International Journal for Numerical Methods in Engineering
Additional Journal Information:
Journal Volume: 102; Journal Issue: 5; Journal ID: ISSN 0029-5981
Publisher:
Wiley
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; model reduction; inverse problem; adaptive Markov chain Monte Carlo; approximate Bayesian inference

Citation Formats

Cui, Tiangang, Marzouk, Youssef M., and Willcox, Karen E. Data–driven model reduction for the Bayesian solution of inverse problems. United States: N. p., 2014. Web. doi:10.1002/nme.4748.
Cui, Tiangang, Marzouk, Youssef M., & Willcox, Karen E. Data–driven model reduction for the Bayesian solution of inverse problems. United States. https://doi.org/10.1002/nme.4748
Cui, Tiangang, Marzouk, Youssef M., and Willcox, Karen E. Fri . "Data–driven model reduction for the Bayesian solution of inverse problems". United States. https://doi.org/10.1002/nme.4748. https://www.osti.gov/servlets/purl/1557833.
@article{osti_1557833,
title = {Data–driven model reduction for the Bayesian solution of inverse problems},
author = {Cui, Tiangang and Marzouk, Youssef M. and Willcox, Karen E.},
abstractNote = {One of the prime challenges in the Bayesian solution of inverse problems governed by partial differential equations (PDEs) is the computational cost of repeatedly evaluating numerical PDE models, as required by Markov chain Monte Carlo (MCMC) methods for posterior sampling. This paper presents a data-driven projection-based model reduction technique to reduce this computational cost. The proposed technique has two distinctive features. To begin, the model reduction strategy is tailored to inverse problems: the snapshots used to construct the reduced-order model are computed adaptively from the posterior distribution. Posterior exploration and model reduction are thus pursued simultaneously. Second, to avoid repeated evaluations of the full-scale numerical model as in a standard MCMC method, we couple the full-scale model and the reduced-order model together in the MCMC algorithm. This maintains accurate inference while reducing its overall computational cost. In numerical experiments considering steady-state flow in a porous medium, the data-driven reduced-order model achieves better accuracy than a reduced-order model constructed using the classical approach. It also improves posterior sampling efficiency by several orders of magnitude compared with a standard MCMC method.},
doi = {10.1002/nme.4748},
journal = {International Journal for Numerical Methods in Engineering},
number = 5,
volume = 102,
place = {United States},
year = {Fri Aug 15 00:00:00 EDT 2014},
month = {Fri Aug 15 00:00:00 EDT 2014}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Citation Metrics:
Cited by: 103 works
Citation information provided by
Web of Science

Figures / Tables:

Figure 1 Figure 1: A two-dimensional example demonstrating data-driven model reduction. The prior distribution and posterior distribution are represented by the blue contours and red contours, respectively. The black dots represent samples used for computing the snapshots in the reduced-order model construction. Left: sampling using the classical approach (from the prior). Right:more » our data-driven approach.« less

Save / Share:

Works referenced in this record:

Statistical inversion of South Atlantic circulation in an abyssal neutral density layer
journal, July 2005

  • McKeague, Ian W.; Nicholls, Geoff; Speer, Kevin
  • Journal of Marine Research, Vol. 63, Issue 4
  • DOI: 10.1357/0022240054663240

Markov chain Monte Carlo methods for high dimensional inversion in remote sensing
journal, August 2004

  • Haario, H.; Laine, M.; Lehtinen, M.
  • Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 66, Issue 3
  • DOI: 10.1111/j.1467-9868.2004.02053.x

A Stochastic Newton MCMC Method for Large-Scale Statistical Inverse Problems with Application to Seismic Inversion
journal, January 2012

  • Martin, James; Wilcox, Lucas C.; Burstedde, Carsten
  • SIAM Journal on Scientific Computing, Vol. 34, Issue 3
  • DOI: 10.1137/110845598

Stochastic spectral methods for efficient Bayesian solution of inverse problems
journal, June 2007

  • Marzouk, Youssef M.; Najm, Habib N.; Rahn, Larry A.
  • Journal of Computational Physics, Vol. 224, Issue 2
  • DOI: 10.1016/j.jcp.2006.10.010

Dimensionality reduction and polynomial chaos acceleration of Bayesian inference in inverse problems
journal, April 2009


Predicting Vehicle Crashworthiness: Validation of Computer Models for Functional and Hierarchical Data
journal, September 2009

  • Bayarri, M. J.; Berger, James O.; Kennedy, Marc C.
  • Journal of the American Statistical Association, Vol. 104, Issue 487
  • DOI: 10.1198/jasa.2009.ap06623

Electrical impedance tomography imaging with reduced-order model based on proper orthogonal decomposition
journal, April 2013

  • Lipponen, Antti; Seppänen, Aku; Kaipio, Jari
  • Journal of Electronic Imaging, Vol. 22, Issue 2
  • DOI: 10.1117/1.JEI.22.2.023008

Parameter and State Model Reduction for Large-Scale Statistical Inverse Problems
journal, January 2010

  • Lieberman, Chad; Willcox, Karen; Ghattas, Omar
  • SIAM Journal on Scientific Computing, Vol. 32, Issue 5
  • DOI: 10.1137/090775622

Using Bayesian statistics in the estimation of heat source in radiation
journal, January 2005


Turbulence and the dynamics of coherent structures. I. Coherent structures
journal, January 1987

  • Sirovich, Lawrence
  • Quarterly of Applied Mathematics, Vol. 45, Issue 3
  • DOI: 10.1090/qam/910462

Adaptive Reduced-Order Controllers for a Thermal Flow System Using Proper Orthogonal Decomposition
journal, January 2002


Proper orthogonal decomposition for optimality systems
journal, January 2008

  • Kunisch, Karl; Volkwein, Stefan
  • ESAIM: Mathematical Modelling and Numerical Analysis, Vol. 42, Issue 1
  • DOI: 10.1051/m2an:2007054

Inverse problems: A Bayesian perspective
journal, May 2010


Monte Carlo sampling methods using Markov chains and their applications
journal, April 1970


Equation of State Calculations by Fast Computing Machines
journal, June 1953

  • Metropolis, Nicholas; Rosenbluth, Arianna W.; Rosenbluth, Marshall N.
  • The Journal of Chemical Physics, Vol. 21, Issue 6
  • DOI: 10.1063/1.1699114

An Adaptive Metropolis Algorithm
journal, April 2001

  • Haario, Heikki; Saksman, Eero; Tamminen, Johanna
  • Bernoulli, Vol. 7, Issue 2
  • DOI: 10.2307/3318737

Coupling and Ergodicity of Adaptive Markov Chain Monte Carlo Algorithms
journal, June 2007


Riemann manifold Langevin and Hamiltonian Monte Carlo methods: Riemann Manifold Langevin and Hamiltonian Monte Carlo Methods
journal, March 2011

  • Girolami, Mark; Calderhead, Ben
  • Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 73, Issue 2
  • DOI: 10.1111/j.1467-9868.2010.00765.x

Sequential Monte Carlo Methods for Dynamic Systems
journal, September 1998


Markov chain Monte Carlo Using an Approximation
journal, December 2005

  • Christen, J. Andrés; Fox, Colin
  • Journal of Computational and Graphical Statistics, Vol. 14, Issue 4
  • DOI: 10.1198/106186005X76983

Missing Point Estimation in Models Described by Proper Orthogonal Decomposition
journal, November 2008

  • Astrid, Patricia; Weiland, Siep; Willcox, Karen
  • IEEE Transactions on Automatic Control, Vol. 53, Issue 10
  • DOI: 10.1109/TAC.2008.2006102

An ‘empirical interpolation’ method: application to efficient reduced-basis discretization of partial differential equations
journal, November 2004

  • Barrault, Maxime; Maday, Yvon; Nguyen, Ngoc Cuong
  • Comptes Rendus Mathematique, Vol. 339, Issue 9
  • DOI: 10.1016/j.crma.2004.08.006

Nonlinear Model Reduction via Discrete Empirical Interpolation
journal, January 2010

  • Chaturantabut, Saifon; Sorensen, Danny C.
  • SIAM Journal on Scientific Computing, Vol. 32, Issue 5
  • DOI: 10.1137/090766498

A training set and multiple bases generation approach for parameterized model reduction based on adaptive grids in parameter space
journal, August 2011

  • Haasdonk, Bernard; Dihlmann, Markus; Ohlberger, Mario
  • Mathematical and Computer Modelling of Dynamical Systems, Vol. 17, Issue 4
  • DOI: 10.1080/13873954.2011.547674

Nonlinear model order reduction based on local reduced-order bases: NONLINEAR MODEL REDUCTION BASED ON LOCAL REDUCED-ORDER BASES
journal, June 2012

  • Amsallem, David; Zahr, Matthew J.; Farhat, Charbel
  • International Journal for Numerical Methods in Engineering, Vol. 92, Issue 10
  • DOI: 10.1002/nme.4371

Parameter multi-domain ‘hp’ empirical interpolation: PARAMETER MULTI-DOMAIN ‘hp’ EMPIRICAL INTERPOLATION
journal, March 2012

  • Eftang, Jens L.; Stamm, Benjamin
  • International Journal for Numerical Methods in Engineering, Vol. 90, Issue 4
  • DOI: 10.1002/nme.3327

Localized Discrete Empirical Interpolation Method
journal, January 2014

  • Peherstorfer, Benjamin; Butnaru, Daniel; Willcox, Karen
  • SIAM Journal on Scientific Computing, Vol. 36, Issue 1
  • DOI: 10.1137/130924408

Efficient model reduction in non-linear dynamics using the Karhunen-Lo�ve expansion and dual-weighted-residual methods
journal, May 2003


Model Reduction for Large-Scale Systems with High-Dimensional Parametric Input Space
journal, January 2008

  • Bui-Thanh, T.; Willcox, K.; Ghattas, O.
  • SIAM Journal on Scientific Computing, Vol. 30, Issue 6
  • DOI: 10.1137/070694855

Reduced basis approximation and a posteriori error estimation for affinely parametrized elliptic coercive partial differential equations
journal, September 2007

  • Rozza, G.; Huynh, D. B. P.; Patera, A. T.
  • Archives of Computational Methods in Engineering, Vol. 15, Issue 3
  • DOI: 10.1007/BF03024948

Coupling and Ergodicity of Adaptive Markov Chain Monte Carlo Algorithms
journal, June 2007

  • Roberts, Gareth O.; Rosenthal, Jeffrey S.
  • Journal of Applied Probability, Vol. 44, Issue 2
  • DOI: 10.1239/jap/1183667414

Sequential Monte Carlo Methods for Dynamic Systems
journal, September 1998

  • Liu, Jun S.; Chen, Rong
  • Journal of the American Statistical Association, Vol. 93, Issue 443
  • DOI: 10.2307/2669847

Coupling and Ergodicity of Adaptive Markov Chain Monte Carlo Algorithms
journal, June 2007


Missing point estimation in models described by proper orthogonal decomposition
conference, January 2004

  • Astrid, P.; Weiland, S.; Willcox, K.
  • 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601)
  • DOI: 10.1109/cdc.2004.1430301

Stochastic spectral methods for efficient Bayesian solution of inverse problems
conference, January 2005

  • Marzouk, Youssef M.
  • BAYESIAN INFERENCE AND MAXIMUM ENTROPY METHODS IN SCIENCE AND ENGINEERING: 25th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering, AIP Conference Proceedings
  • DOI: 10.1063/1.2149785

Efficient Model Reduction in Non-linear Dynamics Using the Karhunen-Loève Expansion and Dual-Weighted-Residual Methods
text, January 2003


Works referencing / citing this record:

An adaptive and efficient greedy procedure for the optimal training of parametric reduced-order models: ADAPTIVE GREEDY PROCEDURE FOR THE OPTIMAL TRAINING OF PARAMETRIC ROMS
journal, September 2014

  • Paul-Dubois-Taine, A.; Amsallem, D.
  • International Journal for Numerical Methods in Engineering, Vol. 102, Issue 5
  • DOI: 10.1002/nme.4759

A posteriori stochastic correction of reduced models in delayed-acceptance MCMC, with application to multiphase subsurface inverse problems: Stochastic correction of reduced models in delayed-acceptance MCMC
journal, March 2019

  • Cui, Tiangang; Fox, Colin; O'Sullivan, Michael J.
  • International Journal for Numerical Methods in Engineering, Vol. 118, Issue 10
  • DOI: 10.1002/nme.6028

Adaptive multi‐index collocation for uncertainty quantification and sensitivity analysis
journal, November 2019

  • Jakeman, John D.; Eldred, Michael S.; Geraci, Gianluca
  • International Journal for Numerical Methods in Engineering, Vol. 121, Issue 6
  • DOI: 10.1002/nme.6268

The Use of Radial Basis Function Surrogate Models for Sampling Process Acceleration in Bayesian Inversion
book, April 2019

  • Domesová, Simona
  • AETA 2018 - Recent Advances in Electrical Engineering and Related Sciences: Theory and Application
  • DOI: 10.1007/978-3-030-14907-9_23

Multilevel model reduction for uncertainty quantification in computational structural dynamics
journal, October 2016


Fast model updating coupling Bayesian inference and PGD model reduction
journal, April 2018

  • Rubio, Paul-Baptiste; Louf, François; Chamoin, Ludovic
  • Computational Mechanics, Vol. 62, Issue 6
  • DOI: 10.1007/s00466-018-1575-8

A transport-based multifidelity preconditioner for Markov chain Monte Carlo
journal, November 2019


Surrogate accelerated Bayesian inversion for the determination of the thermal diffusivity of a material
journal, January 2019


Statistical Inference Method for Liner Impedance Eduction with a Shear Grazing Flow
journal, March 2019

  • Roncen, R.; Méry, F.; Piot, E.
  • AIAA Journal, Vol. 57, Issue 3
  • DOI: 10.2514/1.j057559

Adaptivity in Bayesian Inverse Finite Element Problems: Learning and Simultaneous Control of Discretisation and Sampling Errors
journal, February 2019

  • Kerfriden, Pierre; Kundu, Abhishek; Claus, Susanne
  • Materials, Vol. 12, Issue 4
  • DOI: 10.3390/ma12040642

Joint state-parameter estimation of a nonlinear stochastic energy balance model from sparse noisy data
journal, January 2019

  • Lu, Fei; Weitzel, Nils; Monahan, Adam H.
  • Nonlinear Processes in Geophysics, Vol. 26, Issue 3
  • DOI: 10.5194/npg-26-227-2019

Adaptive multi‐index collocation for uncertainty quantification and sensitivity analysis
journal, August 2020

  • Jakeman, John D.; Eldred, Michael S.; Geraci, Gianluca
  • International Journal for Numerical Methods in Engineering, Vol. 121, Issue 19
  • DOI: 10.1002/nme.6450

Bayesian Estimation for Stochastic Gene Expression Using Multifidelity Models
journal, February 2019

  • Vo, Huy D.; Fox, Zachary; Baetica, Ania
  • The Journal of Physical Chemistry B, Vol. 123, Issue 10
  • DOI: 10.1021/acs.jpcb.8b10946

A transport-based multifidelity preconditioner for Markov chain Monte Carlo
preprint, January 2018