KullbackLeibler Approximation for Probability Measures on Infinite Dimensional Spaces
Abstract
In a variety of applications it is important to extract information from a probability measure μ on an infinite dimensional space. Examples include the Bayesian approach to inverse problems and (possibly conditioned) continuous time Markov processes. It may then be of interest to find a measure ν, from within a simple class of measures, which approximates μ. Here, this problem is studied in the case where the Kullback–Leibler divergence is employed to measure the quality of the approximation. A calculus of variations viewpoint is adopted, and the particular case where ν is chosen from the set of Gaussian measures is studied in detail. Basic existence and uniqueness theorems are established, together with properties of minimizing sequences. Furthermore, parameterization of the class of Gaussians through the mean and inverse covariance is introduced, the need for regularization is explained, and a regularized minimization is studied in detail. The calculus of variations framework resulting from this work provides the appropriate underpinning for computational algorithms.
 Authors:

 Univ. of Cincinnati, OH (United States). Dept. of Physics
 Drexel Univ., Philadelphia, PA (United States). Dept. of Mathematics
 Warwick Univ., Coventry (United Kingdom). Mathematics Inst.
 Publication Date:
 Research Org.:
 Warwick Univ., Coventry (United Kingdom)
 Sponsoring Org.:
 USDOE Office of Science (SC); European Research Council (ERC); Engineering and Physical Sciences Research Council (EPSRC); US Department of the Navy, Office of Naval Research (ONR); National Science Foundation (NSF)
 OSTI Identifier:
 1459163
 Grant/Contract Number:
 SC0002085; OISE0967140
 Resource Type:
 Accepted Manuscript
 Journal Name:
 SIAM Journal of Mathematical Analysis
 Additional Journal Information:
 Journal Volume: 47; Journal Issue: 6; Journal ID: ISSN 00361410
 Country of Publication:
 United States
 Language:
 English
 Subject:
 97 MATHEMATICS AND COMPUTING; Kullback–Leibler divergence; relative entropy; Gaussian measures
Citation Formats
Pinski, Frank J., Simpson, G., Stuart, A. M., and Weber, Hendrik. KullbackLeibler Approximation for Probability Measures on Infinite Dimensional Spaces. United States: N. p., 2015.
Web. doi:10.1137/140962802.
Pinski, Frank J., Simpson, G., Stuart, A. M., & Weber, Hendrik. KullbackLeibler Approximation for Probability Measures on Infinite Dimensional Spaces. United States. doi:10.1137/140962802.
Pinski, Frank J., Simpson, G., Stuart, A. M., and Weber, Hendrik. Thu .
"KullbackLeibler Approximation for Probability Measures on Infinite Dimensional Spaces". United States. doi:10.1137/140962802. https://www.osti.gov/servlets/purl/1459163.
@article{osti_1459163,
title = {KullbackLeibler Approximation for Probability Measures on Infinite Dimensional Spaces},
author = {Pinski, Frank J. and Simpson, G. and Stuart, A. M. and Weber, Hendrik},
abstractNote = {In a variety of applications it is important to extract information from a probability measure μ on an infinite dimensional space. Examples include the Bayesian approach to inverse problems and (possibly conditioned) continuous time Markov processes. It may then be of interest to find a measure ν, from within a simple class of measures, which approximates μ. Here, this problem is studied in the case where the Kullback–Leibler divergence is employed to measure the quality of the approximation. A calculus of variations viewpoint is adopted, and the particular case where ν is chosen from the set of Gaussian measures is studied in detail. Basic existence and uniqueness theorems are established, together with properties of minimizing sequences. Furthermore, parameterization of the class of Gaussians through the mean and inverse covariance is introduced, the need for regularization is explained, and a regularized minimization is studied in detail. The calculus of variations framework resulting from this work provides the appropriate underpinning for computational algorithms.},
doi = {10.1137/140962802},
journal = {SIAM Journal of Mathematical Analysis},
number = 6,
volume = 47,
place = {United States},
year = {2015},
month = {11}
}
Web of Science
Works referencing / citing this record:
Computationally Efficient Variational Approximations for Bayesian Inverse Problems
journal, July 2016
 Tsilifis, Panagiotis; Bilionis, Ilias; Katsounaros, Ioannis
 Journal of Verification, Validation and Uncertainty Quantification, Vol. 1, Issue 3
Computationally Efficient Variational Approximations for Bayesian Inverse Problems
journal, July 2016
 Tsilifis, Panagiotis; Bilionis, Ilias; Katsounaros, Ioannis
 Journal of Verification, Validation and Uncertainty Quantification, Vol. 1, Issue 3
A Generalized Relative (α, β)Entropy: Geometric Properties and Applications to Robust Statistical Inference
journal, May 2018
 Ghosh, Abhik; Basu, Ayanendranath
 Entropy, Vol. 20, Issue 5