Bayesian estimation of Karhunen–Loève expansions; A random subspace approach
Abstract
One of the most widelyused statistical procedures for dimensionality reduction of high dimensional random fields is Principal Component Analysis (PCA), which is based on the KarhunenLo eve expansion (KLE) of a stochastic process with finite variance. The KLE is analogous to a Fourier series expansion for a random process, where the goal is to find an orthogonal transformation for the data such that the projection of the data onto this orthogonal subspace is optimal in the L^{2} sense, i.e, which minimizes the mean square error. In practice, this orthogonal transformation is determined by performing an SVD (Singular Value Decomposition) on the sample covariance matrix or on the data matrix itself. Sampling error is typically ignored when quantifying the principal components, or, equivalently, basis functions of the KLE. Furthermore, it is exacerbated when the sample size is much smaller than the dimension of the random field. In this paper, we introduce a Bayesian KLE procedure, allowing one to obtain a probabilistic model on the principal components, which can account for inaccuracies due to limited sample size. The probabilistic model is built via Bayesian inference, from which the posterior becomes the matrix Bingham density over the space of orthonormal matrices. We usemore »
 Authors:

 Sandia National Lab. (SNLCA), Livermore, CA (United States)
 Publication Date:
 Research Org.:
 Sandia National Lab. (SNLCA), Livermore, CA (United States)
 Sponsoring Org.:
 USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC21)
 OSTI Identifier:
 1427230
 Alternate Identifier(s):
 OSTI ID: 1324851
 Report Number(s):
 SAND20151440J
Journal ID: ISSN 00219991; 567285; TRN: US1802979
 Grant/Contract Number:
 AC0494AL85000; AC0494AL85000
 Resource Type:
 Accepted Manuscript
 Journal Name:
 Journal of Computational Physics
 Additional Journal Information:
 Journal Volume: 319; Journal Issue: C; Journal ID: ISSN 00219991
 Publisher:
 Elsevier
 Country of Publication:
 United States
 Language:
 English
 Subject:
 97 MATHEMATICS AND COMPUTING
Citation Formats
Chowdhary, Kenny, and Najm, Habib N. Bayesian estimation of Karhunen–Loève expansions; A random subspace approach. United States: N. p., 2016.
Web. doi:10.1016/j.jcp.2016.02.056.
Chowdhary, Kenny, & Najm, Habib N. Bayesian estimation of Karhunen–Loève expansions; A random subspace approach. United States. doi:10.1016/j.jcp.2016.02.056.
Chowdhary, Kenny, and Najm, Habib N. Wed .
"Bayesian estimation of Karhunen–Loève expansions; A random subspace approach". United States. doi:10.1016/j.jcp.2016.02.056. https://www.osti.gov/servlets/purl/1427230.
@article{osti_1427230,
title = {Bayesian estimation of Karhunen–Loève expansions; A random subspace approach},
author = {Chowdhary, Kenny and Najm, Habib N.},
abstractNote = {One of the most widelyused statistical procedures for dimensionality reduction of high dimensional random fields is Principal Component Analysis (PCA), which is based on the KarhunenLo eve expansion (KLE) of a stochastic process with finite variance. The KLE is analogous to a Fourier series expansion for a random process, where the goal is to find an orthogonal transformation for the data such that the projection of the data onto this orthogonal subspace is optimal in the L2 sense, i.e, which minimizes the mean square error. In practice, this orthogonal transformation is determined by performing an SVD (Singular Value Decomposition) on the sample covariance matrix or on the data matrix itself. Sampling error is typically ignored when quantifying the principal components, or, equivalently, basis functions of the KLE. Furthermore, it is exacerbated when the sample size is much smaller than the dimension of the random field. In this paper, we introduce a Bayesian KLE procedure, allowing one to obtain a probabilistic model on the principal components, which can account for inaccuracies due to limited sample size. The probabilistic model is built via Bayesian inference, from which the posterior becomes the matrix Bingham density over the space of orthonormal matrices. We use a modified Gibbs sampling procedure to sample on this space and then build a probabilistic KarhunenLo eve expansions over random subspaces to obtain a set of lowdimensional surrogates of the stochastic process. We illustrate this probabilistic procedure with a finite dimensional stochastic process inspired by Brownian motion.},
doi = {10.1016/j.jcp.2016.02.056},
journal = {Journal of Computational Physics},
number = C,
volume = 319,
place = {United States},
year = {2016},
month = {4}
}