Bayesian estimation of Karhunen–Loève expansions; A random subspace approach
Abstract
One of the most widely-used statistical procedures for dimensionality reduction of high dimensional random fields is Principal Component Analysis (PCA), which is based on the Karhunen-Lo eve expansion (KLE) of a stochastic process with finite variance. The KLE is analogous to a Fourier series expansion for a random process, where the goal is to find an orthogonal transformation for the data such that the projection of the data onto this orthogonal subspace is optimal in the L2 sense, i.e, which minimizes the mean square error. In practice, this orthogonal transformation is determined by performing an SVD (Singular Value Decomposition) on the sample covariance matrix or on the data matrix itself. Sampling error is typically ignored when quantifying the principal components, or, equivalently, basis functions of the KLE. Furthermore, it is exacerbated when the sample size is much smaller than the dimension of the random field. In this paper, we introduce a Bayesian KLE procedure, allowing one to obtain a probabilistic model on the principal components, which can account for inaccuracies due to limited sample size. The probabilistic model is built via Bayesian inference, from which the posterior becomes the matrix Bingham density over the space of orthonormal matrices. We usemore »
- Authors:
-
- Sandia National Lab. (SNL-CA), Livermore, CA (United States)
- Publication Date:
- Research Org.:
- Sandia National Lab. (SNL-CA), Livermore, CA (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
- OSTI Identifier:
- 1427230
- Alternate Identifier(s):
- OSTI ID: 1324851
- Report Number(s):
- SAND-2015-1440J
Journal ID: ISSN 0021-9991; 567285; TRN: US1802979
- Grant/Contract Number:
- AC04-94AL85000; AC04-94-AL85000
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Journal of Computational Physics
- Additional Journal Information:
- Journal Volume: 319; Journal Issue: C; Journal ID: ISSN 0021-9991
- Publisher:
- Elsevier
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 97 MATHEMATICS AND COMPUTING
Citation Formats
Chowdhary, Kenny, and Najm, Habib N. Bayesian estimation of Karhunen–Loève expansions; A random subspace approach. United States: N. p., 2016.
Web. doi:10.1016/j.jcp.2016.02.056.
Chowdhary, Kenny, & Najm, Habib N. Bayesian estimation of Karhunen–Loève expansions; A random subspace approach. United States. https://doi.org/10.1016/j.jcp.2016.02.056
Chowdhary, Kenny, and Najm, Habib N. Wed .
"Bayesian estimation of Karhunen–Loève expansions; A random subspace approach". United States. https://doi.org/10.1016/j.jcp.2016.02.056. https://www.osti.gov/servlets/purl/1427230.
@article{osti_1427230,
title = {Bayesian estimation of Karhunen–Loève expansions; A random subspace approach},
author = {Chowdhary, Kenny and Najm, Habib N.},
abstractNote = {One of the most widely-used statistical procedures for dimensionality reduction of high dimensional random fields is Principal Component Analysis (PCA), which is based on the Karhunen-Lo eve expansion (KLE) of a stochastic process with finite variance. The KLE is analogous to a Fourier series expansion for a random process, where the goal is to find an orthogonal transformation for the data such that the projection of the data onto this orthogonal subspace is optimal in the L2 sense, i.e, which minimizes the mean square error. In practice, this orthogonal transformation is determined by performing an SVD (Singular Value Decomposition) on the sample covariance matrix or on the data matrix itself. Sampling error is typically ignored when quantifying the principal components, or, equivalently, basis functions of the KLE. Furthermore, it is exacerbated when the sample size is much smaller than the dimension of the random field. In this paper, we introduce a Bayesian KLE procedure, allowing one to obtain a probabilistic model on the principal components, which can account for inaccuracies due to limited sample size. The probabilistic model is built via Bayesian inference, from which the posterior becomes the matrix Bingham density over the space of orthonormal matrices. We use a modified Gibbs sampling procedure to sample on this space and then build a probabilistic Karhunen-Lo eve expansions over random subspaces to obtain a set of low-dimensional surrogates of the stochastic process. We illustrate this probabilistic procedure with a finite dimensional stochastic process inspired by Brownian motion.},
doi = {10.1016/j.jcp.2016.02.056},
journal = {Journal of Computational Physics},
number = C,
volume = 319,
place = {United States},
year = {Wed Apr 13 00:00:00 EDT 2016},
month = {Wed Apr 13 00:00:00 EDT 2016}
}
Web of Science
Works referenced in this record:
A Visual Explanation of Jensen's Inequality
journal, October 1993
- Needham, Tristan
- The American Mathematical Monthly, Vol. 100, Issue 8
Estimation of covariance matrices based on hierarchical inverse-Wishart priors
journal, April 2013
- Bouriga, M.; Féron, O.
- Journal of Statistical Planning and Inference, Vol. 143, Issue 4
Simulation of the Matrix Bingham–von Mises–Fisher Distribution, With Applications to Multivariate and Relational Data
journal, January 2009
- Hoff, Peter D.
- Journal of Computational and Graphical Statistics, Vol. 18, Issue 2
Probabilistic Principal Component Analysis
journal, August 1999
- Tipping, Michael E.; Bishop, Christopher M.
- Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 61, Issue 3, p. 611-622
A tutorial on approximate Bayesian computation
journal, April 2012
- Turner, Brandon M.; Van Zandt, Trisha
- Journal of Mathematical Psychology, Vol. 56, Issue 2
Sampling from compositional and directional distributions
journal, September 2006
- Kume, Alfred; Walker, Stephen G.
- Statistics and Computing, Vol. 16, Issue 3
On the numerical expansion of a second order stochastic process
journal, June 1992
- Gutiérrez, Ramón; Ruiz, Juan Carlos; Valderrama, Mariano J.
- Applied Stochastic Models and Data Analysis, Vol. 8, Issue 2
On the Karhunen-Loeve expansion for transformed processes
journal, September 1987
- Jaimez, Ramón Gutiérrez; Bonnet, Mariano J. Valderrama
- Trabajos de Estadistica, Vol. 2, Issue 2
Spectral Representation and Reduced Order Modeling of the Dynamics of Stochastic Reaction Networks via Adaptive Data Partitioning
journal, January 2010
- Sargsyan, Khachik; Debusschere, Bert; Najm, Habib
- SIAM Journal on Scientific Computing, Vol. 31, Issue 6
Works referencing / citing this record:
Bayesian adaptation of chaos representations using variational inference and sampling on geodesics
journal, September 2018
- Tsilifis, P.; Ghanem, R. G.
- Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 474, Issue 2217
Recent Trends in the Modeling and Quantification of Non-probabilistic Uncertainty
journal, February 2019
- Faes, Matthias; Moens, David
- Archives of Computational Methods in Engineering, Vol. 27, Issue 3
Compressive sensing adaptation for polynomial chaos expansions
journal, March 2019
- Tsilifis, Panagiotis; Huan, Xun; Safta, Cosmin
- Journal of Computational Physics, Vol. 380