High-Dimensional Intrinsic Interpolation Using Gaussian Process Regression and Diffusion Maps

Thimmisetty, Charanraj A.; Ghanem, Roger G.; White, Joshua A.; Chen, Xiao

doi:10.1007/s11004-017-9705-y

Title: High-Dimensional Intrinsic Interpolation Using Gaussian Process Regression and Diffusion Maps

Abstract

This article considers the challenging task of estimating geologic properties of interest using a suite of proxy measurements. The current work recast this task as a manifold learning problem. In this process, this article introduces a novel regression procedure for intrinsic variables constrained onto a manifold embedded in an ambient space. The procedure is meant to sharpen high-dimensional interpolation by inferring non-linear correlations from the data being interpolated. The proposed approach augments manifold learning procedures with a Gaussian process regression. It first identifies, using diffusion maps, a low-dimensional manifold embedded in an ambient high-dimensional space associated with the data. It relies on the diffusion distance associated with this construction to define a distance function with which the data model is equipped. This distance metric function is then used to compute the correlation structure of a Gaussian process that describes the statistical dependence of quantities of interest in the high-dimensional ambient space. The proposed method is applicable to arbitrarily high-dimensional data sets. Here, it is applied to subsurface characterization using a suite of well log measurements. The predictions obtained in original, principal component, and diffusion space are compared using both qualitative and quantitative metrics. Considerable improvement in the prediction of themore »« less

Authors:

Thimmisetty, Charanraj A. ^[1]; Ghanem, Roger G. ^[2]; White, Joshua A. ^[3]; Chen, Xiao ^[1]

Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States). Center for Applied Scientific Computing
Univ. of Southern California, Los Angeles, CA (United States). Sonny Astani Dept. of Civil and Environmental Engineering
Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States). Atmospheric, Earth and Energy Division

Publication Date:: 2017-10-10

Research Org.:: Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)

Sponsoring Org.:: USDOE Laboratory Directed Research and Development (LDRD) Program

OSTI Identifier:: 1400095

Report Number(s):: LLNL-JRNL-728760
Journal ID: ISSN 1874-8961; TRN: US1703098

Grant/Contract Number:: AC52-07NA27344

Resource Type:: Accepted Manuscript

Journal Name:: Mathematical Geosciences

Additional Journal Information:: Journal Volume: 50; Journal Issue: 1; Journal ID: ISSN 1874-8961

Publisher:: Springer

Country of Publication:: United States

Language:: English

Subject:: 58 GEOSCIENCES; 97 MATHEMATICS AND COMPUTING; Gaussian process regression; Kriging; Interpolation on manifold; Intrinsic interpolation; Intrinsic metrics; Diffusion distance

Citation Formats


                    Thimmisetty, Charanraj A., Ghanem, Roger G., White, Joshua A., and Chen, Xiao. High-Dimensional Intrinsic Interpolation Using Gaussian Process Regression and Diffusion Maps.  United States: N. p., 2017. 
Web.  doi:10.1007/s11004-017-9705-y.

Copy to clipboard


                    Thimmisetty, Charanraj A., Ghanem, Roger G., White, Joshua A., & Chen, Xiao. High-Dimensional Intrinsic Interpolation Using Gaussian Process Regression and Diffusion Maps.  United States.  https://doi.org/10.1007/s11004-017-9705-y

Copy to clipboard


                    Thimmisetty, Charanraj A., Ghanem, Roger G., White, Joshua A., and Chen, Xiao. Tue .  
"High-Dimensional Intrinsic Interpolation Using Gaussian Process Regression and Diffusion Maps".  United States.  https://doi.org/10.1007/s11004-017-9705-y.  https://www.osti.gov/servlets/purl/1400095.

Copy to clipboard


                    
@article{osti_1400095,

  title        = {High-Dimensional Intrinsic Interpolation Using Gaussian Process Regression and Diffusion Maps},

  author       = {Thimmisetty, Charanraj A. and Ghanem, Roger G. and White, Joshua A. and Chen, Xiao},

  abstractNote = {This article considers the challenging task of estimating geologic properties of interest using a suite of proxy measurements. The current work recast this task as a manifold learning problem. In this process, this article introduces a novel regression procedure for intrinsic variables constrained onto a manifold embedded in an ambient space. The procedure is meant to sharpen high-dimensional interpolation by inferring non-linear correlations from the data being interpolated. The proposed approach augments manifold learning procedures with a Gaussian process regression. It first identifies, using diffusion maps, a low-dimensional manifold embedded in an ambient high-dimensional space associated with the data. It relies on the diffusion distance associated with this construction to define a distance function with which the data model is equipped. This distance metric function is then used to compute the correlation structure of a Gaussian process that describes the statistical dependence of quantities of interest in the high-dimensional ambient space. The proposed method is applicable to arbitrarily high-dimensional data sets. Here, it is applied to subsurface characterization using a suite of well log measurements. The predictions obtained in original, principal component, and diffusion space are compared using both qualitative and quantitative metrics. Considerable improvement in the prediction of the geological structural properties is observed with the proposed method.},

  doi          = {10.1007/s11004-017-9705-y},

  journal      = {Mathematical Geosciences},

  number       = 1,

  volume       = 50,

  place        = {United States},

  year         = {2017},

  month        = {10}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1007/s11004-017-9705-y

Other availability

Search WorldCat to find libraries that may hold this journal

Save / Share:

Export Metadata

Save to My Library

Works referenced in this record:

The origins of kriging
journal, April 1990

Cressie, Noel
Mathematical Geology, Vol. 22, Issue 3
DOI: 10.1007/BF00889887

Practical application of fuzzy logic and neural networks to fractured reservoir characterization
journal, October 2000

Ouenes, Ahmed
Computers & Geosciences, Vol. 26, Issue 8
DOI: 10.1016/S0098-3004(00)00031-5

Principles of geostatistics
journal, December 1963

Matheron, Georges
Economic Geology, Vol. 58, Issue 8
DOI: 10.2113/gsecongeo.58.8.1246

Gaussian Processes for Machine Learning
book, January 2005

Rasmussen, Carl Edward; Williams, Christopher K. I.
The MIT Press
DOI: 10.7551/mitpress/3206.001.0001

Detecting intrinsic slow variables in stochastic dynamical systems by anisotropic diffusion maps
journal, August 2009

Singer, A.; Erban, R.; Kevrekidis, I. G.
Proceedings of the National Academy of Sciences, Vol. 106, Issue 38
DOI: 10.1073/pnas.0905547106

Data-driven probability concentration and sampling on manifold
journal, September 2016

Soize, C.; Ghanem, R.
Journal of Computational Physics, Vol. 321
DOI: 10.1016/j.jcp.2016.05.044

Automated manifold surgery: constructing geometrically accurate and topologically correct models of the human cerebral cortex
journal, January 2001

Fischl, B.; Liu, A.; Dale, A. M.
IEEE Transactions on Medical Imaging, Vol. 20, Issue 1
DOI: 10.1109/42.906426

Diffusion maps and coarse-graining: a unified framework for dimensionality reduction, graph partitioning, and data set parameterization
journal, September 2006

Lafon, S.; Lee, A. B.
IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 28, Issue 9, p. 1393-1403
DOI: 10.1109/TPAMI.2006.184

COGNITION: The Manifold Ways of Perception
journal, December 2000

Seung, H. S.
Science, Vol. 290, Issue 5500
DOI: 10.1126/science.290.5500.2268

Statistical shape analysis: clustering, learning, and testing
journal, April 2005

Srivastava, A.; Joshi, S. H.; Mio, W.
IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 27, Issue 4
DOI: 10.1109/TPAMI.2005.86

Diffusion maps
journal, July 2006

Coifman, Ronald R.; Lafon, Stéphane
Applied and Computational Harmonic Analysis, Vol. 21, Issue 1
DOI: 10.1016/j.acha.2006.04.006

Geometric diffusions as a tool for harmonic analysis and structure definition of data: Diffusion maps
journal, May 2005

Coifman, R. R.; Lafon, S.; Lee, A. B.
Proceedings of the National Academy of Sciences, Vol. 102, Issue 21
DOI: 10.1073/pnas.0500334102

Bayesian Prediction of Transformed Gaussian Random Fields
journal, December 1997

De Oliveira, Victor; Kedem, Benjamin; Short, David A.
Journal of the American Statistical Association, Vol. 92, Issue 440
DOI: 10.1080/01621459.1997.10473663

Prediction of Hydrocarbon Reservoirs Permeability Using Support Vector Machine
journal, January 2012

Gholami, R.; Shahraki, A. R.; Jamali Paghaleh, M.
Mathematical Problems in Engineering, Vol. 2012
DOI: 10.1155/2012/670723

Screening, Predicting, and Computer Experiments
journal, February 1992

Welch, William J.; Buck, Robert. J.; Sacks, Jerome
Technometrics, Vol. 34, Issue 1
DOI: 10.2307/1269548

Past, present and future intelligent reservoir characterization trends
journal, November 2001

Nikravesh, Masoud; Aminzadeh, F.
Journal of Petroleum Science and Engineering, Vol. 31, Issue 2-4
DOI: 10.1016/S0920-4105(01)00121-8

Works referencing / citing this record:

Multiscale Stochastic Representations Using Polynomial Chaos Expansions with Gaussian Process Coefficients
journal, January 2018

Thimmisetty, Charanraj; Aminzadeh, Fred; Rose, Kelly
Data-Enabled Discovery and Applications, Vol. 2, Issue 1
DOI: 10.1007/s41688-018-0015-4

Similar Records in DOE PAGES and OSTI.GOV collections:

Time-series forecasting using manifold learning, radial basis function interpolation, and geometric harmonics

Journal Article Papaioannou, Panagiotis G. ; Talmon, Ronen ; Kevrekidis, Ioannis G. ; ... - Chaos: An Interdisciplinary Journal of Nonlinear Science

We address a three-tier numerical framework based on nonlinear manifold learning for the forecasting of high-dimensional time series, relaxing the “curse of dimensionality” related to the training phase of surrogate/machine learning models. At the first step, we embed the high-dimensional time series into a reduced low-dimensional space using nonlinear manifold learning (local linear embedding and parsimonious diffusion maps). Then, we construct reduced-order surrogate models on the manifold (here, for our illustrations, we used multivariate autoregressive and Gaussian process regression models) to forecast the embedded dynamics. Finally, we solve the pre-image problem, thus lifting the embedded time series back to themore »« less
https://doi.org/10.1063/5.0094887
Data-driven surrogates for high dimensional models using Gaussian process regression on the Grassmann manifold

Journal Article Giovanis, D. G. ; Shields, M. D. - Computer Methods in Applied Mechanics and Engineering

This paper introduces a surrogate modeling scheme based on Grassmannian manifold learning to be used for cost-efficient predictions of high-dimensional stochastic systems. The method exploits subspace-structured features of each solution by projecting it onto a Grassmann manifold. This point-wise linear dimensionality reduction harnesses the structural information to assess the similarity between solutions at different points in the input parameter space. The method utilizes a solution clustering approach in order to identify regions of the parameter space over which solutions are sufficiently similarly such that they can be interpolated on the Grassmannian. In this clustering, the reduced-order solutions are partitioned intomore »« less
https://doi.org/10.1016/j.cma.2020.113269

Full Text Available
Latent-space time evolution of non-intrusive reduced-order models using Gaussian process emulation

Journal Article Maulik, Romit ; Botsas, Themistoklis ; Ramachandra, Nesar ; ... - Physica. D, Nonlinear Phenomena

Non-intrusive reduced-order models (ROMs) have recently generated considerable interest for constructing computationally efficient counterparts of nonlinear dynamical systems emerging from various domain sciences. They provide a low-dimensional emulation framework for systems that may be intrinsically high-dimensional. This is accomplished by utilizing a construction algorithm that is purely data-driven. It is no surprise, therefore, that the algorithmic advances of machine learning have led to non-intrusive ROMs with greater accuracy and computational gains. However, in bypassing the utilization of an equation-based evolution, it is often seen that the interpretability of the ROM framework suffers. This becomes more problematic when black-box deep learningmore »« less
https://doi.org/10.1016/j.physd.2020.132797

Full Text Available
Graph interpolating activation improves both natural and robust accuracies in data-efficient deep learning

Journal Article WANG, BAO ; OSHER, STAN J. - European Journal of Applied Mathematics

Improving the accuracy and robustness of deep neural nets (DNNs) and adapting them to small training data are primary tasks in deep learning (DL) research. In this paper, we replace the output activation function of DNNs, typically the data-agnostic softmax function, with a graph Laplacian-based high-dimensional interpolating function which, in the continuum limit, converges to the solution of a Laplace–Beltrami equation on a high-dimensional manifold. Furthermore, we propose end-to-end training and testing algorithms for this new architecture. The proposed DNN with graph interpolating activation integrates the advantages of both deep learning and manifold learning. Compared to the conventional DNNs withmore »« less
https://doi.org/10.1017/s0956792520000406
Gaussian processes with built-in dimensionality reduction: Applications to high-dimensional uncertainty propagation

Journal Article Tripathy, Rohit ; Bilionis, Ilias ; Gonzalez, Marcial - Journal of Computational Physics

Uncertainty quantification (UQ) tasks, such as model calibration, uncertainty propagation, and optimization under uncertainty, typically require several thousand evaluations of the underlying computer codes. To cope with the cost of simulations, one replaces the real response surface with a cheap surrogate based, e.g., on polynomial chaos expansions, neural networks, support vector machines, or Gaussian processes (GP). However, the number of simulations required to learn a generic multivariate response grows exponentially as the input dimension increases. This curse of dimensionality can only be addressed, if the response exhibits some special structure that can be discovered and exploited. A wide range ofmore »« less
https://doi.org/10.1016/J.JCP.2016.05.039

Similar Records

Title: High-Dimensional Intrinsic Interpolation Using Gaussian Process Regression and Diffusion Maps

Abstract

Citation Formats

The origins of kriging journal, April 1990

Practical application of fuzzy logic and neural networks to fractured reservoir characterization journal, October 2000

Principles of geostatistics journal, December 1963

Gaussian Processes for Machine Learning book, January 2005

Detecting intrinsic slow variables in stochastic dynamical systems by anisotropic diffusion maps journal, August 2009

Data-driven probability concentration and sampling on manifold journal, September 2016

Automated manifold surgery: constructing geometrically accurate and topologically correct models of the human cerebral cortex journal, January 2001

Diffusion maps and coarse-graining: a unified framework for dimensionality reduction, graph partitioning, and data set parameterization journal, September 2006

COGNITION: The Manifold Ways of Perception journal, December 2000

Statistical shape analysis: clustering, learning, and testing journal, April 2005

Diffusion maps journal, July 2006

Geometric diffusions as a tool for harmonic analysis and structure definition of data: Diffusion maps journal, May 2005

Bayesian Prediction of Transformed Gaussian Random Fields journal, December 1997

Prediction of Hydrocarbon Reservoirs Permeability Using Support Vector Machine journal, January 2012

Screening, Predicting, and Computer Experiments journal, February 1992

Past, present and future intelligent reservoir characterization trends journal, November 2001

Multiscale Stochastic Representations Using Polynomial Chaos Expansions with Gaussian Process Coefficients journal, January 2018

The origins of kriging
journal, April 1990

Practical application of fuzzy logic and neural networks to fractured reservoir characterization
journal, October 2000

Principles of geostatistics
journal, December 1963

Gaussian Processes for Machine Learning
book, January 2005

Detecting intrinsic slow variables in stochastic dynamical systems by anisotropic diffusion maps
journal, August 2009

Data-driven probability concentration and sampling on manifold
journal, September 2016

Automated manifold surgery: constructing geometrically accurate and topologically correct models of the human cerebral cortex
journal, January 2001

Diffusion maps and coarse-graining: a unified framework for dimensionality reduction, graph partitioning, and data set parameterization
journal, September 2006

COGNITION: The Manifold Ways of Perception
journal, December 2000

Statistical shape analysis: clustering, learning, and testing
journal, April 2005

Diffusion maps
journal, July 2006

Geometric diffusions as a tool for harmonic analysis and structure definition of data: Diffusion maps
journal, May 2005

Bayesian Prediction of Transformed Gaussian Random Fields
journal, December 1997

Prediction of Hydrocarbon Reservoirs Permeability Using Support Vector Machine
journal, January 2012

Screening, Predicting, and Computer Experiments
journal, February 1992

Past, present and future intelligent reservoir characterization trends
journal, November 2001

Multiscale Stochastic Representations Using Polynomial Chaos Expansions with Gaussian Process Coefficients
journal, January 2018