Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

Bayesian learning of orthogonal embeddings for multi-fidelity Gaussian Processes

Journal Article · · Computer Methods in Applied Mechanics and Engineering
 [1];  [2];  [2];  [2];  [2];  [2]
  1. General Electric Research, Niskayuna, NY (United States); OSTI
  2. General Electric Research, Niskayuna, NY (United States)
+ Show Author Affiliations
Uncertainty propagation in complex engineering systems often poses significant computational challenges related to modeling and quantifying probability distributions of model outputs, as those emerge as the result of various sources of uncertainty that are inherent in the system under investigation. Gaussian Processes regression (GPs) is a robust meta-modeling technique that allows for fast model prediction and exploration of response surfaces. Multi-fidelity variations of GPs further leverage information from cheap and low fidelity model simulations in order to improve their predictive performance on the high fidelity model. In order to cope with the high volume of data required to train GPs in high dimensional design spaces, a common practice is to introduce latent design variables that are typically projections of the original input space to a lower dimensional subspace, and therefore substitute the problem of learning the initial high dimensional mapping, with that of training a GP on a low dimensional space. Here in this paper, we present a Bayesian approach to identify optimal transformations that map the input points to low dimensional latent variables. The \projection" mapping consists of an orthonormal matrix that is considered a priori unknown and needs to be inferred jointly with the GP parameters, conditioned on the available training data. The proposed Bayesian inference scheme relies on a two-step iterative algorithm that samples from the marginal posteriors of the GP parameters and the projection matrix respectively, both using Markov Chain Monte Carlo (MCMC) sampling. In order to take into account the orthogonality constraints imposed on the orthonormal projection matrix, a Geodesic Monte Carlo sampling algorithm is employed, that is suitable for exploiting probability measures on manifolds. We extend the proposed framework to multi-fidelity models using GPs including the scenarios of training multiple outputs together. We validate our framework on three synthetic problems with a known lower-dimensional subspace. The benefits of our proposed framework, are illustrated on the computationally challenging aerodynamic optimization of a last-stage blade for an industrial gas turbine, where we study the effect of an 85-dimensional shape parameterization of a three-dimensional airfoil on two output quantities of interest, specifically on the aerodynamic efficiency and the degree of reaction
Research Organization:
General Electric Co., Boston, MA (United States); General Electric Research, Niskayuna, NY (United States)
Sponsoring Organization:
USDOE Advanced Research Projects Agency - Energy (ARPA-E)
Grant/Contract Number:
AR0001204
OSTI ID:
1976970
Alternate ID(s):
OSTI ID: 1821027
Journal Information:
Computer Methods in Applied Mechanics and Engineering, Journal Name: Computer Methods in Applied Mechanics and Engineering Journal Issue: C Vol. 386; ISSN 0045-7825
Publisher:
ElsevierCopyright Statement
Country of Publication:
United States
Language:
English

References (42)

Uncertainty propagation in CFD using polynomial chaos decomposition journal September 2006
Quasi-Monte Carlo Integration journal December 1995
Support-vector networks journal September 1995
Bayes–Hermite quadrature journal November 1991
Surrogate-based sequential Bayesian experimental design using non-stationary Gaussian Processes journal November 2021
Kernel principal component analysis for stochastic input model generation journal August 2011
Multi-output local Gaussian process regression: Applications to uncertainty quantification journal July 2012
Multi-output separable Gaussian process: Towards an efficient, fully Bayesian paradigm for uncertainty quantification journal May 2013
Uncertainty propagation using infinite mixture of Gaussian processes and variational Bayesian inference journal March 2015
Gaussian processes with built-in dimensionality reduction: Applications to high-dimensional uncertainty propagation journal September 2016
Reduced Wiener Chaos representation of random fields via basis adaptation and projection journal July 2017
Machine learning of linear differential equations using Gaussian processes journal November 2017
Discovering variable fractional orders of advection–dispersion equations from field data using multi-fidelity Bayesian optimization journal November 2017
Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification journal December 2018
Compressive sensing adaptation for polynomial chaos expansions journal March 2019
Structured Bayesian Gaussian process latent variable model: Applications to data-driven dimensionality reduction and high-dimensional inversion journal April 2019
Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data journal October 2019
Sparse Polynomial Chaos expansions using variational relevance vector machines journal September 2020
LIII. On lines and planes of closest fit to systems of points in space journal November 1901
Predicting the output from a complex computer code when fast approximations are available journal March 2000
Model inversion via multi-fidelity Bayesian optimization: a new paradigm for parameter estimation in haemodynamics, and beyond journal May 2016
Multi-fidelity optimization via surrogate modelling
  • Forrester, Alexander I. J.; Sóbester, András; Keane, Andy J.
  • Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 463, Issue 2088 https://doi.org/10.1098/rspa.2007.1900
journal September 2007
Multi-fidelity modelling via recursive co-kriging and Gaussian–Markov random fields journal July 2015
Nonlinear information fusion algorithms for data-efficient multi-fidelity modelling journal February 2017
Bayesian adaptation of chaos representations using variational inference and sampling on geodesics journal September 2018
Markov Chain Monte Carlo Inference of Parametric Dictionaries for Sparse Bayesian Approximations journal June 2016
Bayesian calibration of computer models journal August 2001
Riemann manifold Langevin and Hamiltonian Monte Carlo methods: Riemann Manifold Langevin and Hamiltonian Monte Carlo Methods journal March 2011
Geodesic Monte Carlo on Embedded Manifolds: Geodesic Monte Carlo journal September 2013
Review of Metamodeling Techniques in Support of Engineering Design Optimization journal May 2006
Gradient-Informed Basis Adaptation for Legendre Chaos Expansions journal March 2018
Scalable Fully Bayesian Gaussian Process Modeling and Calibration With Adaptive Sequential Monte Carlo for Industrial Applications journal March 2021
A Limited Memory Algorithm for Bound Constrained Optimization journal September 1995
Multidimensional Adaptive Relevance Vector Machines for Uncertainty Quantification journal January 2012
Bayesian Analysis of Hierarchical Multifidelity Codes journal January 2013
Active Subspace Methods in Theory and Practice: Applications to Kriging Surfaces journal January 2014
Multifidelity Information Fusion Algorithms for High-Dimensional Systems and Massive Data sets journal January 2016
Numerical Gaussian Processes for Time-Dependent and Nonlinear Partial Differential Equations journal January 2018
Multifidelity Dimension Reduction via Active Subspaces journal January 2020
The Wiener--Askey Polynomial Chaos for Stochastic Differential Equations journal January 2002
A History of the Metropolis–Hastings Algorithm journal November 2003
Statistical Approximations for Multidisciplinary Design Optimization: The Problem of Size journal January 1999

Similar Records

Gaussian processes meet NeuralODEs: a Bayesian framework for learning the dynamics of partially observed systems from scarce and noisy data
Journal Article · Mon Jun 20 00:00:00 EDT 2022 · Philosophical Transactions of the Royal Society. A, Mathematical, Physical and Engineering Sciences · OSTI ID:1979597
Scalable posterior approximations for large-scale Bayesian inverse problems via likelihood-informed parameter and state reduction
Journal Article · Mon Mar 28 20:00:00 EDT 2016 · Journal of Computational Physics · OSTI ID:1548326
Accelerating multilevel Markov Chain Monte Carlo using machine learning models
Journal Article · Sun Apr 06 20:00:00 EDT 2025 · Physica Scripta · OSTI ID:2566341

Related Subjects

42 ENGINEERING
97 MATHEMATICS AND COMPUTING
Bayesian inference
Gaussian process regression
dimension reduction
geodesic Monte Carlo
multi-fidelity simulations
uncertainty propagation