Title: A localized ensemble of approximate Gaussian processes for fast sequential emulation

Journal Article · · Stat
DOI: https://doi.org/10.1002/sta4.576 · OSTI ID:1972761

More attention has been given to the computational cost associated with the fitting of an emulator. Substantially less attention is given to the computational cost of using that emulator for prediction. This is primarily because the cost of fitting an emulator is usually far greater than that of obtaining a single prediction, and predictions can often be obtained in parallel. In many settings, especially those requiring Markov Chain Monte Carlo, predictions may arrive sequentially and parallelization is not possible. In this case, using an emulator procedure which can produce accurate predictions efficiently can lead to substantial time savings in practice. In this paper, we propose a global model approximate Gaussian process framework via extension of  a popular local approximate Gaussian process (laGP) framework. Our proposed emulator can be viewed as a treed Gaussian process where the leaf nodes are laGP models, and the tree structure is learned greedily as a function of the prediction stream. The suggested method (called leapGP) has interpretable tuning parameters which control the time‐memory trade‐off. One reasonable choice of settings leads to an emulator with a training cost and makes predictions rapidly with an asymptotic amortized cost of .

Research Organization:
Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
Sponsoring Organization:
USDOE; USDOE Laboratory Directed Research and Development (LDRD) Program; USDOE National Nuclear Security Administration (NNSA)
Grant/Contract Number:
89233218CNA000001
OSTI ID:
1972761
Report Number(s):
LA-UR-22-30978; e576
Journal Information:
Stat, Journal Name: Stat Journal Issue: 1 Vol. 12; ISSN 2049-1573
Publisher:
Wiley Blackwell (John Wiley & Sons)Copyright Statement
Country of Publication:
United Kingdom
Language:
English

References (35)

Precision aggregated local models journal October 2021
Faster k-Medoids Clustering: Improving the PAM, CLARA, and CLARANS Algorithms book January 2019
Gaussian Processes in Machine Learning book January 2004
Bkd-Tree: A Dynamic Scalable kd-Tree book January 2003
Hilbert space methods for reduced-rank Gaussian process regression journal August 2019
Bayesian projection pursuit regression journal November 2023
A random forest guided tour journal April 2016
Vecchia Approximations of Gaussian-Process Predictions journal June 2020
Optimal Latin-hypercube designs for computer experiments journal April 1994
Genetic Algorithms journal July 1992
Approximating likelihoods for large spatial data sets journal May 2004
Fast matrix algebra for Bayesian model calibration journal December 2020
Projection Pursuit Regression journal December 1981
Modeling Data from Computer Experiments: An Empirical Comparison of Kriging with MARS and Projection Pursuit Regression journal October 2007
Local Gaussian Process Approximation for Large Computer Experiments journal April 2015
Vecchia-Approximated Deep Gaussian Processes for Computer Experiments journal November 2022
Local Gaussian process regression for real-time model-based robot control conference September 2008
When Gaussian Process Meets Big Data: A Review of Scalable GPs journal November 2020
Bayesian calibration of computer models journal August 2001
Estimation and Model Identification for Continuous Spatial Processes journal January 1988
A hierarchical sparse Gaussian process for in situ inference in expensive physics simulations conference October 2022
Combining Field Data and Computer Simulations for Calibration and Prediction journal January 2004
Sequential Experiment Design for Contour Estimation From Complex Computer Codes journal November 2008
Bayesian Treed Gaussian Process Models With an Application to Computer Modeling journal September 2008
Surrogates book January 2020
BART: Bayesian additive regression trees journal March 2010
A General Framework for Vecchia Approximations of Gaussian Processes journal February 2021
Multivariate Adaptive Regression Splines journal March 1991
Simulated Annealing journal February 1993
Active Learning with Statistical Models journal January 1996
tgp : An R Package for Bayesian Nonstationary, Semiparametric Nonlinear Regression and Design by Treed Gaussian Process Models journal January 2007
laGP : Large-Scale Spatial Modeling via Local Approximate Gaussian Processes in R journal January 2016
BASS : An R Package for Fitting and Performing Sensitivity Analysis of Bayesian Adaptive Spline Surfaces journal January 2020
Nonparametric Machine Learning and Efficient Computation with Bayesian Additive Regression Trees: The BART R Package journal January 2021
Design and analysis of computer experiments when the output is highly correlated over the input space journal March 2002