Advanced stationary and nonstationary kernel designs for domain-aware Gaussian processes
Abstract
Gaussian process regression is a widely-applied method for function approximation and uncertainty quantification. The technique has gained popularity recently in the machine learning community due to its robustness and interpretability. The mathematical methods we discuss in this paper are an extension of the Gaussian-process framework. We are proposing advanced kernel designs that only allow for functions with certain desirable characteristics to be elements of the reproducing kernel Hilbert space (RKHS) that underlies all kernel methods and serves as the sample space for Gaussian process regression. These desirable characteristics reflect the underlying physics; two obvious examples are symmetry and periodicity constraints. In addition, non-stationary kernel designs can be defined in the same framework to yield flexible multi-task Gaussian processes. We will show the impact of advanced kernel designs on Gaussian processes using several synthetic and two scientific data sets. The results of our research show that including domain knowledge, communicated through advanced kernel designs, has a significant impact on the accuracy and relevance of the function approximation.
- Authors:
-
- Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). The Center for Advanced Mathematics for Energy Research Applications (CAMERA)
- Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). The Center for Advanced Mathematics for Energy Research Applications (CAMERA); Univ. of California, Berkeley, CA (United States)
- Publication Date:
- Research Org.:
- Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC), High Energy Physics (HEP); Center for Advanced Mathematics for Energy Research Applications (CAMERA)
- OSTI Identifier:
- 1825900
- Grant/Contract Number:
- AC02-05CH11231
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Communications in Applied Mathematics and Computational Science
- Additional Journal Information:
- Journal Volume: 17; Journal Issue: 1; Journal ID: ISSN 1559-3940
- Publisher:
- Mathematical Sciences Publishers
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 97 MATHEMATICS AND COMPUTING; Gaussian processes; machine learning; uncertainty quantification
Citation Formats
Noack, Marcus M., and Sethian, James A. Advanced stationary and nonstationary kernel designs for domain-aware Gaussian processes. United States: N. p., 2022.
Web. doi:10.2140/camcos.2022.17.131.
Noack, Marcus M., & Sethian, James A. Advanced stationary and nonstationary kernel designs for domain-aware Gaussian processes. United States. https://doi.org/10.2140/camcos.2022.17.131
Noack, Marcus M., and Sethian, James A. Fri .
"Advanced stationary and nonstationary kernel designs for domain-aware Gaussian processes". United States. https://doi.org/10.2140/camcos.2022.17.131. https://www.osti.gov/servlets/purl/1825900.
@article{osti_1825900,
title = {Advanced stationary and nonstationary kernel designs for domain-aware Gaussian processes},
author = {Noack, Marcus M. and Sethian, James A.},
abstractNote = {Gaussian process regression is a widely-applied method for function approximation and uncertainty quantification. The technique has gained popularity recently in the machine learning community due to its robustness and interpretability. The mathematical methods we discuss in this paper are an extension of the Gaussian-process framework. We are proposing advanced kernel designs that only allow for functions with certain desirable characteristics to be elements of the reproducing kernel Hilbert space (RKHS) that underlies all kernel methods and serves as the sample space for Gaussian process regression. These desirable characteristics reflect the underlying physics; two obvious examples are symmetry and periodicity constraints. In addition, non-stationary kernel designs can be defined in the same framework to yield flexible multi-task Gaussian processes. We will show the impact of advanced kernel designs on Gaussian processes using several synthetic and two scientific data sets. The results of our research show that including domain knowledge, communicated through advanced kernel designs, has a significant impact on the accuracy and relevance of the function approximation.},
doi = {10.2140/camcos.2022.17.131},
journal = {Communications in Applied Mathematics and Computational Science},
number = 1,
volume = 17,
place = {United States},
year = {Fri Oct 07 00:00:00 EDT 2022},
month = {Fri Oct 07 00:00:00 EDT 2022}
}
Free Publicly Available Full Text
Publisher's Version of Record
Other availability
Search WorldCat to find libraries that may hold this journal
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
A Survey of Constrained Gaussian Process Regression: Approaches and Implementation Challenges
journal, January 2020
- Swiler, Laura P.; Gulian, Mamikon; Frankel, Ari L.
- Journal of Machine Learning for Modeling and Computing, Vol. 1, Issue 2
Estimating Shape Constrained Functions Using Gaussian Processes
journal, January 2016
- Wang, Xiaojing; Berger, James O.
- SIAM/ASA Journal on Uncertainty Quantification, Vol. 4, Issue 1
High-Performance Hybrid-Global-Deflated-Local Optimization with Applications to Active Learning
conference, November 2021
- Noack, Marcus Michael; Perryman, David; Krishnan, Harinarayan
- 2021 3rd Annual Workshop on Extreme-scale Experiment-in-the-Loop Computing (XLOOP)
A Review of Kernel Methods for Feature Extraction in Nonlinear Process Monitoring
journal, December 2019
- Pilario, Karl Ezra; Shafiee, Mahmood; Cao, Yi
- Processes, Vol. 8, Issue 1
Regression-based covariance functions for nonstationary spatial modeling: Regression-based covariance functions
journal, March 2015
- Risser, Mark D.; Calder, Catherine A.
- Environmetrics, Vol. 26, Issue 4
Gaussian processes for autonomous data acquisition at large-scale synchrotron and neutron facilities
journal, July 2021
- Noack, Marcus M.; Zwart, Petrus H.; Ushizima, Daniela M.
- Nature Reviews Physics, Vol. 3, Issue 10
Constrained Gaussian Process Learning for Model Predictive Control
journal, January 2020
- Matschek, Janine; Himmel, Andreas; Sundmacher, Kai
- IFAC-PapersOnLine, Vol. 53, Issue 2
Polarized inelastic neutron scattering of nonreciprocal spin waves in MnSi
journal, August 2019
- Weber, T.; Waizner, J.; Steffens, P.
- Physical Review B, Vol. 100, Issue 6
Learning Gaussian processes from multiple tasks
conference, January 2005
- Yu, Kai; Tresp, Volker; Schwaighofer, Anton
- Proceedings of the 22nd international conference on Machine learning - ICML '05
Autonomous materials discovery driven by Gaussian process regression with inhomogeneous measurement noise and anisotropic kernels
journal, October 2020
- Noack, Marcus M.; Doerk, Gregory S.; Li, Ruipeng
- Scientific Reports, Vol. 10, Issue 1
Hybrid genetic deflated Newton method for global optimisation
journal, December 2017
- Noack, Marcus M.; Funke, Simon W.
- Journal of Computational and Applied Mathematics, Vol. 325
A survey on multi-output regression: Multi-output regression survey
journal, July 2015
- Borchani, Hanen; Varando, Gherardo; Bielza, Concha
- Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, Vol. 5, Issue 5
Spatial modelling using a new class of nonstationary covariance functions
journal, January 2006
- Paciorek, Christopher J.; Schervish, Mark J.
- Environmetrics, Vol. 17, Issue 5
Hybrid genetic deflated Newton method for global optimisation
journal, December 2017
- Noack, Marcus M.; Funke, Simon W.
- Journal of Computational and Applied Mathematics, Vol. 325
Constrained Gaussian Process Learning for Model Predictive Control
preprint, January 2019
- Matschek, Janine; Himmel, Andreas; Sundmacher, Kai
- arXiv