Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

Deep kernel methods learn better: from cards to process optimization

Journal Article · · Machine Learning: Science and Technology
; ; ;
Abstract

The ability of deep learning methods to perform classification and regression tasks relies heavily on their capacity to uncover manifolds in high-dimensional data spaces and project them into low-dimensional representation spaces. In this study, we investigate the structure and character of the manifolds generated by classical variational autoencoder (VAE) approaches and deep kernel learning (DKL). In the former case, the structure of the latent space is determined by the properties of the input data alone, while in the latter, the latent manifold forms as a result of an active learning process that balances the data distribution and target functionalities. We show that DKL with active learning can produce a more compact and smooth latent space which is more conducive to optimization compared to previously reported methods, such as the VAE. We demonstrate this behavior using a simple cards dataset and extend it to the optimization of domain-generated trajectories in physical systems. Our findings suggest that latent manifolds constructed through active learning have a more beneficial structure for optimization problems, especially in feature-rich target-poor scenarios that are common in domain sciences, such as materials synthesis, energy storage, and molecular discovery. The Jupyter Notebooks that encapsulate the complete analysis accompany the article.

Research Organization:
University of Washington, Seattle, WA (United States)
Sponsoring Organization:
USDOE; USDOE Office of Science (SC), Basic Energy Sciences (BES). Materials Sciences & Engineering Division (MSE); USDOE Office of Science (SC), Basic Energy Sciences (BES). Scientific User Facilities (SUF)
Grant/Contract Number:
SC0019288
OSTI ID:
2281927
Alternate ID(s):
OSTI ID: 2339579
OSTI ID: 2274985
Journal Information:
Machine Learning: Science and Technology, Journal Name: Machine Learning: Science and Technology Journal Issue: 1 Vol. 5; ISSN 2632-2153
Publisher:
IOP PublishingCopyright Statement
Country of Publication:
United Kingdom
Language:
English

References (44)

Latent Mechanisms of Polarization Switching from In Situ Electron Microscopy Observations journal March 2022
Hypothesis Learning in Automated Experiment: Application to Combinatorial Materials Libraries journal April 2022
Good practices for Bayesian optimization of high dimensional structured spaces journal May 2021
Unsupervised Learning Algorithms book January 2016
A thermodynamics-informed active learning approach to perception and reasoning about fluids journal February 2023
Indoor Scene Understanding with RGB-D Images: Bottom-up Segmentation, Object Detection and Semantic Segmentation journal November 2014
Regression markets and application to energy forecasting journal May 2022
An application of supervised and unsupervised learning approaches to telecommunications fraud detection journal October 2008
Psychedelic-assisted psychotherapy to treat psychiatric and existential distress in life-threatening medical illnesses and palliative care journal September 2022
Bayesian Optimization book January 2023
Combining Machine Learning and Computational Chemistry for Predictive Insights Into Chemical Systems journal July 2021
A Model Ensemble Approach Enables Data-Driven Property Prediction for Chemically Deconstructable Thermosets in the Low-Data Regime journal September 2023
Disentangling Rotational Dynamics and Ordering Transitions in a System of Self-Organizing Protein Nanorods via Rotationally Invariant Latent Representations journal April 2021
Bayesian Active Learning for Scanning Probe Microscopy: From Gaussian Processes to Hypothesis Learning journal September 2022
On-the-fly closed-loop materials discovery via Bayesian active learning journal November 2020
Active learning in materials science with emphasis on adaptive sampling using uncertainties for targeted design journal February 2019
Ensemble learning-iterative training machine learning for uncertainty quantification and automated experiment in atom-resolved microscopy journal July 2021
Active learning to overcome exponential-wall problem for effective structure prediction of chemical-disordered materials journal January 2023
Learning the right channel in multimodal imaging: automated experiment in piezoresponse force microscopy journal March 2023
Closed-loop optimization of fast-charging protocols for batteries with machine learning journal February 2020
Optimal charging profiles for mechanically constrained lithium-ion batteries journal January 2014
Constrained Bayesian optimization for automatic chemical design using variational autoencoders journal January 2020
Bayesian optimization in continuous spaces via virtual process embeddings journal January 2022
Quantifying the performance of machine learning models in materials discovery journal January 2023
Guided search for desired functional responses via Bayesian optimization of generative model: Hysteresis loop shape engineering in ferroelectrics journal July 2020
Unsupervised learning of ferroic variants from atomically resolved STEM images journal October 2022
Discovery of structure–property relations for molecules via hypothesis-driven active learning over the chemical space journal October 2023
Variational Inference: A Review for Statisticians journal July 2016
Analysis of ferroelectric switching in finite media as a Landau-type phase transition journal January 1998
Real-time model predictive control for the optimal charging of a lithium-ion battery conference July 2015
Unsupervised Anomaly Detection of Industrial Robots Using Sliding-Window Convolutional Variational Autoencoder journal January 2020
Towards Visually Explaining Variational Autoencoders conference June 2020
Multi-Sensor Data Annotation Using Sequence-based Active Learning conference October 2022
Weather and Light Level Classification for Autonomous Driving: Dataset, Baseline and Active Learning conference September 2021
Deep Active Learning for Efficient Training of a LiDAR 3D Object Detector conference June 2019
Discriminative Active Learning for Robotic Grasping in Cluttered Scene journal March 2023
Time Constraint Finite-Horizon Path Planning Solution for Micromouse Extreme Problem conference October 2022
Understanding Convolution for Semantic Segmentation conference March 2018
Exploring order parameters and dynamic processes in disordered systems via variational autoencoders journal April 2021
An Introduction to Variational Autoencoders journal January 2019
Deep Gaussian Process for Crop Yield Prediction Based on Remote Sensing Data journal February 2017
Unmanned Aerial Vehicle Classification, Applications and Challenges: A Review preprint November 2018
Reduced-Complexity End-to-End Variational Autoencoder for on Board Satellite Image Compression journal January 2021
Applied Regression: An Introduction journal January 2016

Similar Records

Related Subjects

97 MATHEMATICS AND COMPUTING
Bayesian optimization
deep kernel learning
ferroelectric kinetic model
high-dimensional optimization
latent space
process optimization