Optimizing training trajectories in variational autoencoders via latent Bayesian optimization approach*
Unsupervised and semi-supervised ML methods such as variational autoencoders (VAE) have become widely adopted across multiple areas of physics, chemistry, and materials sciences due to their capability in disentangling representations and ability to find latent manifolds for classification and/or regression of complex experimental data. Like other ML problems, VAEs require hyperparameter tuning, e.g. balancing the Kullback–Leibler and reconstruction terms. However, the training process and resulting manifold topology and connectivity depend not only on hyperparameters, but also their evolution during training. Because of the inefficiency of exhaustive search in a high-dimensional hyperparameter space for the expensive-to-train models, here we have explored a latent Bayesian optimization (zBO) approach for the hyperparameter trajectory optimization for the unsupervised and semi-supervised ML and demonstrated for joint-VAE with rotational invariances. We have demonstrated an application of this method for finding joint discrete and continuous rotationally invariant representations for modified national institute of standards and technology database (MNIST) and experimental data of a plasmonic nanoparticles material system. The performance of the proposed approach has been discussed extensively, where it allows for any high dimensional hyperparameter trajectory optimization of other ML models.
- Research Organization:
- Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States); Energy Frontier Research Centers (EFRC) (United States). Center for the Science of Synthesis Across Scales (CSSAS); Univ. of Washington, Seattle, WA (United States)
- Sponsoring Organization:
- USDOE Office of Science (SC), Basic Energy Sciences (BES)
- Grant/Contract Number:
- AC05-00OR22725; SC0019288
- OSTI ID:
- 1923196
- Alternate ID(s):
- OSTI ID: 1960555
- Journal Information:
- Machine Learning: Science and Technology, Vol. 4, Issue 1; ISSN 2632-2153
- Publisher:
- IOP PublishingCopyright Statement
- Country of Publication:
- United States
- Language:
- English
Similar Records
Deep kernel methods learn better: from cards to process optimization
Why it is Unfortunate that Linear Machine Learning “Works” so well in Electromechanical Switching of Ferroelectric Thin Films