Adaptive hyperparameter updating for training restricted Boltzmann machines on quantum annealers
- Florida A & M Univ.-Florida State Univ., Tallahassee, FL (United States). Dept. of Mechanical Engineering; Florida Center for Advanced Aero-Propulsion (FCAAP), Tallahassee, FL (United States); OSTI
- Florida A & M Univ.-Florida State Univ., Tallahassee, FL (United States). Dept. of Mechanical Engineering; Florida Center for Advanced Aero-Propulsion (FCAAP), Tallahassee, FL (United States)
Restricted Boltzmann Machines (RBMs) have been proposed for developing neural networks for a variety of unsupervised machine learning applications such as image recognition, drug discovery, and materials design. The Boltzmann probability distribution is used as a model to identify network parameters by optimizing the likelihood of predicting an output given hidden states trained on available data. Training such networks often requires sampling over a large probability space that must be approximated during gradient based optimization. Quantum annealing has been proposed as a means to search this space more efficiently which has been experimentally investigated on D-Wave hardware. D-Wave implementation requires selection of an effective inverse temperature or hyperparameter (β) within the Boltzmann distribution which can strongly influence optimization. Here, we show how this parameter can be estimated as a hyperparameter applied to D-Wave hardware during neural network training by maximizing the likelihood or minimizing the Shannon entropy. We find both methods improve training RBMs based upon D-Wave hardware experimental validation on an image recognition problem. Neural network image reconstruction errors are evaluated using Bayesian uncertainty analysis which illustrate more than an order magnitude lower image reconstruction error using the maximum likelihood over manually optimizing the hyperparameter. The maximum likelihood method is also shown to out-perform minimizing the Shannon entropy for image reconstruction.
- Research Organization:
- Florida A & M Univ.-Florida State Univ., Tallahassee, FL (United States)
- Sponsoring Organization:
- USDOE Office of Science (SC), Basic Energy Sciences (BES). Scientific User Facilities Division
- Grant/Contract Number:
- AC05-00OR22725
- OSTI ID:
- 1816563
- Journal Information:
- Scientific Reports, Journal Name: Scientific Reports Journal Issue: 1 Vol. 11; ISSN 2045-2322
- Publisher:
- Nature Publishing GroupCopyright Statement
- Country of Publication:
- United States
- Language:
- English
Similar Records
Restricted Boltzmann Machines for galaxy morphology classification with a quantum annealer