Bayesian-based Hyperparameter Optimization for Spiking Neuromorphic Systems

Parsa, Maryam; Mitchell, Parker; Schuman, Catherine; Patton, Robert; Potok, Thomas; Roy, Kaushik

doi:10.1109/BigData47090.2019.9006383

Bayesian-based Hyperparameter Optimization for Spiking Neuromorphic Systems

Conference · Sat Nov 30 23:00:00 EST 2019

DOI:https://doi.org/10.1109/BigData47090.2019.9006383· OSTI ID:1606790

Parsa, Maryam ^[1]; Mitchell, Parker ^[2]; ^[2]; ^[2]; ^[2]; Roy, Kaushik ^[1]

Purdue University
ORNL

Designing a neuromorphic computing system involves selection of several hyperparameters that not only affect the accuracy of the framework, but also the energy efficiency and speed of inference and training. These hyperparameters might be inherent to the training of the spiking neural network (SNN), the input/output encoding of the real-world data to spikes, or the underlying neuromorphic hardware. In this work, we present a Bayesian-based hyperparameter optimization approach for spiking neuromorphic systems, and we show how this optimization framework can lead to significant improvement in designing accurate neuromorphic computing systems. In particular, we show that this hyperparameter optimization approach can discover the same optimal hyperparameter set for input encoding as a grid search, but with far fewer evaluations and far less time. We also show the impact of hardware-specific hyperparameters on the performance of the system, and we demonstrate that by optimizing these hyperparameters, we can achieve significantly better application performance.

View Conference

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Organization:: USDOE; USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)

DOE Contract Number:: AC05-00OR22725

OSTI ID:: 1606790

Country of Publication:: United States

Language:: English

References (15)

The TENNLab Exploratory Neuromorphic Computing Framework Plank, James S.; Schuman, Catherine D.; Bruer, Grant IEEE Letters of the Computer Society, Vol. 1, Issue 2 https://doi.org/10.1109/LOCS.2018.2885976	journal	July 2018
Benchmarks for progress in neuromorphic computing Davies, Mike Nature Machine Intelligence, Vol. 1, Issue 9 https://doi.org/10.1038/s42256-019-0097-1	journal	September 2019
Neuromorphic computing gets ready for the (really) big time Monroe, Don Communications of the ACM, Vol. 57, Issue 6 https://doi.org/10.1145/2601069	journal	June 2014
DANNA 2: Dynamic Adaptive Neural Network Arrays Mitchell, J. Parker; Dean, Mark E.; Bruer, Grant R. Proceedings of the International Conference on Neuromorphic Systems - ICONS '18 https://doi.org/10.1145/3229884.3229894	conference	January 2018
TrueNorth: Design and Tool Flow of a 65 mW 1 Million Neuron Programmable Neurosynaptic Chip Akopyan, Filipp; Sawada, Jun; Cassidy, Andrew IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, Vol. 34, Issue 10 https://doi.org/10.1109/TCAD.2015.2474396	journal	October 2015
PUMA: A Programmable Ultra-efficient Memristor-based Accelerator for Machine Learning Inference Ankit, Aayush; Hajj, Izzat El; Chalamalasetti, Sai Rahul ASPLOS '19: Architectural Support for Programming Languages and Operating Systems, Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems https://doi.org/10.1145/3297858.3304049	conference	April 2019
Non-Traditional Input Encoding Schemes for Spiking Neuromorphic Systems Schuman, Catherine D.; Plank, James S.; Bruer, Grant 2019 International Joint Conference on Neural Networks (IJCNN) https://doi.org/10.1109/IJCNN.2019.8852139	conference	July 2019
Analysis of Liquid Ensembles for Enhancing the Performance and Accuracy of Liquid State Machines Wijesinghe, Parami; Srinivasan, Gopalakrishnan; Panda, Priyadarshini Frontiers in Neuroscience, Vol. 13 https://doi.org/10.3389/fnins.2019.00504	journal	May 2019
PABO: Pseudo Agent-Based Multi-Objective Bayesian Hyperparameter Optimization for Efficient Neural Accelerator Design Parsa, Maryam; Ankit, Aayush; Ziabari, Amirkoushyar 2019 IEEE/ACM International Conference on Computer-Aided Design (ICCAD) https://doi.org/10.1109/ICCAD45719.2019.8942046	conference	November 2019
Evolving neural network controllers for unstable systems Wieland, A. P. IJCNN-91-Seattle International Joint Conference on Neural Networks https://doi.org/10.1109/IJCNN.1991.155416	conference	January 1991
An empirical evaluation of deep architectures on problems with many factors of variation Larochelle, Hugo; Erhan, Dumitru; Courville, Aaron Proceedings of the 24th international conference on Machine learning - ICML '07 https://doi.org/10.1145/1273496.1273556	conference	January 2007
An evolutionary optimization framework for neural networks and neuromorphic architectures Schuman, Catherine D.; Plank, James S.; Disney, Adam 2016 International Joint Conference on Neural Networks (IJCNN) https://doi.org/10.1109/IJCNN.2016.7727192	conference	July 2016
Going Deeper in Spiking Neural Networks: VGG and Residual Architectures Sengupta, Abhronil; Ye, Yuting; Wang, Robert Frontiers in Neuroscience, Vol. 13 https://doi.org/10.3389/fnins.2019.00095	journal	March 2019
Loihi: A Neuromorphic Manycore Processor with On-Chip Learning Davies, Mike; Srinivasa, Narayan; Lin, Tsung-Han IEEE Micro, Vol. 38, Issue 1 https://doi.org/10.1109/MM.2018.112130359	journal	January 2018
Simulation of a memristor-based spiking neural network immune to device variations Querlioz, Damien; Bichler, Olivier; Gamrat, Christian The 2011 International Joint Conference on Neural Networks https://doi.org/10.1109/IJCNN.2011.6033439	conference	July 2011

Similar Records

Multi-Objective Hyperparameter Optimization for Spiking Neural Network Neuroevolution

Conference · Tue Jun 01 00:00:00 EDT 2021 · OSTI ID:1814329

A Software Framework for Comparing Training Approaches for Spiking Neuromorphic Systems

Conference · Thu Jul 01 00:00:00 EDT 2021 · OSTI ID:1823363

Bayesian-based Hyperparameter Optimization for Spiking Neuromorphic Systems

Citation Formats

References (15)

Similar Records

Related Subjects