Training Spiking Neural Networks with Synaptic Plasticity under Integer Representation

Kulkarni, Shruti; Parsa, Maryam; Mitchell, Parker; Schuman, Catherine

doi:10.1145/3477145.3477152

Training Spiking Neural Networks with Synaptic Plasticity under Integer Representation

Conference · Thu Jul 01 00:00:00 EDT 2021

DOI:https://doi.org/10.1145/3477145.3477152· OSTI ID:1876349

^[1]; Parsa, Maryam ^[1]; Mitchell, Parker ^[1]; ^[1]

ORNL

Neuromorphic computing is emerging as a promising Beyond Moore computing paradigm that employs event-triggered computation and non-von Neumann hardware. Spike Timing Dependent Plasticity (STDP) is a well-known bio-inspired learning rule that relies on activities of locally connected neurons to adjust the weights of their respective synapses. In this work, we analyze a basic STDP rule and its sensitivity on the different hyperparameters for training spiking neural networks (SNNs) with supervision, customized for a neuromorphic hardware implementation with integer weights. We compare the classification performance on four UCI datasets (iris, wine, breast cancer and digits) that depict varying levels of complexity. We perform a search for optimal set of hyperparameters using both grid search and Bayesian optimization. Through the use of Bayesian optimization, we show the general trends in hyperparameter sensitivity in SNN classification problem. With the best sets of hyperparameters, we achieve accuracies comparable to some of the best performing SNNs on these four datasets. With a highly optimized supervised STDP rule we show that these accuracies can be achieved with just 20 epochs of training.

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Organization:: USDOE; USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)

DOE Contract Number:: AC05-00OR22725

OSTI ID:: 1876349

Country of Publication:: United States

Language:: English

References (12)

The TENNLab Exploratory Neuromorphic Computing Framework Plank, James S.; Schuman, Catherine D.; Bruer, Grant IEEE Letters of the Computer Society, Vol. 1, Issue 2 https://doi.org/10.1109/LOCS.2018.2885976	journal	July 2018
Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type Bi, Guo-qiang; Poo, Mu-ming The Journal of Neuroscience, Vol. 18, Issue 24 https://doi.org/10.1523/JNEUROSCI.18-24-10464.1998	journal	December 1998
Non-Traditional Input Encoding Schemes for Spiking Neuromorphic Systems Schuman, Catherine D.; Plank, James S.; Bruer, Grant 2019 International Joint Conference on Neural Networks (IJCNN) https://doi.org/10.1109/IJCNN.2019.8852139	conference	July 2019
PABO: Pseudo Agent-Based Multi-Objective Bayesian Hyperparameter Optimization for Efficient Neural Accelerator Design Parsa, Maryam; Ankit, Aayush; Ziabari, Amirkoushyar 2019 IEEE/ACM International Conference on Computer-Aided Design (ICCAD) https://doi.org/10.1109/ICCAD45719.2019.8942046	conference	November 2019
Spike Timing–Dependent Plasticity: A Hebbian Learning Rule Caporale, Natalia; Dan, Yang Annual Review of Neuroscience, Vol. 31, Issue 1 https://doi.org/10.1146/annurev.neuro.31.060407.125639	journal	July 2008
Building Brain-Inspired Computing Systems: Examining the Role of Nanoscale Devices Nandakumar, S. R.; Kulkarni, Shruti R.; Babu, Anakha V. IEEE Nanotechnology Magazine, Vol. 12, Issue 3 https://doi.org/10.1109/MNANO.2018.2845078	journal	September 2018
Unsupervised speech recognition through spike-timing-dependent plasticity in a convolutional spiking neural network Dong, Meng; Huang, Xuhui; Xu, Bo PLOS ONE, Vol. 13, Issue 11 https://doi.org/10.1371/journal.pone.0204596	journal	November 2018
Caspian Mitchell, J. Parker; Schuman, Catherine D.; Patton, Robert M. Proceedings of the Neuro-inspired Computational Elements Workshop https://doi.org/10.1145/3381755.3381764	conference	March 2020
Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-Based Optimization to Spiking Neural Networks Neftci, Emre O.; Mostafa, Hesham; Zenke, Friedemann IEEE Signal Processing Magazine, Vol. 36, Issue 6 https://doi.org/10.1109/MSP.2019.2931595	journal	November 2019
Bayesian-based Hyperparameter Optimization for Spiking Neuromorphic Systems Parsa, Maryam; Mitchell, J. Parker; Schuman, Catherine D. 2019 IEEE International Conference on Big Data (Big Data) https://doi.org/10.1109/BigData47090.2019.9006383	conference	December 2019
Training deep neural networks for binary communication with the Whetstone method Severa, William; Vineyard, Craig M.; Dellana, Ryan Nature Machine Intelligence, Vol. 1, Issue 2 https://doi.org/10.1038/s42256-018-0015-y	journal	January 2019
First-Spike-Based Visual Categorization Using Reward-Modulated STDP Mozafari, Milad; Kheradpisheh, Saeed Reza; Masquelier, Timothee IEEE Transactions on Neural Networks and Learning Systems, Vol. 29, Issue 12 https://doi.org/10.1109/TNNLS.2018.2826721	journal	December 2018

Similar Records

Training Spiking Neural Networks Using Combined Learning Approaches

Conference · Mon Nov 30 23:00:00 EST 2020 · OSTI ID:1760122

Bayesian-based Hyperparameter Optimization for Spiking Neuromorphic Systems

Conference · Sat Nov 30 23:00:00 EST 2019 · OSTI ID:1606790

A Software Framework for Comparing Training Approaches for Spiking Neuromorphic Systems

Conference · Thu Jul 01 00:00:00 EDT 2021 · OSTI ID:1823363

Training Spiking Neural Networks with Synaptic Plasticity under Integer Representation

Citation Formats

References (12)

Similar Records

Related Subjects