Training Spiking Neural Networks Using Combined Learning Approaches

Elbrecht, Daniel; Parsa, Maryam; Kulkarni, Shruti; Mitchell, Parker; Schuman, Catherine

doi:10.1109/SSCI47803.2020.9308443

Training Spiking Neural Networks Using Combined Learning Approaches

Conference · Mon Nov 30 23:00:00 EST 2020

DOI:https://doi.org/10.1109/SSCI47803.2020.9308443· OSTI ID:1760122

^[1]; Parsa, Maryam ^[1]; ^[1]; Mitchell, Parker ^[1]; ^[1]

ORNL

Spiking neural networks (SNNs), the class of neural networks used in neuromorphic computing, are difficult to train using traditional back-propagation techniques. Spike timingdependent plasticity (STDP) is a biologically inspired learning mechanism that can be used to train SNNs. Evolutionary algorithms have also been demonstrated as a method for training SNNs. In this work, we explore the relationship between these two training methodologies. We evaluate STDP and evolutionary optimization as standalone methods for training networks, and also evaluate a combined approach where STDP weight updates are applied within an evolutionary algorithm. We also apply Bayesian hyperparameter optimization as a meta learner for each of the algorithms. We find that STDP by itself is not an ideal learning rule for randomly connected networks, while the inclusion of STDP within an evolutionary algorithm leads to similar performance, with a few interesting differences. This study suggests future work in understanding the relationship between network topology and learning rules.

View Conference

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Organization:: USDOE; USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)

DOE Contract Number:: AC05-00OR22725

OSTI ID:: 1760122

Country of Publication:: United States

Language:: English

References (15)

Spike Timing-Dependent Plasticity of Neural Circuits Dan, Yang; Poo, Mu-ming Neuron, Vol. 44, Issue 1 https://doi.org/10.1016/j.neuron.2004.09.007	journal	September 2004
Bayesian Multi-objective Hyperparameter Optimization for Accurate, Fast, and Efficient Neural Network Accelerator Design Parsa, Maryam; Mitchell, John P.; Schuman, Catherine D. Frontiers in Neuroscience, Vol. 14 https://doi.org/10.3389/fnins.2020.00667	journal	July 2020
Evolutionary Optimization for Neuromorphic Systems Schuman, Catherine D.; Mitchell, J. Parker; Patton, Robert M. NICE '20: Neuro-inspired Computational Elements Workshop, Proceedings of the Neuro-inspired Computational Elements Workshop https://doi.org/10.1145/3381755.3381758	conference	June 2020
Error-backpropagation in temporally encoded networks of spiking neurons Bohte, Sander M.; Kok, Joost N.; La Poutré, Han Neurocomputing, Vol. 48, Issue 1-4 https://doi.org/10.1016/S0925-2312(01)00658-0	journal	October 2002
Non-Traditional Input Encoding Schemes for Spiking Neuromorphic Systems Schuman, Catherine D.; Plank, James S.; Bruer, Grant 2019 International Joint Conference on Neural Networks (IJCNN) https://doi.org/10.1109/IJCNN.2019.8852139	conference	July 2019
PABO: Pseudo Agent-Based Multi-Objective Bayesian Hyperparameter Optimization for Efficient Neural Accelerator Design Parsa, Maryam; Ankit, Aayush; Ziabari, Amirkoushyar 2019 IEEE/ACM International Conference on Computer-Aided Design (ICCAD) https://doi.org/10.1109/ICCAD45719.2019.8942046	conference	November 2019
An evolutionary optimization framework for neural networks and neuromorphic architectures Schuman, Catherine D.; Plank, James S.; Disney, Adam 2016 International Joint Conference on Neural Networks (IJCNN) https://doi.org/10.1109/IJCNN.2016.7727192	conference	July 2016
Hyperparameter Optimization in Binary Communication Networks for Neuromorphic Deployment Parsa, Maryam; Schuman, Catherine D.; Date, Prasanna 2020 International Joint Conference on Neural Networks (IJCNN) https://doi.org/10.1109/IJCNN48605.2020.9206872	conference	July 2020
Caspian Mitchell, J. Parker; Schuman, Catherine D.; Patton, Robert M. Proceedings of the Neuro-inspired Computational Elements Workshop https://doi.org/10.1145/3381755.3381764	conference	March 2020
Loihi: A Neuromorphic Manycore Processor with On-Chip Learning Davies, Mike; Srinivasa, Narayan; Lin, Tsung-Han IEEE Micro, Vol. 38, Issue 1 https://doi.org/10.1109/MM.2018.112130359	journal	January 2018
Evolving spiking neural networks for robot control Batllori, R.; Laramee, C. B.; Land, W. Procedia Computer Science, Vol. 6 https://doi.org/10.1016/j.procs.2011.08.060	journal	January 2011
Bayesian-based Hyperparameter Optimization for Spiking Neuromorphic Systems Parsa, Maryam; Mitchell, J. Parker; Schuman, Catherine D. 2019 IEEE International Conference on Big Data (Big Data) https://doi.org/10.1109/BigData47090.2019.9006383	conference	December 2019
Learning Real-World Stimuli in a Neural Network with Spike-Driven Synaptic Dynamics Brader, Joseph M.; Senn, Walter; Fusi, Stefano Neural Computation, Vol. 19, Issue 11 https://doi.org/10.1162/neco.2007.19.11.2881	journal	November 2007
Training Deep Spiking Neural Networks Using Backpropagation Lee, Jun Haeng; Delbruck, Tobi; Pfeiffer, Michael Frontiers in Neuroscience, Vol. 10 https://doi.org/10.3389/fnins.2016.00508	journal	November 2016
First-Spike-Based Visual Categorization Using Reward-Modulated STDP Mozafari, Milad; Kheradpisheh, Saeed Reza; Masquelier, Timothee IEEE Transactions on Neural Networks and Learning Systems, Vol. 29, Issue 12 https://doi.org/10.1109/TNNLS.2018.2826721	journal	December 2018

Similar Records

Training Spiking Neural Networks with Synaptic Plasticity under Integer Representation

Conference · Thu Jul 01 00:00:00 EDT 2021 · OSTI ID:1876349

Multi-Objective Hyperparameter Optimization for Spiking Neural Network Neuroevolution

Conference · Tue Jun 01 00:00:00 EDT 2021 · OSTI ID:1814329

A Software Framework for Comparing Training Approaches for Spiking Neuromorphic Systems

Conference · Thu Jul 01 00:00:00 EDT 2021 · OSTI ID:1823363

Training Spiking Neural Networks Using Combined Learning Approaches

Citation Formats

References (15)

Similar Records

Related Subjects