Training Spiking Neural Networks Using Combined Learning Approaches
Abstract
Spiking neural networks (SNNs), the class of neural networks used in neuromorphic computing, are difficult to train using traditional back-propagation techniques. Spike timingdependent plasticity (STDP) is a biologically inspired learning mechanism that can be used to train SNNs. Evolutionary algorithms have also been demonstrated as a method for training SNNs. In this work, we explore the relationship between these two training methodologies. We evaluate STDP and evolutionary optimization as standalone methods for training networks, and also evaluate a combined approach where STDP weight updates are applied within an evolutionary algorithm. We also apply Bayesian hyperparameter optimization as a meta learner for each of the algorithms. We find that STDP by itself is not an ideal learning rule for randomly connected networks, while the inclusion of STDP within an evolutionary algorithm leads to similar performance, with a few interesting differences. This study suggests future work in understanding the relationship between network topology and learning rules.
- Authors:
-
- ORNL
- Publication Date:
- Research Org.:
- Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
- OSTI Identifier:
- 1760122
- DOE Contract Number:
- AC05-00OR22725
- Resource Type:
- Conference
- Resource Relation:
- Conference: IEEE Symposium Series on Computational Intelligence (SSCI) - Canberra, , Australia - 12/1/2020 10:00:00 AM-12/4/2020 10:00:00 AM
- Country of Publication:
- United States
- Language:
- English
Citation Formats
Elbrecht, Daniel, Parsa, Maryam, Kulkarni, Shruti, Mitchell, Parker, and Schuman, Catherine. Training Spiking Neural Networks Using Combined Learning Approaches. United States: N. p., 2020.
Web. doi:10.1109/SSCI47803.2020.9308443.
Elbrecht, Daniel, Parsa, Maryam, Kulkarni, Shruti, Mitchell, Parker, & Schuman, Catherine. Training Spiking Neural Networks Using Combined Learning Approaches. United States. https://doi.org/10.1109/SSCI47803.2020.9308443
Elbrecht, Daniel, Parsa, Maryam, Kulkarni, Shruti, Mitchell, Parker, and Schuman, Catherine. 2020.
"Training Spiking Neural Networks Using Combined Learning Approaches". United States. https://doi.org/10.1109/SSCI47803.2020.9308443. https://www.osti.gov/servlets/purl/1760122.
@article{osti_1760122,
title = {Training Spiking Neural Networks Using Combined Learning Approaches},
author = {Elbrecht, Daniel and Parsa, Maryam and Kulkarni, Shruti and Mitchell, Parker and Schuman, Catherine},
abstractNote = {Spiking neural networks (SNNs), the class of neural networks used in neuromorphic computing, are difficult to train using traditional back-propagation techniques. Spike timingdependent plasticity (STDP) is a biologically inspired learning mechanism that can be used to train SNNs. Evolutionary algorithms have also been demonstrated as a method for training SNNs. In this work, we explore the relationship between these two training methodologies. We evaluate STDP and evolutionary optimization as standalone methods for training networks, and also evaluate a combined approach where STDP weight updates are applied within an evolutionary algorithm. We also apply Bayesian hyperparameter optimization as a meta learner for each of the algorithms. We find that STDP by itself is not an ideal learning rule for randomly connected networks, while the inclusion of STDP within an evolutionary algorithm leads to similar performance, with a few interesting differences. This study suggests future work in understanding the relationship between network topology and learning rules.},
doi = {10.1109/SSCI47803.2020.9308443},
url = {https://www.osti.gov/biblio/1760122},
journal = {},
number = ,
volume = ,
place = {United States},
year = {Tue Dec 01 00:00:00 EST 2020},
month = {Tue Dec 01 00:00:00 EST 2020}
}