skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Training Spiking Neural Networks Using Combined Learning Approaches

Abstract

Spiking neural networks (SNNs), the class of neural networks used in neuromorphic computing, are difficult to train using traditional back-propagation techniques. Spike timingdependent plasticity (STDP) is a biologically inspired learning mechanism that can be used to train SNNs. Evolutionary algorithms have also been demonstrated as a method for training SNNs. In this work, we explore the relationship between these two training methodologies. We evaluate STDP and evolutionary optimization as standalone methods for training networks, and also evaluate a combined approach where STDP weight updates are applied within an evolutionary algorithm. We also apply Bayesian hyperparameter optimization as a meta learner for each of the algorithms. We find that STDP by itself is not an ideal learning rule for randomly connected networks, while the inclusion of STDP within an evolutionary algorithm leads to similar performance, with a few interesting differences. This study suggests future work in understanding the relationship between network topology and learning rules.

Authors:
ORCiD logo [1];  [1]; ORCiD logo [1];  [1]; ORCiD logo [1]
  1. ORNL
Publication Date:
Research Org.:
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
OSTI Identifier:
1760122
DOE Contract Number:  
AC05-00OR22725
Resource Type:
Conference
Resource Relation:
Conference: IEEE Symposium Series on Computational Intelligence (SSCI) - Canberra, , Australia - 12/1/2020 10:00:00 AM-12/4/2020 10:00:00 AM
Country of Publication:
United States
Language:
English

Citation Formats

Elbrecht, Daniel, Parsa, Maryam, Kulkarni, Shruti, Mitchell, Parker, and Schuman, Catherine. Training Spiking Neural Networks Using Combined Learning Approaches. United States: N. p., 2020. Web. doi:10.1109/SSCI47803.2020.9308443.
Elbrecht, Daniel, Parsa, Maryam, Kulkarni, Shruti, Mitchell, Parker, & Schuman, Catherine. Training Spiking Neural Networks Using Combined Learning Approaches. United States. https://doi.org/10.1109/SSCI47803.2020.9308443
Elbrecht, Daniel, Parsa, Maryam, Kulkarni, Shruti, Mitchell, Parker, and Schuman, Catherine. 2020. "Training Spiking Neural Networks Using Combined Learning Approaches". United States. https://doi.org/10.1109/SSCI47803.2020.9308443. https://www.osti.gov/servlets/purl/1760122.
@article{osti_1760122,
title = {Training Spiking Neural Networks Using Combined Learning Approaches},
author = {Elbrecht, Daniel and Parsa, Maryam and Kulkarni, Shruti and Mitchell, Parker and Schuman, Catherine},
abstractNote = {Spiking neural networks (SNNs), the class of neural networks used in neuromorphic computing, are difficult to train using traditional back-propagation techniques. Spike timingdependent plasticity (STDP) is a biologically inspired learning mechanism that can be used to train SNNs. Evolutionary algorithms have also been demonstrated as a method for training SNNs. In this work, we explore the relationship between these two training methodologies. We evaluate STDP and evolutionary optimization as standalone methods for training networks, and also evaluate a combined approach where STDP weight updates are applied within an evolutionary algorithm. We also apply Bayesian hyperparameter optimization as a meta learner for each of the algorithms. We find that STDP by itself is not an ideal learning rule for randomly connected networks, while the inclusion of STDP within an evolutionary algorithm leads to similar performance, with a few interesting differences. This study suggests future work in understanding the relationship between network topology and learning rules.},
doi = {10.1109/SSCI47803.2020.9308443},
url = {https://www.osti.gov/biblio/1760122}, journal = {},
number = ,
volume = ,
place = {United States},
year = {Tue Dec 01 00:00:00 EST 2020},
month = {Tue Dec 01 00:00:00 EST 2020}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share: