Understanding Selection And Diversity For Evolution Of Spiking Recurrent Neural Networks

Schuman, Catherine; Bruer, Grant; Young, Aaron; Dean, Mark; Plank, James

doi:10.1109/IJCNN.2018.8489149

Understanding Selection And Diversity For Evolution Of Spiking Recurrent Neural Networks

Conference · Mon Oct 01 00:00:00 EDT 2018

DOI:https://doi.org/10.1109/IJCNN.2018.8489149· OSTI ID:1479770

^[1]; Bruer, Grant ^[2]; Young, Aaron ^[2]; Dean, Mark ^[2]; Plank, James ^[2]

ORNL
University of Tennessee (UT)

Evolutionary optimization or genetic algorithms have been used to optimize a variety of neural network types, including spiking recurrent neural networks, and are attractive for many reasons. However, a key impediment to their widespread use is the potential for slow training times and failure to converge to a good fitness value in a reasonable amount of time. In this work, we evaluate the effect of different selection algorithms on the performance of an evolutionary optimization method for designing spiking recurrent neural networks, including those that are meant to be deployed in a neuromorphic system. We propose a selection approach that utilizes a richer understanding of the fitness of an individual network to inform the selection process and to promote diversity in the population. We show that including this feature can provide a significant increase in performance over utilizing a standard selection approach.

View Conference

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Organization:: USDOE

DOE Contract Number:: AC05-00OR22725

OSTI ID:: 1479770

Country of Publication:: United States

Language:: English

References (24)

A VLSI Implementation of an Analog Neural Network Suited for Genetic Algorithms Schemmel, Johannes; Meier, Karlheinz; Schürmann, Felix Evolvable Systems: From Biology to Hardware https://doi.org/10.1007/3-540-45443-8_5	book	January 2001
Neuroevolution: from architectures to learning Floreano, Dario; Dürr, Peter; Mattiussi, Claudio Evolutionary Intelligence, Vol. 1, Issue 1 https://doi.org/10.1007/s12065-007-0002-4	journal	January 2008
Parallel Evolutionary Optimization for Neuromorphic Network Training Schuman, Catherine D.; Disney, Adam; Singh, Susheela P. 2016 2nd Workshop on Machine Learning in HPC Environments (MLHPC) https://doi.org/10.1109/MLHPC.2016.008	conference	November 2016
Evolving spiking neural networks for personalised modelling, classification and prediction of spatio-temporal patterns with a case study on stroke Kasabov, Nikola; Feigin, Valery; Hou, Zeng-Guang Neurocomputing, Vol. 134 https://doi.org/10.1016/j.neucom.2013.09.049	journal	June 2014
Spike timing dependent plasticity based enhanced self-learning for efficient pattern recognition in spiking neural networks Srinivasan, Gopalakrishnan; Roy, Sourjya; Raghunathan, Vijay 2017 International Joint Conference on Neural Networks (IJCNN) https://doi.org/10.1109/IJCNN.2017.7966075	conference	May 2017
Evolving Neural Networks through Augmenting Topologies Stanley, Kenneth O.; Miikkulainen, Risto Evolutionary Computation, Vol. 10, Issue 2 https://doi.org/10.1162/106365602320169811	journal	June 2002
Error-backpropagation in temporally encoded networks of spiking neurons Bohte, Sander M.; Kok, Joost N.; La Poutré, Han Neurocomputing, Vol. 48, Issue 1-4 https://doi.org/10.1016/S0925-2312(01)00658-0	journal	October 2002
A Digital Liquid State Machine With Biologically Inspired Learning and Its Application to Speech Recognition Zhang, Yong; Li, Peng; Jin, Yingyezhe IEEE Transactions on Neural Networks and Learning Systems, Vol. 26, Issue 11 https://doi.org/10.1109/TNNLS.2015.2388544	journal	November 2015
Mutation-Based Genetic Neural Network Palmes, P. P.; Hayasaka, T.; Usui, S. IEEE Transactions on Neural Networks, Vol. 16, Issue 3 https://doi.org/10.1109/TNN.2005.844858	journal	May 2005
Towards evolving spiking networks with memristive synapses Howard, Gerard; Gale, Ella; Bull, Larry 2011 IEEE Symposium on Artificial Life (ALIFE) https://doi.org/10.1109/ALIFE.2011.5954655	conference	April 2011
Evolutionary reinforcement learning of artificial neural networks Siebel, Nils T.; Sommer, Gerald International Journal of Hybrid Intelligent Systems, Vol. 4, Issue 3 https://doi.org/10.3233/HIS-2007-4304	journal	October 2007
Dynamic Adaptive Neural Network Array Dean, Mark E.; Schuman, Catherine D.; Birdwell, J. Douglas Unconventional Computation and Natural Computation https://doi.org/10.1007/978-3-319-08123-6_11	book	January 2014
An evolutionary optimization framework for neural networks and neuromorphic architectures Schuman, Catherine D.; Plank, James S.; Disney, Adam 2016 International Joint Conference on Neural Networks (IJCNN) https://doi.org/10.1109/IJCNN.2016.7727192	conference	July 2016
Movement prediction from real-world images using a liquid state machine Burgsteiner, Harald; Kröll, Mark; Leopold, Alexander Applied Intelligence, Vol. 26, Issue 2 https://doi.org/10.1007/s10489-006-0007-1	journal	November 2006
Unsupervised learning of digit recognition using spike-timing-dependent plasticity Diehl, Peter U.; Cook, Matthew Frontiers in Computational Neuroscience, Vol. 9 https://doi.org/10.3389/fncom.2015.00099	journal	August 2015
Evolving spiking neural networks for robot control Batllori, R.; Laramee, C. B.; Land, W. Procedia Computer Science, Vol. 6 https://doi.org/10.1016/j.procs.2011.08.060	journal	January 2011
Evolutionary Multi-objective Optimization of Spiking Neural Networks Jin, Yaochu; Wen, Ruojing; Sendhoff, Bernhard Lecture Notes in Computer Science https://doi.org/10.1007/978-3-540-74690-4_38	book	January 2007
Evolving artificial neural networks No authors listed Proceedings of the IEEE, Vol. 87, Issue 9 https://doi.org/10.1109/5.784219	journal	January 1999
Design space exploration and parameter tuning for neuromorphic applications Carlson, Kristofor D.; Nageswaran, Jayram M.; Dutt, Nikil 2013 International Conference on Hardware/Software Codesign and System Synthesis (CODES+ISSS) https://doi.org/10.1109/CODES-ISSS.2013.6659007	conference	September 2013
NeoN: Neuromorphic control for autonomous robotic navigation Mitchell, J. Parker; Bruer, Grant; Dean, Mark E. 2017 IEEE International Symposium on Robotics and Intelligent Sensors (IRIS) https://doi.org/10.1109/IRIS.2017.8250111	conference	October 2017
Networks of spiking neurons: The third generation of neural network models Maass, Wolfgang Neural Networks, Vol. 10, Issue 9 https://doi.org/10.1016/S0893-6080(97)00011-7	journal	December 1997
The Super-Turing Computational Power of Plastic Recurrent Neural Networks Cabessa, JÉRÉMie; Siegelmann, Hava T. International Journal of Neural Systems, Vol. 24, Issue 08 https://doi.org/10.1142/S0129065714500294	journal	November 2014
Spatiotemporal Classification Using Neuroscience-Inspired Dynamic Architectures Schuman, Catherine D.; Birdwell, J. Douglas; Dean, Mark E. Procedia Computer Science, Vol. 41 https://doi.org/10.1016/j.procs.2014.11.089	journal	January 2014
A Unified Hardware/Software Co-Design Framework for Neuromorphic Computing Devices and Applications Plank, James S.; Rose, Garrett S.; Dean, Mark E. 2017 IEEE International Conference on Rebooting Computing (ICRC) https://doi.org/10.1109/ICRC.2017.8123655	conference	November 2017

Similar Records

Resilience and Robustness of Spiking Neural Networks for Neuromorphic Systems

Conference · Wed Jul 01 00:00:00 EDT 2020 · OSTI ID:1671419

Multi-Objective Hyperparameter Optimization for Spiking Neural Network Neuroevolution

Conference · Tue Jun 01 00:00:00 EDT 2021 · OSTI ID:1814329

Multi-Objective Optimization for Size and Resilience of Spiking Neural Networks

Conference · Tue Oct 01 00:00:00 EDT 2019 · OSTI ID:1606935

Understanding Selection And Diversity For Evolution Of Spiking Recurrent Neural Networks

Citation Formats

References (24)

Similar Records

Related Subjects