Multi-Objective Optimization for Size and Resilience of Spiking Neural Networks

Dimovska, Mihaela; Johnston, Travis; Schuman, Catherine; Mitchell, Parker; Potok, Thomas

doi:10.1109/UEMCON47517.2019.8992983

Multi-Objective Optimization for Size and Resilience of Spiking Neural Networks

Conference · Tue Oct 01 04:00:00 EDT 2019

DOI:https://doi.org/10.1109/UEMCON47517.2019.8992983· OSTI ID:1606935

Dimovska, Mihaela ^[1]; ^[2]; ^[2]; Mitchell, Parker ^[2]; ^[2]

University of Minnesota
ORNL

Inspired by the connectivity mechanisms in the brain, neuromorphic computing architectures model Spiking Neural Networks (SNNs) in silicon. As such, neuromorphic architectures are designed and developed with the goal of having small, low power chips that can perform control and machine learning tasks. However, the power consumption of the developed hardware can greatly depend on the size of the network that is being evaluated on the chip. Furthermore, the accuracy of a trained SNN that is evaluated on chip can change due to voltage and current variations in the hardware that perturb the learned weights of the network. While efforts are made on the hardware side to minimize those perturbations, a software based strategy to make the deployed networks more resilient can help further alleviate that issue. In this work, we study Spiking Neural Networks in two neuromorphic architecture implementations with the goal of decreasing their size, while at the same time increasing their resiliency to hardware faults. We leverage an evolutionary algorithm to train the SNNs and propose a multiobjective fitness function to optimize the size and resiliency of the SNN. We demonstrate that this strategy leads to well-performing, small-sized networks that are more resilient to hardware faults.

View Conference

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Organization:: USDOE; USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)

DOE Contract Number:: AC05-00OR22725

OSTI ID:: 1606935

Country of Publication:: United States

Language:: English

References (21)

A Comparison of Neuromorphic Classification Tasks Reynolds, John J. M.; Plank, James S.; Schuman, Catherine D. Proceedings of the International Conference on Neuromorphic Systems - ICONS '18 https://doi.org/10.1145/3229884.3229896	conference	January 2018
The TENNLab Exploratory Neuromorphic Computing Framework Plank, James S.; Schuman, Catherine D.; Bruer, Grant IEEE Letters of the Computer Society, Vol. 1, Issue 2 https://doi.org/10.1109/LOCS.2018.2885976	journal	July 2018
A Novel Pruning Method for Convolutional Neural Networks Based off Identifying Critical Filters Dimovska, Mihaela; Johnston, Travis Proceedings of the Practice and Experience in Advanced Research Computing on Rise of the Machines (learning) https://doi.org/10.1145/3332186.3333057	conference	July 2019
A survey on the industrial readiness for Internet of Things Wang, Xianping; Qiu, Hao; Xie, Fei 2017 IEEE 8th Annual Ubiquitous Computing, Electronics and Mobile Communication Conference (UEMCON) https://doi.org/10.1109/UEMCON.2017.8249015	conference	October 2017
DANNA 2: Dynamic Adaptive Neural Network Arrays Mitchell, J. Parker; Dean, Mark E.; Bruer, Grant R. Proceedings of the International Conference on Neuromorphic Systems - ICONS '18 https://doi.org/10.1145/3229884.3229894	conference	January 2018
Simplifying deep neural networks for neuromorphic architectures Chung, Jaeyong; Shin, Taehwan Proceedings of the 53rd Annual Design Automation Conference https://doi.org/10.1145/2897937.2898092	conference	June 2016
MLPNN Training via a Multiobjective Optimization of Training Error and Stochastic Sensitivity Yeung, Daniel S.; Li, Jin-Cheng; Ng, Wing W. Y. IEEE Transactions on Neural Networks and Learning Systems, Vol. 27, Issue 5 https://doi.org/10.1109/TNNLS.2015.2431251	journal	May 2016
SpiNNaker: A 1-W 18-Core System-on-Chip for Massively-Parallel Neural Network Simulation Painkras, Eustace; Plana, Luis A.; Garside, Jim IEEE Journal of Solid-State Circuits, Vol. 48, Issue 8 https://doi.org/10.1109/JSSC.2013.2259038	journal	August 2013
Challenges and research directions for Internet of Things Hammoudi, Sarra; Aliouat, Zibouda; Harous, Saad Telecommunication Systems, Vol. 67, Issue 2 https://doi.org/10.1007/s11235-017-0343-y	journal	July 2017
Neuroscience-inspired inspired dynamic architectures Schuman, Catherine D.; Birdwell, J. Douglas; Dean, Mark Proceedings of the 2014 Biomedical Sciences and Engineering Conference https://doi.org/10.1109/BSEC.2014.6867735	conference	May 2014
Understanding Synapses: Past, Present, and Future Südhof, Thomas C.; Malenka, Robert C. Neuron, Vol. 60, Issue 3 https://doi.org/10.1016/j.neuron.2008.10.011	journal	November 2008
Homeostatic Fault Tolerance in Spiking Neural Networks: A Dynamic Hardware Perspective Johnson, Anju P.; Liu, Junxiu; Millard, Alan G. IEEE Transactions on Circuits and Systems I: Regular Papers, Vol. 65, Issue 2 https://doi.org/10.1109/TCSI.2017.2726763	journal	February 2018
IoT technologies for embedded computing Samie, Farzad; Bauer, Lars; Henkel, Jörg Proceedings of the Eleventh IEEE/ACM/IFIP International Conference on Hardware/Software Codesign and System Synthesis https://doi.org/10.1145/2968456.2974004	conference	October 2016
An evolutionary optimization framework for neural networks and neuromorphic architectures Schuman, Catherine D.; Plank, James S.; Disney, Adam 2016 International Joint Conference on Neural Networks (IJCNN) https://doi.org/10.1109/IJCNN.2016.7727192	conference	July 2016
Exploiting data representation for fault tolerance Elliott, J.; Hoemmen, M.; Mueller, F. Journal of Computational Science, Vol. 14 https://doi.org/10.1016/j.jocs.2015.12.002	journal	May 2016
Loihi: A Neuromorphic Manycore Processor with On-Chip Learning Davies, Mike; Srinivasa, Narayan; Lin, Tsung-Han IEEE Micro, Vol. 38, Issue 1 https://doi.org/10.1109/MM.2018.112130359	journal	January 2018
Internet of Things and Body area network-an integrated future Chatterjee, Subhajit; Chatterjee, Shreya; Choudhury, Soumyadeep 2017 IEEE 8th Annual Ubiquitous Computing, Electronics and Mobile Communication Conference (UEMCON) https://doi.org/10.1109/UEMCON.2017.8249094	conference	October 2017
Neurogrid: A Mixed-Analog-Digital Multichip System for Large-Scale Neural Simulations Benjamin, Ben Varkey; Gao, Peiran; McQuinn, Emmett Proceedings of the IEEE, Vol. 102, Issue 5 https://doi.org/10.1109/JPROC.2014.2313565	journal	May 2014
ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression Luo, Jian-Hao; Wu, Jianxin; Lin, Weiyao 2017 IEEE International Conference on Computer Vision (ICCV) https://doi.org/10.1109/ICCV.2017.541	conference	October 2017
Training Deep Spiking Neural Networks Using Backpropagation Lee, Jun Haeng; Delbruck, Tobi; Pfeiffer, Michael Frontiers in Neuroscience, Vol. 10 https://doi.org/10.3389/fnins.2016.00508	journal	November 2016
A million spiking-neuron integrated circuit with a scalable communication network and interface Merolla, P. A.; Arthur, J. V.; Alvarez-Icaza, R. Science, Vol. 345, Issue 6197 https://doi.org/10.1126/science.1254642	journal	August 2014

Similar Records

Resilience and Robustness of Spiking Neural Networks for Neuromorphic Systems

Conference · Wed Jul 01 00:00:00 EDT 2020 · OSTI ID:1671419

Multi-Objective Hyperparameter Optimization for Spiking Neural Network Neuroevolution

Conference · Tue Jun 01 00:00:00 EDT 2021 · OSTI ID:1814329

Training Spiking Neural Networks Using Combined Learning Approaches

Conference · Mon Nov 30 23:00:00 EST 2020 · OSTI ID:1760122

Multi-Objective Optimization for Size and Resilience of Spiking Neural Networks

Citation Formats

References (21)

Similar Records

Related Subjects