The capabilities of natural neural systems have inspired both new generations of machine learning algorithms as well as neuromorphic, very large-scale integrated circuits capable of fast, low-power information processing. However, it has been argued that most modern machine learning algorithms are not neurophysiologically plausible. In particular, the workhorse of modern deep learning, the backpropagation algorithm, has proven difficult to translate to neuromorphic hardware. This study presents a neuromorphic, spiking backpropagation algorithm based on synfire-gated dynamical information coordination and processing implemented on Intel’s Loihi neuromorphic research processor. We demonstrate a proof-of-principle three-layer circuit that learns to classify digits and clothing items from the MNIST and Fashion MNIST datasets. To our knowledge, this is the first work to show a Spiking Neural Network implementation of the exact backpropagation algorithm that is fully on-chip without a computer in the loop. It is competitive in accuracy with off-chip trained SNNs and achieves an energy-delay product suitable for edge computing. This implementation shows a path for using in-memory, massively parallel neuromorphic processors for low-power, low-latency implementation of modern deep learning applications.
@article{osti_2476747,
author = {Renner, Alpha Felix Merlin Victor and Sheldon, Forrest Crawford and Zlotnik, Anatoly V. and Tao, Louis and Sornborger, Andrew Tyler},
title = {The backpropagation algorithm implemented on spiking neuromorphic hardware},
annote = {The capabilities of natural neural systems have inspired both new generations of machine learning algorithms as well as neuromorphic, very large-scale integrated circuits capable of fast, low-power information processing. However, it has been argued that most modern machine learning algorithms are not neurophysiologically plausible. In particular, the workhorse of modern deep learning, the backpropagation algorithm, has proven difficult to translate to neuromorphic hardware. This study presents a neuromorphic, spiking backpropagation algorithm based on synfire-gated dynamical information coordination and processing implemented on Intel’s Loihi neuromorphic research processor. We demonstrate a proof-of-principle three-layer circuit that learns to classify digits and clothing items from the MNIST and Fashion MNIST datasets. To our knowledge, this is the first work to show a Spiking Neural Network implementation of the exact backpropagation algorithm that is fully on-chip without a computer in the loop. It is competitive in accuracy with off-chip trained SNNs and achieves an energy-delay product suitable for edge computing. This implementation shows a path for using in-memory, massively parallel neuromorphic processors for low-power, low-latency implementation of modern deep learning applications.},
doi = {10.1038/s41467-024-53827-9},
url = {https://www.osti.gov/biblio/2476747},
journal = {Nature Communications},
issn = {ISSN 2041-1723},
number = {1},
volume = {15},
place = {United States},
publisher = {Nature Publishing Group},
year = {2024},
month = {11}}
Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
Sponsoring Organization:
Natural Science Foundation of China; Swiss National Science Foundation (SNSF); USDOE Laboratory Directed Research and Development (LDRD) Program; USDOE National Nuclear Security Administration (NNSA); University of Zurich
Schemmel, Johannes; Briiderle, Daniel; Griibl, Andreas
2010 IEEE International Symposium on Circuits and Systems - ISCAS 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systemshttps://doi.org/10.1109/ISCAS.2010.5536970