Computing with Spikes: The Advantage of Fine-Grained Timing

Verzi, Stephen J.; Rothganger, Fredrick; Parekh, Ojas D.; Quach, Tu -Thach; Miner, Nadine E.; Vineyard, Craig M.; James, Conrad D.; Aimone, James B.

doi:10.1162/neco_a_01113

Title: Computing with Spikes: The Advantage of Fine-Grained Timing

Journal Article · Wed Jul 18 00:00:00 EDT 2018 · Neural Computation

DOI:https://doi.org/10.1162/neco_a_01113· OSTI ID:1466763

Verzi, Stephen J. ^[1]; Rothganger, Fredrick ^[1]; Parekh, Ojas D. ^[1]; Quach, Tu -Thach ^[1]; Miner, Nadine E. ^[1]; Vineyard, Craig M. ^[1]; James, Conrad D. ^[1]; Aimone, James B. ^[1]

Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)

Neural-inspired spike-based computing machines often claim to achieve considerable advantages in terms of energy and time efficiency by using spikes for computation and communication. However, fundamental questions about spike-based computation remain unanswered. For instance, how much advantage do spike-based approaches have over conventional methods, and under what circumstances does spike-based computing provide a comparative advantage? Simply implementing existing algorithms using spikes as the medium of computation and communication is not guaranteed to yield an advantage. Here, we demonstrate that spike-based communication and computation within algorithms can increase throughput, and they can decrease energy cost in some cases. We present several spiking algorithms, including sorting a set of numbers in ascending/descending order, as well as finding the maximum or minimum or median of a set of numbers. We also provide an example application: a spiking median-filtering approach for image processing providing a low-energy, parallel implementation. Furthermore, the algorithms and analyses presented here demonstrate that spiking algorithms can provide performance advantages and offer efficient computation of fundamental operations useful in more complex algorithms.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA)

Grant/Contract Number:: AC04-94AL85000

OSTI ID:: 1466763

Report Number(s):: SAND-2018-9039J; 667190

Journal Information:: Neural Computation, Vol. 30, Issue 10; ISSN 0899-7667

Publisher:: MIT PressCopyright Statement

Country of Publication:: United States

Language:: English

Citation Metrics:

Cited by: 10 works

Citation information provided by
Web of Science

References (54)

Adaptive impulse detection using center-weighted median filters Chen, T. IEEE Signal Processing Letters, Vol. 8, Issue 1 https://doi.org/10.1109/97.889633	journal	January 2001
Spike-Based Population Coding and Working Memory Boerlin, Martin; Denève, Sophie PLoS Computational Biology, Vol. 7, Issue 2 https://doi.org/10.1371/journal.pcbi.1001080	journal	February 2011
Pattern recognition computation using action potential timing for stimulus representation Hopfield, J. J. Nature, Vol. 376, Issue 6535 https://doi.org/10.1038/376033a0	journal	July 1995
Developing and applying a toolkit from a general neurocomputational framework Eliasmith, Chris; Anderson, Charles H. Neurocomputing, Vol. 26-27 https://doi.org/10.1016/S0925-2312(99)00098-3	journal	June 1999
A million spiking-neuron integrated circuit with a scalable communication network and interface Merolla, P. A.; Arthur, J. V.; Alvarez-Icaza, R. Science, Vol. 345, Issue 6197 https://doi.org/10.1126/science.1254642	journal	August 2014
Role of experience and oscillations in transforming a rate code into a temporal code Mehta, M. R.; Lee, A. K.; Wilson, M. A. Nature, Vol. 417, Issue 6890 https://doi.org/10.1038/nature00807	journal	June 2002
Lapicque’s introduction of the integrate-and-fire model neuron (1907) Abbott, L. F. Brain Research Bulletin, Vol. 50, Issue 5-6 https://doi.org/10.1016/S0361-9230(99)00161-6	journal	November 1999
Distinct relationships of parietal and prefrontal cortices to evidence accumulation Hanks, Timothy D.; Kopec, Charles D.; Brunton, Bingni W. Nature, Vol. 520, Issue 7546 https://doi.org/10.1038/nature14066	journal	January 2015
Dynamics of decision-making: from evidence accumulation to preference and belief Usher, Marius; Tsetsos, Konstantinos; Yu, Erica C. Frontiers in Psychology, Vol. 4 https://doi.org/10.3389/fpsyg.2013.00758	journal	January 2013
STICK: Spike Time Interval Computational Kernel, a Framework for General Purpose Computation Using Neurons, Precise Timing, Delays, and Synchrony Lagorce, Xavier; Benosman, Ryad Neural Computation, Vol. 27, Issue 11 https://doi.org/10.1162/NECO_a_00783	journal	November 2015
Routing, merging, and sorting on parallel models of computation Borodin, A.; Hopcroft, J. E. Journal of Computer and System Sciences, Vol. 30, Issue 1 https://doi.org/10.1016/0022-0000(85)90008-X	journal	February 1985
A universal noise removal algorithm with an impulse detector Garnett, R.; Huegerich, T.; Chui, C. IEEE Transactions on Image Processing, Vol. 14, Issue 11 https://doi.org/10.1109/TIP.2005.857261	journal	November 2005
On the Computational Power of Winner-Take-All Maass, Wolfgang Neural Computation, Vol. 12, Issue 11 https://doi.org/10.1162/089976600300014827	journal	November 2000
Computational Account of Spontaneous Activity as a Signature of Predictive Coding Koren, Veronika; Denève, Sophie PLOS Computational Biology, Vol. 13, Issue 1 https://doi.org/10.1371/journal.pcbi.1005355	journal	January 2017
Predictive Coding of Dynamical Variables in Balanced Spiking Networks Boerlin, Martin; Machens, Christian K.; Denève, Sophie PLoS Computational Biology, Vol. 9, Issue 11 https://doi.org/10.1371/journal.pcbi.1003258	journal	November 2013
Expressibility and Parallel Complexity Immerman, Neil SIAM Journal on Computing, Vol. 18, Issue 3 https://doi.org/10.1137/0218043	journal	June 1989
Neurogrid: A Mixed-Analog-Digital Multichip System for Large-Scale Neural Simulations Benjamin, Ben Varkey; Gao, Peiran; McQuinn, Emmett Proceedings of the IEEE, Vol. 102, Issue 5 https://doi.org/10.1109/JPROC.2014.2313565	journal	May 2014
An Improved Median Filtering Algorithm for Image Noise Reduction Zhu, Youlian; Huang, Cheng Physics Procedia, Vol. 25 https://doi.org/10.1016/j.phpro.2012.03.133	journal	January 2012
Improving Spiking Dynamical Networks: Accurate Delays, Higher-Order Synapses, and Time Cells Voelker, Aaron R.; Eliasmith, Chris Neural Computation, Vol. 30, Issue 3 https://doi.org/10.1162/neco_a_01046	journal	March 2018
A discrete time neural network model with spiking neurons: Rigorous results on the spontaneous dynamics Cessac, B. Journal of Mathematical Biology, Vol. 56, Issue 3 https://doi.org/10.1007/s00285-007-0117-3	journal	September 2007
Belief Propagation in Networks of Spiking Neurons Steimer, Andreas; Maass, Wolfgang; Douglas, Rodney Neural Computation, Vol. 21, Issue 9 https://doi.org/10.1162/neco.2009.08-08-837	journal	September 2009
Spike-based strategies for rapid processing Thorpe, Simon; Delorme, Arnaud; Van Rullen, Rufin Neural Networks, Vol. 14, Issue 6-7 https://doi.org/10.1016/S0893-6080(01)00083-1	journal	July 2001
The median and its extensions Beliakov, Gleb; Bustince, Humberto; Fernandez, Javier Fuzzy Sets and Systems, Vol. 175, Issue 1 https://doi.org/10.1016/j.fss.2011.01.002	journal	July 2011
A parallel median algorithm Cole, Richard; Yap, Chee K. Information Processing Letters, Vol. 20, Issue 3 https://doi.org/10.1016/0020-0190(85)90080-8	journal	April 1985
The SpiNNaker Project Furber, Steve B.; Galluppi, Francesco; Temple, Steve Proceedings of the IEEE, Vol. 102, Issue 5 https://doi.org/10.1109/JPROC.2014.2304638	journal	May 2014
Computation with Spikes in a Winner-Take-All Network Oster, Matthias; Douglas, Rodney; Liu, Shih-Chii Neural Computation, Vol. 21, Issue 9 https://doi.org/10.1162/neco.2009.07-08-829	journal	September 2009
Computer Recreations Dewdney, A. K. Scientific American, Vol. 250, Issue 6 https://doi.org/10.1038/scientificamerican0684-19	journal	June 1984
General-Purpose Computation with Neural Networks: A Survey of Complexity Theoretic Results Šíma, Jiří; Orponen, Pekka Neural Computation, Vol. 15, Issue 12 https://doi.org/10.1162/089976603322518731	journal	December 2003
Spike times make sense VanRullen, Rufin; Guyonneau, Rudy; Thorpe, Simon J. Trends in Neurosciences, Vol. 28, Issue 1 https://doi.org/10.1016/j.tins.2004.10.010	journal	January 2005
O(log*n) algorithms on a Sum-CRCW PRAM Eisenstat, S. C. Computing, Vol. 79, Issue 1 https://doi.org/10.1007/s00607-006-0214-y	journal	December 2006
An optimal algorithm for parallel selection Akl, Selim G. Information Processing Letters, Vol. 19, Issue 1 https://doi.org/10.1016/0020-0190(84)90128-5	journal	July 1984
Accumulation of Evidence during Sequential Decision Making: The Importance of Top-Down Factors de Lange, F. P.; Jensen, O.; Dehaene, S. Journal of Neuroscience, Vol. 30, Issue 2 https://doi.org/10.1523/JNEUROSCI.4080-09.2010	journal	January 2010
Parallelism in Comparison Problems Valiant, Leslie G. SIAM Journal on Computing, Vol. 4, Issue 3 https://doi.org/10.1137/0204030	journal	September 1975
Convolutional networks for fast, energy-efficient neuromorphic computing Esser, Steven K.; Merolla, Paul A.; Arthur, John V. Proceedings of the National Academy of Sciences, Vol. 113, Issue 41 https://doi.org/10.1073/pnas.1604850113	journal	September 2016
Contour Enhancement, Short Term Memory, and Constancies in Reverberating Neural Networks Grossberg, Stephen Studies in Applied Mathematics, Vol. 52, Issue 3 https://doi.org/10.1002/sapm1973523213	journal	September 1973
Finding the maximum, merging, and sorting in a parallel computation model Shiloach, Yossi; Vishkin, Uzi Journal of Algorithms, Vol. 2, Issue 1 https://doi.org/10.1016/0196-6774(81)90010-9	journal	March 1981
Neural codes: Firing rates and beyond Gerstner, W.; Kreiter, A. K.; Markram, H. Proceedings of the National Academy of Sciences, Vol. 94, Issue 24 https://doi.org/10.1073/pnas.94.24.12740	journal	November 1997
Local limit theorems for the maxima of discrete random variables Anderson, C. W. Mathematical Proceedings of the Cambridge Philosophical Society, Vol. 88, Issue 1 https://doi.org/10.1017/S0305004100057443	journal	July 1980
A quantitative description of membrane current and its application to conduction and excitation in nerve Hodgkin, A. L.; Huxley, A. F. The Journal of Physiology, Vol. 117, Issue 4 https://doi.org/10.1113/jphysiol.1952.sp004764	journal	August 1952
Networks of spiking neurons: The third generation of neural network models Maass, Wolfgang Neural Networks, Vol. 10, Issue 9 https://doi.org/10.1016/S0893-6080(97)00011-7	journal	December 1997
How Precise is Neuronal Synchronization? König, Peter; Engel, Andreas K.; Roelfsema, Pieter R. Neural Computation, Vol. 7, Issue 3 https://doi.org/10.1162/neco.1995.7.3.469	journal	May 1995
A logical calculus of the ideas immanent in nervous activity McCulloch, Warren S.; Pitts, Walter The Bulletin of Mathematical Biophysics, Vol. 5, Issue 4 https://doi.org/10.1007/BF02478259	journal	December 1943
REVIEW ARTICLE: Neuronal coding and spiking randomness: Neuronal coding and spiking randomness Kostal, Lubomir; Lansky, Petr; Rospars, Jean-Pierre European Journal of Neuroscience, Vol. 26, Issue 10 https://doi.org/10.1111/j.1460-9568.2007.05880.x	journal	November 2007
Bayesian Spiking Neurons I: Inference Deneve, Sophie Neural Computation, Vol. 20, Issue 1 https://doi.org/10.1162/neco.2008.20.1.91	journal	January 2008
Fine analog coding minimizes information transmission Softky, William R. Neural Networks, Vol. 9, Issue 1 https://doi.org/10.1016/0893-6080(95)00012-7	journal	January 1996
Efficient codes and balanced networks Denève, Sophie; Machens, Christian K. Nature Neuroscience, Vol. 19, Issue 3 https://doi.org/10.1038/nn.4243	journal	February 2016
Polychronization: Computation with Spikes Izhikevich, Eugene M. Neural Computation, Vol. 18, Issue 2 https://doi.org/10.1162/089976606775093882	journal	February 2006
Rate coding versus temporal order coding: a theoretical approach Gautrais, Jacques; Thorpe, Simon Biosystems, Vol. 48, Issue 1-3 https://doi.org/10.1016/S0303-2647(98)00050-1	journal	November 1998
On the Power of some pram Models Akl, Selim G.; Chen, Lin Parallel Algorithms and Applications, Vol. 13, Issue 4 https://doi.org/10.1080/01495739908947372	journal	January 1999
Generalized Integrate-and-Fire Models of Neuronal Activity Approximate Spike Trains of a Detailed Model to a High Degree of Accuracy Jolivet, Renaud; Lewis, Timothy J.; Gerstner, Wulfram Journal of Neurophysiology, Vol. 92, Issue 2 https://doi.org/10.1152/jn.00190.2004	journal	August 2004
Parallel Merge Sort Cole, Richard SIAM Journal on Computing, Vol. 17, Issue 4 https://doi.org/10.1137/0217049	journal	August 1988
A Large-Scale Model of the Functioning Brain Eliasmith, C.; Stewart, T. C.; Choo, X. Science, Vol. 338, Issue 6111 https://doi.org/10.1126/science.1225266	journal	November 2012
A spiking neural network architecture for nonlinear function approximation Iannella, Nicolangelo; Back, Andrew D. Neural Networks, Vol. 14, Issue 6-7 https://doi.org/10.1016/S0893-6080(01)00080-6	journal	July 2001
Optimization-based computation with spiking neurons Verzi, Stephen J.; Vineyard, Craig M.; Vugrin, Eric D. 2017 International Joint Conference on Neural Networks (IJCNN) https://doi.org/10.1109/IJCNN.2017.7966098	conference	May 2017

Cited By (1)

A critical survey of STDP in Spiking Neural Networks for Pattern Recognition Vigneron, Alex; Martinet, Jean 2020 International Joint Conference on Neural Networks (IJCNN) https://doi.org/10.1109/ijcnn48605.2020.9207239	conference	July 2020

Similar Records

Neuromorphic scaling advantages for energy-efficient random walk computations

Journal Article · Mon Feb 14 00:00:00 EST 2022 · Nature Electronics · OSTI ID:1466763

Smith, J. Darby; Hill, Aaron J.; Reeder, Leah E.; +5 more

Modular Spiking Neural Circuits for Mapping Long Short-Term Memory on a Neurosynaptic Processor

Journal Article · Fri Jul 13 00:00:00 EDT 2018 · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · OSTI ID:1466763

Shrestha, Amar; Ahmed, Khadeer; Wang, Yanzhi; +4 more

DFSynthesizer: Dataflow-based Synthesis of Spiking Neural Networks to Neuromorphic Hardware

Journal Article · Sat May 28 00:00:00 EDT 2022 · ACM Transactions on Embedded Computing Systems · OSTI ID:1466763

Song, Shihao; Chong, Harry; Balaji, Adarsha; +3 more

Related Subjects

97 MATHEMATICS AND COMPUTING
Spiking Neural Network
Temporal Coding
Parallel Sorting
Sparse Activation
Median-filtering

Title: Computing with Spikes: The Advantage of Fine-Grained Timing

Citation Formats

References (54)

Cited By (1)

Similar Records

Related Subjects