Computing with Spikes: The Advantage of Fine-Grained Timing
Abstract
Neural-inspired spike-based computing machines often claim to achieve considerable advantages in terms of energy and time efficiency by using spikes for computation and communication. However, fundamental questions about spike-based computation remain unanswered. For instance, how much advantage do spike-based approaches have over conventional methods, and under what circumstances does spike-based computing provide a comparative advantage? Simply implementing existing algorithms using spikes as the medium of computation and communication is not guaranteed to yield an advantage. Here, we demonstrate that spike-based communication and computation within algorithms can increase throughput, and they can decrease energy cost in some cases. We present several spiking algorithms, including sorting a set of numbers in ascending/descending order, as well as finding the maximum or minimum or median of a set of numbers. We also provide an example application: a spiking median-filtering approach for image processing providing a low-energy, parallel implementation. Furthermore, the algorithms and analyses presented here demonstrate that spiking algorithms can provide performance advantages and offer efficient computation of fundamental operations useful in more complex algorithms.
- Authors:
-
- Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
- Publication Date:
- Research Org.:
- Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
- Sponsoring Org.:
- USDOE National Nuclear Security Administration (NNSA)
- OSTI Identifier:
- 1466763
- Report Number(s):
- SAND-2018-9039J
Journal ID: ISSN 0899-7667; 667190
- Grant/Contract Number:
- AC04-94AL85000
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Neural Computation
- Additional Journal Information:
- Journal Volume: 30; Journal Issue: 10; Journal ID: ISSN 0899-7667
- Publisher:
- MIT Press
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 97 MATHEMATICS AND COMPUTING; Spiking Neural Network; Temporal Coding; Parallel Sorting; Sparse Activation; Median-filtering
Citation Formats
Verzi, Stephen J., Rothganger, Fredrick, Parekh, Ojas D., Quach, Tu -Thach, Miner, Nadine E., Vineyard, Craig M., James, Conrad D., and Aimone, James B. Computing with Spikes: The Advantage of Fine-Grained Timing. United States: N. p., 2018.
Web. doi:10.1162/neco_a_01113.
Verzi, Stephen J., Rothganger, Fredrick, Parekh, Ojas D., Quach, Tu -Thach, Miner, Nadine E., Vineyard, Craig M., James, Conrad D., & Aimone, James B. Computing with Spikes: The Advantage of Fine-Grained Timing. United States. https://doi.org/10.1162/neco_a_01113
Verzi, Stephen J., Rothganger, Fredrick, Parekh, Ojas D., Quach, Tu -Thach, Miner, Nadine E., Vineyard, Craig M., James, Conrad D., and Aimone, James B. Wed .
"Computing with Spikes: The Advantage of Fine-Grained Timing". United States. https://doi.org/10.1162/neco_a_01113. https://www.osti.gov/servlets/purl/1466763.
@article{osti_1466763,
title = {Computing with Spikes: The Advantage of Fine-Grained Timing},
author = {Verzi, Stephen J. and Rothganger, Fredrick and Parekh, Ojas D. and Quach, Tu -Thach and Miner, Nadine E. and Vineyard, Craig M. and James, Conrad D. and Aimone, James B.},
abstractNote = {Neural-inspired spike-based computing machines often claim to achieve considerable advantages in terms of energy and time efficiency by using spikes for computation and communication. However, fundamental questions about spike-based computation remain unanswered. For instance, how much advantage do spike-based approaches have over conventional methods, and under what circumstances does spike-based computing provide a comparative advantage? Simply implementing existing algorithms using spikes as the medium of computation and communication is not guaranteed to yield an advantage. Here, we demonstrate that spike-based communication and computation within algorithms can increase throughput, and they can decrease energy cost in some cases. We present several spiking algorithms, including sorting a set of numbers in ascending/descending order, as well as finding the maximum or minimum or median of a set of numbers. We also provide an example application: a spiking median-filtering approach for image processing providing a low-energy, parallel implementation. Furthermore, the algorithms and analyses presented here demonstrate that spiking algorithms can provide performance advantages and offer efficient computation of fundamental operations useful in more complex algorithms.},
doi = {10.1162/neco_a_01113},
journal = {Neural Computation},
number = 10,
volume = 30,
place = {United States},
year = {Wed Jul 18 00:00:00 EDT 2018},
month = {Wed Jul 18 00:00:00 EDT 2018}
}
Free Publicly Available Full Text
Publisher's Version of Record
Other availability
Search WorldCat to find libraries that may hold this journal
Cited by: 10 works
Citation information provided by
Web of Science
Web of Science
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
Adaptive impulse detection using center-weighted median filters
journal, January 2001
- Chen, T.
- IEEE Signal Processing Letters, Vol. 8, Issue 1
Spike-Based Population Coding and Working Memory
journal, February 2011
- Boerlin, Martin; Denève, Sophie
- PLoS Computational Biology, Vol. 7, Issue 2
Pattern recognition computation using action potential timing for stimulus representation
journal, July 1995
- Hopfield, J. J.
- Nature, Vol. 376, Issue 6535
Developing and applying a toolkit from a general neurocomputational framework
journal, June 1999
- Eliasmith, Chris; Anderson, Charles H.
- Neurocomputing, Vol. 26-27
A million spiking-neuron integrated circuit with a scalable communication network and interface
journal, August 2014
- Merolla, P. A.; Arthur, J. V.; Alvarez-Icaza, R.
- Science, Vol. 345, Issue 6197
Role of experience and oscillations in transforming a rate code into a temporal code
journal, June 2002
- Mehta, M. R.; Lee, A. K.; Wilson, M. A.
- Nature, Vol. 417, Issue 6890
Lapicque’s introduction of the integrate-and-fire model neuron (1907)
journal, November 1999
- Abbott, L. F.
- Brain Research Bulletin, Vol. 50, Issue 5-6
Distinct relationships of parietal and prefrontal cortices to evidence accumulation
journal, January 2015
- Hanks, Timothy D.; Kopec, Charles D.; Brunton, Bingni W.
- Nature, Vol. 520, Issue 7546
Dynamics of decision-making: from evidence accumulation to preference and belief
journal, January 2013
- Usher, Marius; Tsetsos, Konstantinos; Yu, Erica C.
- Frontiers in Psychology, Vol. 4
STICK: Spike Time Interval Computational Kernel, a Framework for General Purpose Computation Using Neurons, Precise Timing, Delays, and Synchrony
journal, November 2015
- Lagorce, Xavier; Benosman, Ryad
- Neural Computation, Vol. 27, Issue 11
Routing, merging, and sorting on parallel models of computation
journal, February 1985
- Borodin, A.; Hopcroft, J. E.
- Journal of Computer and System Sciences, Vol. 30, Issue 1
A universal noise removal algorithm with an impulse detector
journal, November 2005
- Garnett, R.; Huegerich, T.; Chui, C.
- IEEE Transactions on Image Processing, Vol. 14, Issue 11
On the Computational Power of Winner-Take-All
journal, November 2000
- Maass, Wolfgang
- Neural Computation, Vol. 12, Issue 11
Computational Account of Spontaneous Activity as a Signature of Predictive Coding
journal, January 2017
- Koren, Veronika; Denève, Sophie
- PLOS Computational Biology, Vol. 13, Issue 1
Predictive Coding of Dynamical Variables in Balanced Spiking Networks
journal, November 2013
- Boerlin, Martin; Machens, Christian K.; Denève, Sophie
- PLoS Computational Biology, Vol. 9, Issue 11
Expressibility and Parallel Complexity
journal, June 1989
- Immerman, Neil
- SIAM Journal on Computing, Vol. 18, Issue 3
Neurogrid: A Mixed-Analog-Digital Multichip System for Large-Scale Neural Simulations
journal, May 2014
- Benjamin, Ben Varkey; Gao, Peiran; McQuinn, Emmett
- Proceedings of the IEEE, Vol. 102, Issue 5
An Improved Median Filtering Algorithm for Image Noise Reduction
journal, January 2012
- Zhu, Youlian; Huang, Cheng
- Physics Procedia, Vol. 25
Improving Spiking Dynamical Networks: Accurate Delays, Higher-Order Synapses, and Time Cells
journal, March 2018
- Voelker, Aaron R.; Eliasmith, Chris
- Neural Computation, Vol. 30, Issue 3
A discrete time neural network model with spiking neurons: Rigorous results on the spontaneous dynamics
journal, September 2007
- Cessac, B.
- Journal of Mathematical Biology, Vol. 56, Issue 3
Belief Propagation in Networks of Spiking Neurons
journal, September 2009
- Steimer, Andreas; Maass, Wolfgang; Douglas, Rodney
- Neural Computation, Vol. 21, Issue 9
Spike-based strategies for rapid processing
journal, July 2001
- Thorpe, Simon; Delorme, Arnaud; Van Rullen, Rufin
- Neural Networks, Vol. 14, Issue 6-7
The median and its extensions
journal, July 2011
- Beliakov, Gleb; Bustince, Humberto; Fernandez, Javier
- Fuzzy Sets and Systems, Vol. 175, Issue 1
A parallel median algorithm
journal, April 1985
- Cole, Richard; Yap, Chee K.
- Information Processing Letters, Vol. 20, Issue 3
The SpiNNaker Project
journal, May 2014
- Furber, Steve B.; Galluppi, Francesco; Temple, Steve
- Proceedings of the IEEE, Vol. 102, Issue 5
Computation with Spikes in a Winner-Take-All Network
journal, September 2009
- Oster, Matthias; Douglas, Rodney; Liu, Shih-Chii
- Neural Computation, Vol. 21, Issue 9
General-Purpose Computation with Neural Networks: A Survey of Complexity Theoretic Results
journal, December 2003
- Šíma, Jiří; Orponen, Pekka
- Neural Computation, Vol. 15, Issue 12
Spike times make sense
journal, January 2005
- VanRullen, Rufin; Guyonneau, Rudy; Thorpe, Simon J.
- Trends in Neurosciences, Vol. 28, Issue 1
O(log*n) algorithms on a Sum-CRCW PRAM
journal, December 2006
- Eisenstat, S. C.
- Computing, Vol. 79, Issue 1
An optimal algorithm for parallel selection
journal, July 1984
- Akl, Selim G.
- Information Processing Letters, Vol. 19, Issue 1
Accumulation of Evidence during Sequential Decision Making: The Importance of Top-Down Factors
journal, January 2010
- de Lange, F. P.; Jensen, O.; Dehaene, S.
- Journal of Neuroscience, Vol. 30, Issue 2
Parallelism in Comparison Problems
journal, September 1975
- Valiant, Leslie G.
- SIAM Journal on Computing, Vol. 4, Issue 3
Convolutional networks for fast, energy-efficient neuromorphic computing
journal, September 2016
- Esser, Steven K.; Merolla, Paul A.; Arthur, John V.
- Proceedings of the National Academy of Sciences, Vol. 113, Issue 41
Contour Enhancement, Short Term Memory, and Constancies in Reverberating Neural Networks
journal, September 1973
- Grossberg, Stephen
- Studies in Applied Mathematics, Vol. 52, Issue 3
Finding the maximum, merging, and sorting in a parallel computation model
journal, March 1981
- Shiloach, Yossi; Vishkin, Uzi
- Journal of Algorithms, Vol. 2, Issue 1
Neural codes: Firing rates and beyond
journal, November 1997
- Gerstner, W.; Kreiter, A. K.; Markram, H.
- Proceedings of the National Academy of Sciences, Vol. 94, Issue 24
Local limit theorems for the maxima of discrete random variables
journal, July 1980
- Anderson, C. W.
- Mathematical Proceedings of the Cambridge Philosophical Society, Vol. 88, Issue 1
A quantitative description of membrane current and its application to conduction and excitation in nerve
journal, August 1952
- Hodgkin, A. L.; Huxley, A. F.
- The Journal of Physiology, Vol. 117, Issue 4
Networks of spiking neurons: The third generation of neural network models
journal, December 1997
- Maass, Wolfgang
- Neural Networks, Vol. 10, Issue 9
How Precise is Neuronal Synchronization?
journal, May 1995
- König, Peter; Engel, Andreas K.; Roelfsema, Pieter R.
- Neural Computation, Vol. 7, Issue 3
A logical calculus of the ideas immanent in nervous activity
journal, December 1943
- McCulloch, Warren S.; Pitts, Walter
- The Bulletin of Mathematical Biophysics, Vol. 5, Issue 4
REVIEW ARTICLE: Neuronal coding and spiking randomness: Neuronal coding and spiking randomness
journal, November 2007
- Kostal, Lubomir; Lansky, Petr; Rospars, Jean-Pierre
- European Journal of Neuroscience, Vol. 26, Issue 10
Bayesian Spiking Neurons I: Inference
journal, January 2008
- Deneve, Sophie
- Neural Computation, Vol. 20, Issue 1
Fine analog coding minimizes information transmission
journal, January 1996
- Softky, William R.
- Neural Networks, Vol. 9, Issue 1
Efficient codes and balanced networks
journal, February 2016
- Denève, Sophie; Machens, Christian K.
- Nature Neuroscience, Vol. 19, Issue 3
Polychronization: Computation with Spikes
journal, February 2006
- Izhikevich, Eugene M.
- Neural Computation, Vol. 18, Issue 2
Rate coding versus temporal order coding: a theoretical approach
journal, November 1998
- Gautrais, Jacques; Thorpe, Simon
- Biosystems, Vol. 48, Issue 1-3
On the Power of some pram Models
journal, January 1999
- Akl, Selim G.; Chen, Lin
- Parallel Algorithms and Applications, Vol. 13, Issue 4
Generalized Integrate-and-Fire Models of Neuronal Activity Approximate Spike Trains of a Detailed Model to a High Degree of Accuracy
journal, August 2004
- Jolivet, Renaud; Lewis, Timothy J.; Gerstner, Wulfram
- Journal of Neurophysiology, Vol. 92, Issue 2
A Large-Scale Model of the Functioning Brain
journal, November 2012
- Eliasmith, C.; Stewart, T. C.; Choo, X.
- Science, Vol. 338, Issue 6111
A spiking neural network architecture for nonlinear function approximation
journal, July 2001
- Iannella, Nicolangelo; Back, Andrew D.
- Neural Networks, Vol. 14, Issue 6-7
Optimization-based computation with spiking neurons
conference, May 2017
- Verzi, Stephen J.; Vineyard, Craig M.; Vugrin, Eric D.
- 2017 International Joint Conference on Neural Networks (IJCNN)
Works referencing / citing this record:
A critical survey of STDP in Spiking Neural Networks for Pattern Recognition
conference, July 2020
- Vigneron, Alex; Martinet, Jean
- 2020 International Joint Conference on Neural Networks (IJCNN)