A historical survey of algorithms and hardware architectures for neural-inspired and neuromorphic computing applications

James, Conrad D.; Aimone, James B.; Miner, Nadine E.; Vineyard, Craig M.; Rothganger, Fredrick H.; Carlson, Kristofor D.; Mulder, Samuel A.; Draelos, Timothy J.; Faust, Aleksandra; Marinella, Matthew J.; Naegle, John H.; Plimpton, Steven J.

doi:10.1016/j.bica.2016.11.002

Title: A historical survey of algorithms and hardware architectures for neural-inspired and neuromorphic computing applications

Journal Article · Wed Jan 04 00:00:00 EST 2017 · Biologically Inspired Cognitive Architectures

DOI:https://doi.org/10.1016/j.bica.2016.11.002· OSTI ID:1340263

James, Conrad D. ^[1]; Aimone, James B. ^[1]; Miner, Nadine E. ^[1]; Vineyard, Craig M. ^[1]; Rothganger, Fredrick H. ^[1]; Carlson, Kristofor D. ^[1]; Mulder, Samuel A. ^[1]; Draelos, Timothy J. ^[1]; Faust, Aleksandra ^[2]; Marinella, Matthew J. ^[1]; Naegle, John H. ^[1]; Plimpton, Steven J. ^[1]

Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
Sandia National Lab. (SNL-NM), Albuquerque, NM (United States); Google X, Mountain View, CA (United States)

In this study, biological neural networks continue to inspire new developments in algorithms and microelectronic hardware to solve challenging data processing and classification problems. Here in this research, we survey the history of neural-inspired and neuromorphic computing in order to examine the complex and intertwined trajectories of the mathematical theory and hardware developed in this field. Early research focused on adapting existing hardware to emulate the pattern recognition capabilities of living organisms. Contributions from psychologists, mathematicians, engineers, neuroscientists, and other professions were crucial to maturing the field from narrowly-tailored demonstrations to more generalizable systems capable of addressing difficult problem classes such as object detection and speech recognition. Algorithms that leverage fundamental principles found in neuroscience such as hierarchical structure, temporal integration, and robustness to error have been developed, and some of these approaches are achieving world-leading performance on particular data classification tasks. Additionally, novel microelectronic hardware is being developed to perform logic and to serve as memory in neuromorphic computing systems with optimized system integration and improved energy efficiency. Key to such advancements was the incorporation of new discoveries in neuroscience research, the transition away from strict structural replication and towards the functional replication of neural systems, and the use of mathematical theory frameworks to guide algorithm and hardware developments.

View Accepted Manuscript (DOE)

View Accepted Manuscript (Publisher)

Cite

Export

Save

Research Organization:: Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA)

Grant/Contract Number:: AC04-94AL85000

OSTI ID:: 1340263

Alternate ID(s):: OSTI ID: 1397404

Report Number(s):: SAND-2016-12503J; 649832

Journal Information:: Biologically Inspired Cognitive Architectures, Vol. 19, Issue C; ISSN 2212-683X

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

Citation Metrics:

Cited by: 58 works

Citation information provided by
Web of Science

References (157)

Symbolic neural networks for cognitive capacities Achler, Tsvi Biologically Inspired Cognitive Architectures, Vol. 9 https://doi.org/10.1016/j.bica.2014.07.001	journal	July 2014
Energy Scaling Advantages of Resistive Memory Crossbar Based Computation and Its Application to Sparse Coding Agarwal, Sapan; Quach, Tu-Thach; Parekh, Ojas Frontiers in Neuroscience, Vol. 9 https://doi.org/10.3389/fnins.2015.00484	journal	January 2016
Adult neurogenesis: integrating theories and separating functions Aimone, James B.; Deng, Wei; Gage, Fred H. Trends in Cognitive Sciences, Vol. 14, Issue 7 https://doi.org/10.1016/j.tics.2010.04.003	journal	July 2010
Computational Influence of Adult Neurogenesis on Memory Encoding Aimone, James B.; Wiles, Janet; Gage, Fred H. Neuron, Vol. 61, Issue 2 https://doi.org/10.1016/j.neuron.2008.11.026	journal	January 2009
Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning Alipanahi, Babak; Delong, Andrew; Weirauch, Matthew T. Nature Biotechnology, Vol. 33, Issue 8 https://doi.org/10.1038/nbt.3300	journal	July 2015
Memristors Empower Spiking Neurons With Stochasticity Al-Shedivat, Maruan; Naous, Rawan; Cauwenberghs, Gert IEEE Journal on Emerging and Selected Topics in Circuits and Systems, Vol. 5, Issue 2 https://doi.org/10.1109/JETCAS.2015.2435512	journal	June 2015
Statistical mechanics of neural networks near saturation Amit, Daniel J.; Gutfreund, Hanoch; Sompolinsky, H. Annals of Physics, Vol. 173, Issue 1 https://doi.org/10.1016/0003-4916(87)90092-3	journal	January 1987
FPGA implementation of a systems identification module based upon Hopfield networks Atencia, Miguel; Boumeridja, Hafida; Joya, Gonzalo Neurocomputing, Vol. 70, Issue 16-18 https://doi.org/10.1016/j.neucom.2006.06.012	journal	October 2007
A Mechanical Model of the Conditioned Reflex Baernstein, H. D.; Hull, Clark L. The Journal of General Psychology, Vol. 5, Issue 1 https://doi.org/10.1080/00221309.1931.9918381	journal	January 1931
Searching for exotic particles in high-energy physics with deep learning Baldi, P.; Sadowski, P.; Whiteson, D. Nature Communications, Vol. 5, Issue 1 https://doi.org/10.1038/ncomms5308	journal	July 2014
A Comparison of Decision Tree Ensemble Creation Techniques Banfield, Robert E.; Hall, Lawrence O.; Bowyer, Kevin W. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 29, Issue 1 https://doi.org/10.1109/TPAMI.2007.250609	journal	January 2007
Summation and inhibition in the frog's retina Barlow, H. B. The Journal of Physiology, Vol. 119, Issue 1 https://doi.org/10.1113/jphysiol.1953.sp004829	journal	January 1953
Statistical Inference for Probabilistic Functions of Finite State Markov Chains Baum, Leonard E.; Petrie, Ted The Annals of Mathematical Statistics, Vol. 37, Issue 6 https://doi.org/10.1214/aoms/1177699147	journal	December 1966
Nengo: a Python tool for building large-scale functional brain models Bekolay, Trevor; Bergstra, James; Hunsberger, Eric Frontiers in Neuroinformatics, Vol. 7 https://doi.org/10.3389/fninf.2013.00048	journal	January 2014
A Markovian Decision Process Bellman, Richard Indiana University Mathematics Journal, Vol. 6, Issue 4 https://doi.org/10.1512/iumj.1957.6.56038	journal	January 1957
Learning Deep Architectures for AI Bengio, Y. Foundations and Trends® in Machine Learning, Vol. 2, Issue 1 https://doi.org/10.1561/2200000006	journal	January 2009
Neurogrid: A Mixed-Analog-Digital Multichip System for Large-Scale Neural Simulations Benjamin, Ben Varkey; Gao, Peiran; McQuinn, Emmett Proceedings of the IEEE, Vol. 102, Issue 5 https://doi.org/10.1109/JPROC.2014.2313565	journal	May 2014
Support vector machines: hype or hallelujah? Bennett, Kristin P.; Campbell, Colin ACM SIGKDD Explorations Newsletter, Vol. 2, Issue 2 https://doi.org/10.1145/380995.380999	journal	December 2000
Neuromorphic Microchips Boahen, Kwabena Scientific American, Vol. 292, Issue 5 https://doi.org/10.1038/scientificamerican0505-56	journal	May 2005
Millisecond-timescale, genetically targeted optical control of neural activity Boyden, Edward S.; Zhang, Feng; Bamberg, Ernst Nature Neuroscience, Vol. 8, Issue 9 https://doi.org/10.1038/nn1525	journal	August 2005
A New Mechanical “Learner” Bradner, Hugh The Journal of General Psychology, Vol. 17, Issue 2 https://doi.org/10.1080/00221309.1937.9918012	journal	October 1937
Random Forests Breiman, Leo Machine Learning, Vol. 45, Issue 1, p. 5-32 https://doi.org/10.1023/A:1010933404324	journal	January 2001
A Steepest-Ascent Method for Solving Optimum Programming Problems Bryson, A. E.; Denham, W. F. Journal of Applied Mechanics, Vol. 29, Issue 2 https://doi.org/10.1115/1.3640537	journal	June 1962
The Chronnectome: Time-Varying Connectivity Networks as the Next Frontier in fMRI Data Discovery Calhoun, Vince D.; Miller, Robyn; Pearlson, Godfrey Neuron, Vol. 84, Issue 2 https://doi.org/10.1016/j.neuron.2014.10.015	journal	October 2014
An efficient automated parameter tuning framework for spiking neural networks Carlson, Kristofor D.; Nageswaran, Jayram Moorkanikara; Dutt, Nikil Frontiers in Neuroscience, Vol. 8 https://doi.org/10.3389/fnins.2014.00010	journal	January 2014
A vlsi recurrent network of integrate-and-fire neurons connected by plastic synapses with long-term memory Chicca, E.; Badoni, D.; Dante, V. IEEE Transactions on Neural Networks, Vol. 14, Issue 5 https://doi.org/10.1109/TNN.2003.816367	journal	September 2003
Neuromorphic Electronic Circuits for Building Autonomous Cognitive Systems Chicca, Elisabetta; Stefanini, Fabio; Bartolozzi, Chiara Proceedings of the IEEE, Vol. 102, Issue 9 https://doi.org/10.1109/JPROC.2014.2313954	journal	September 2014
Neural population dynamics during reaching Churchland, Mark M.; Cunningham, John P.; Kaufman, Matthew T. Nature, Vol. 487, Issue 7405 https://doi.org/10.1038/nature11129	journal	June 2012
Support-vector networks Cortes, Corinna; Vapnik, Vladimir Machine Learning, Vol. 20, Issue 3 https://doi.org/10.1007/BF00994018	journal	September 1995
Energy-Efficient Neuron, Synapse and STDP Integrated Circuits Cruz-Albrecht, J. M.; Yung, M. W.; Srinivasa, N. IEEE Transactions on Biomedical Circuits and Systems, Vol. 6, Issue 3 https://doi.org/10.1109/TBCAS.2011.2174152	journal	June 2012
Spike Timing-Dependent Plasticity of Neural Circuits Dan, Yang; Poo, Mu-ming Neuron, Vol. 44, Issue 1 https://doi.org/10.1016/j.neuron.2004.09.007	journal	September 2004
Optogenetics: 10 years of microbial opsins in neuroscience Deisseroth, Karl Nature Neuroscience, Vol. 18, Issue 9 https://doi.org/10.1038/nn.4091	journal	August 2015
Silicon retina with correlation-based, velocity-tuned pixels Delbruck, T. IEEE Transactions on Neural Networks, Vol. 4, Issue 3 https://doi.org/10.1109/72.217194	journal	May 1993
An Efficient and Scalable Semiconductor Architecture for Parallel Automata Processing Dlugosch, Paul; Brown, Dave; Glendenning, Paul IEEE Transactions on Parallel and Distributed Systems, Vol. 25, Issue 12 https://doi.org/10.1109/TPDS.2014.8	journal	December 2014
Neuromorphic Analogue VLSI Douglas, R.; Mahowald, M.; Mead, C. Annual Review of Neuroscience, Vol. 18, Issue 1 https://doi.org/10.1146/annurev.ne.18.030195.001351	journal	March 1995
Dopaminergic inputs in the dentate gyrus direct the choice of memory encoding Du, Huiyun; Deng, Wei; Aimone, James B. Proceedings of the National Academy of Sciences, Vol. 113, Issue 37 https://doi.org/10.1073/pnas.1606951113	journal	August 2016
A Large-Scale Model of the Functioning Brain Eliasmith, C.; Stewart, T. C.; Choo, X. Science, Vol. 338, Issue 6111 https://doi.org/10.1126/science.1225266	journal	November 2012
NeuFlow: A runtime reconfigurable dataflow processor for vision Farabet, Clement; Martini, Berin; Corda, Benoit 2011 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPR Workshops), CVPR 2011 WORKSHOPS https://doi.org/10.1109/CVPRW.2011.5981829	conference	June 2011
Simulation of self-organizing systems by digital computer Farley, B.; Clark, W. Transactions of the IRE Professional Group on Information Theory, Vol. 4, Issue 4 https://doi.org/10.1109/TIT.1954.1057468	journal	September 1954
Synaptic Mechanisms for Plasticity in Neocortex Feldman, Daniel E. Annual Review of Neuroscience, Vol. 32, Issue 1 https://doi.org/10.1146/annurev.neuro.051508.135516	journal	June 2009
Distributed Hierarchical Processing in the Primate Cerebral Cortex Felleman, D. J.; Van Essen, D. C. Cerebral Cortex, Vol. 1, Issue 1 https://doi.org/10.1093/cercor/1.1.1	journal	January 1991
Stimulus correlates of visual pattern recognition: a probability approach. Fitts, Paul M.; Weinstein, Meyer; Rappaport, Maurice Journal of Experimental Psychology, Vol. 51, Issue 1 https://doi.org/10.1037/h0044302	journal	January 1956
Pattern recognition in the presence of visual noise. French, Robert Stanton Journal of Experimental Psychology, Vol. 47, Issue 1 https://doi.org/10.1037/h0058298	journal	January 1954
Cognitron: A self-organizing multilayered neural network Fukushima, Kunihiko Biological Cybernetics, Vol. 20, Issue 3-4 https://doi.org/10.1007/BF00342633	journal	January 1975
Neocognitron: A hierarchical neural network capable of visual pattern recognition Fukushima, Kunihiko Neural Networks, Vol. 1, Issue 2 https://doi.org/10.1016/0893-6080(88)90014-7	journal	January 1988
The SpiNNaker Project Furber, Steve B.; Galluppi, Francesco; Temple, Steve Proceedings of the IEEE, Vol. 102, Issue 5 https://doi.org/10.1109/JPROC.2014.2304638	journal	May 2014
Neurophysiology of a VLSI spiking neural network: LANN21 Fusi, S.; Del Giudice, P.; Amit, D. J. Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium https://doi.org/10.1109/IJCNN.2000.861291	conference	January 2000
The grand challenge of computer Go: Monte Carlo tree search and extensions Gelly, Sylvain; Kocsis, Levente; Schoenauer, Marc Communications of the ACM, Vol. 55, Issue 3 https://doi.org/10.1145/2093548.2093574	journal	March 2012
Kerneltron: support vector "machine" in silicon Genov, R.; Cauwenberghs, G. IEEE Transactions on Neural Networks, Vol. 14, Issue 5 https://doi.org/10.1109/TNN.2003.816345	journal	September 2003
NEST (NEural Simulation Tool) Gewaltig, Marc-Oliver; Diesmann, Markus Scholarpedia, Vol. 2, Issue 4 https://doi.org/10.4249/scholarpedia.1430	journal	January 2007
NeuroML: A Language for Describing Data Driven Models of Neurons and Networks with a High Degree of Biological Detail Gleeson, Padraig; Crook, Sharon; Cannon, Robert C. PLoS Computational Biology, Vol. 6, Issue 6 https://doi.org/10.1371/journal.pcbi.1000815	journal	June 2010
A Survey of Bio-Inspired and Other Alternative Architectures Hammerstrom, Dan Nanotechnology https://doi.org/10.1002/9783527628155.nanotech045	book	January 2010
Finding a roadmap to achieve large neuromorphic hardware systems Hasler, Jennifer; Marr, Bo Frontiers in Neuroscience, Vol. 7 https://doi.org/10.3389/fnins.2013.00118	journal	January 2013
Cell-Type-Based Analysis of MicroRNA Profiles in the Mouse Brain He, Miao; Liu, Yu; Wang, Xiaowo Neuron, Vol. 73, Issue 1 https://doi.org/10.1016/j.neuron.2011.11.010	journal	January 2012
Reducing the Dimensionality of Data with Neural Networks Hinton, G. E. Science, Vol. 313, Issue 5786 https://doi.org/10.1126/science.1127647	journal	July 2006
The random subspace method for constructing decision forests IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 20, Issue 8 https://doi.org/10.1109/34.709601	journal	January 1998
Neural networks and physical systems with emergent collective computational abilities. Hopfield, J. J. Proceedings of the National Academy of Sciences, Vol. 79, Issue 8 https://doi.org/10.1073/pnas.79.8.2554	journal	April 1982
Neurons with graded response have collective computational properties like those of two-state neurons. Hopfield, J. J. Proceedings of the National Academy of Sciences, Vol. 81, Issue 10 https://doi.org/10.1073/pnas.81.10.3088	journal	May 1984
Fast-spiking, parvalbumin+ GABAergic interneurons: From cellular design to microcircuit function Hu, H.; Gan, J.; Jonas, P. Science, Vol. 345, Issue 6196 https://doi.org/10.1126/science.1255263	journal	July 2014
Receptive fields of single neurones in the cat's striate cortex Hubel, D. H.; Wiesel, T. N. The Journal of Physiology, Vol. 148, Issue 3 https://doi.org/10.1113/jphysiol.1959.sp006308	journal	October 1959
Integration of nanoscale memristor synapses in neuromorphic computing architectures Indiveri, Giacomo; Linares-Barranco, Bernabé; Legenstein, Robert Nanotechnology, Vol. 24, Issue 38 https://doi.org/10.1088/0957-4484/24/38/384010	journal	September 2013
Neuromorphic Silicon Neuron Circuits Indiveri, Giacomo; Linares-Barranco, Bernabé; Hamilton, Tara Julia Frontiers in Neuroscience, Vol. 5 https://doi.org/10.3389/fnins.2011.00073	journal	January 2011
The NIH BRAIN Initiative Insel, T. R.; Landis, S. C.; Collins, F. S. Science, Vol. 340, Issue 6133 https://doi.org/10.1126/science.1239276	journal	May 2013
Solving the Distal Reward Problem through Linkage of STDP and Dopamine Signaling Izhikevich, E. M. Cerebral Cortex, Vol. 17, Issue 10 https://doi.org/10.1093/cercor/bhl152	journal	January 2007
Large-scale model of mammalian thalamocortical systems Izhikevich, E. M.; Edelman, G. M. Proceedings of the National Academy of Sciences, Vol. 105, Issue 9 https://doi.org/10.1073/pnas.0712231105	journal	February 2008
Nanoscale electronic synapses using phase change devices Jackson, Bryan L.; Padilla, Alvaro; Schrott, Alex G. ACM Journal on Emerging Technologies in Computing Systems, Vol. 9, Issue 2 https://doi.org/10.1145/2463585.2463588	journal	May 2013
Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication Jaeger, H. Science, Vol. 304, Issue 5667 https://doi.org/10.1126/science.1091277	journal	April 2004
Nanoscale Memristor Device as Synapse in Neuromorphic Systems Jo, Sung Hyun; Chang, Ting; Ebong, Idongesit Nano Letters, Vol. 10, Issue 4, p. 1297-1301 https://doi.org/10.1021/nl904092h	journal	April 2010
Computer science: The learning machines Jones, Nicola Nature, Vol. 505, Issue 7482 https://doi.org/10.1038/505146a	journal	January 2014
Reinforcement Learning: A Survey Kaelbling, L. P.; Littman, M. L.; Moore, A. W. Journal of Artificial Intelligence Research, Vol. 4 https://doi.org/10.1613/jair.301	journal	January 1996
An analog vlsi chip emulating sustained and transient response channels of the vertebrate retina Kameda, S.; Yagi, T. IEEE Transactions on Neural Networks, Vol. 14, Issue 5 https://doi.org/10.1109/TNN.2003.816343	journal	September 2003
Ferroelectric Artificial Synapses for Recognition of a Multishaded Image Kaneko, Yukihiro; Nishitani, Yu; Ueda, Michihito IEEE Transactions on Electron Devices, Vol. 61, Issue 8 https://doi.org/10.1109/TED.2014.2331707	journal	August 2014
A hardware system for emulating the early vision utilizing a silicon retina and SpiNNaker chips Kawasetsu, Takumi; Ishida, Ryoya; Sanada, Tadashi 2014 IEEE Biomedical Circuits and Systems Conference (BioCAS) Proceedings https://doi.org/10.1109/BioCAS.2014.6981785	conference	October 2014
Gradient Theory of Optimal Flight Paths Kelley, Henry J. ARS Journal, Vol. 30, Issue 10 https://doi.org/10.2514/8.5282	journal	October 1960
A new spin on magnetic memories Kent, Andrew D.; Worledge, Daniel C. Nature Nanotechnology, Vol. 10, Issue 3 https://doi.org/10.1038/nnano.2015.24	journal	March 2015
Lower-current and fast switching of a perpendicular TMR for high speed and high density spin-transfer-torque MRAM Kishi, T.; Yoda, H.; Kai, T. 2008 IEEE International Electron Devices Meeting (IEDM) https://doi.org/10.1109/IEDM.2008.4796680	conference	December 2008
Nonvolatile memory based on solid electrolytes Kozicki, M. N.; Gopalan, C.; Balakrishnan, M. 2004 Non-Volatile Memory Technology Symposium, Proceedings. 2004 IEEE Computational Systems Bioinformatics Conference https://doi.org/10.1109/NVMT.2004.1380792	conference	January 2004
Synaptic electronics: materials, devices and applications Kuzum, Duygu; Yu, Shimeng; Philip Wong, H-S Nanotechnology, Vol. 24, Issue 38 https://doi.org/10.1088/0957-4484/24/38/382001	journal	September 2013
Human-level concept learning through probabilistic program induction Lake, B. M.; Salakhutdinov, R.; Tenenbaum, J. B. Science, Vol. 350, Issue 6266 https://doi.org/10.1126/science.aab3050	journal	December 2015
Backpropagation Applied to Handwritten Zip Code Recognition LeCun, Y.; Boser, B.; Denker, J. S. Neural Computation, Vol. 1, Issue 4 https://doi.org/10.1162/neco.1989.1.4.541	journal	December 1989
Gradient-based learning applied to document recognition Lecun, Y.; Bottou, L.; Bengio, Y. Proceedings of the IEEE, Vol. 86, Issue 11 https://doi.org/10.1109/5.726791	journal	January 1998
A fast, high-endurance and scalable non-volatile memory device made from asymmetric Ta₂O_5−x/TaO_2−x bilayer structures Lee, Myoung-Jae; Lee, Chang Bum; Lee, Dongsoo Nature Materials, Vol. 10, Issue 8, p. 625-630 https://doi.org/10.1038/nmat3070	journal	July 2011
An analog electronic cochlea Lyon, R. F.; Mead, C. IEEE Transactions on Acoustics, Speech, and Signal Processing, Vol. 36, Issue 7 https://doi.org/10.1109/29.1639	journal	July 1988
Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations Maass, Wolfgang; Natschläger, Thomas; Markram, Henry Neural Computation, Vol. 14, Issue 11 https://doi.org/10.1162/089976602760407955	journal	November 2002
Characterizing cancer subtypes as attractors of Hopfield networks Maetschke, Stefan R.; Ragan, Mark A. Bioinformatics, Vol. 30, Issue 9 https://doi.org/10.1093/bioinformatics/btt773	journal	January 2014
Memristive and neuromorphic behavior in a LixCoO2 nanobattery Mai, V. H.; Moradpour, A.; Senzier, P. Auban Scientific Reports, Vol. 5, Issue 1 https://doi.org/10.1038/srep07761	journal	January 2015
Novel synaptic memory device for neuromorphic computing Mandal, Saptarshi; El-Amin, Ammaarah; Alexander, Kaitlyn Scientific Reports, Vol. 4, Issue 1 https://doi.org/10.1038/srep05333	journal	June 2014
The Blue Brain Project Markram, Henry Nature Reviews Neuroscience, Vol. 7, Issue 2 https://doi.org/10.1038/nrn1848	journal	February 2006
The Human Brain Project Markram, Henry Scientific American, Vol. 306, Issue 6 https://doi.org/10.1038/scientificamerican0612-50	journal	May 2012
Reconstruction and Simulation of Neocortical Microcircuitry Markram, Henry; Muller, Eilif; Ramaswamy, Srikanth Cell, Vol. 163, Issue 2 https://doi.org/10.1016/j.cell.2015.09.029	journal	October 2015
A logical calculus of the ideas immanent in nervous activity McCulloch, Warren S.; Pitts, Walter The Bulletin of Mathematical Biophysics, Vol. 5, Issue 4 https://doi.org/10.1007/BF02478259	journal	December 1943
A silicon model of early visual processing Mead, Carver A.; Mahowald, M. A. Neural Networks, Vol. 1, Issue 1 https://doi.org/10.1016/0893-6080(88)90024-X	journal	January 1988
A million spiking-neuron integrated circuit with a scalable communication network and interface Merolla, P. A.; Arthur, J. V.; Alvarez-Icaza, R. Science, Vol. 345, Issue 6197 https://doi.org/10.1126/science.1254642	journal	August 2014
A digital neurosynaptic core using embedded crossbar memory with 45pJ per spike in 45nm Merolla, Paul; Arthur, John; Akopyan, Filipp 2011 IEEE Custom Integrated Circuits Conference - CICC 2011, 2011 IEEE Custom Integrated Circuits Conference (CICC) https://doi.org/10.1109/CICC.2011.6055294	conference	September 2011
Isothermal Switching and Detailed Filament Evolution in Memristive Systems Mickel, Patrick R.; Lohn, Andrew J.; James, Conrad D. Advanced Materials, Vol. 26, Issue 26, p. 4486-4490 https://doi.org/10.1002/adma.201306182	journal	April 2014
Real-Time Classification of Complex Patterns Using Spike-Based Learning in Neuromorphic VLSI Mitra, Srinjoy; Fusi, Stefano; Indiveri, Giacomo IEEE Transactions on Biomedical Circuits and Systems, Vol. 3, Issue 1 https://doi.org/10.1109/TBCAS.2008.2005781	journal	February 2009
Human-level control through deep reinforcement learning Mnih, Volodymyr; Kavukcuoglu, Koray; Silver, David Nature, Vol. 518, Issue 7540 https://doi.org/10.1038/nature14236	journal	February 2015
Computational roles for dopamine in behavioural control Montague, P. Read; Hyman, Steven E.; Cohen, Jonathan D. Nature, Vol. 431, Issue 7010 https://doi.org/10.1038/nature03015	journal	October 2004
A configurable simulation environment for the efficient simulation of large-scale spiking neural networks on graphics processors Nageswaran, Jayram Moorkanikara; Dutt, Nikil; Krichmar, Jeffrey L. Neural Networks, Vol. 22, Issue 5-6 https://doi.org/10.1016/j.neunet.2009.06.028	journal	July 2009
Synthesizing cognition in neuromorphic electronic systems Neftci, E.; Binas, J.; Rutishauser, U. Proceedings of the National Academy of Sciences, Vol. 110, Issue 37 https://doi.org/10.1073/pnas.1212083110	journal	July 2013
The extracellular patch clamp: A method for resolving currents through individual open channels in biological membranes Neher, Erwin; Sakmann, Bert; Steinbach, Joe Henry Pfl�gers Archiv European Journal of Physiology, Vol. 375, Issue 2 https://doi.org/10.1007/BF00584247	journal	July 1978
Parallel implementation of a spiking neuronal network model of unsupervised olfactory learning on NVidia^® CUDA™ Nowotny, Thomas The 2010 International Joint Conference on Neural Networks (IJCNN) https://doi.org/10.1109/IJCNN.2010.5596358	conference	July 2010
Place units in the hippocampus of the freely moving rat O'Keefe, John Experimental Neurology, Vol. 51, Issue 1 https://doi.org/10.1016/0014-4886(76)90055-8	journal	January 1976
Phase relationship between hippocampal place units and the EEG theta rhythm O'Keefe, John; Recce, Michael L. Hippocampus, Vol. 3, Issue 3 https://doi.org/10.1002/hipo.450030307	journal	July 1993
Real-Time Emulator for Reproducing Graded Potentials in Vertebrate Retina Okuno, Hirotsugu; Hasegawa, Jun; Sanada, Tadashi IEEE Transactions on Biomedical Circuits and Systems, Vol. 9, Issue 2 https://doi.org/10.1109/TBCAS.2014.2327103	journal	April 2015
Simultaneous all-optical manipulation and recording of neural circuit activity with cellular resolution in vivo Packer, Adam M.; Russell, Lloyd E.; Dalgleish, Henry W. P. Nature Methods, Vol. 12, Issue 2 https://doi.org/10.1038/nmeth.3217	journal	December 2014
Image restoration using a modified Hopfield network Paik, J. K.; Katsaggelos, A. K. IEEE Transactions on Image Processing, Vol. 1, Issue 1 https://doi.org/10.1109/83.128030	journal	January 1992
Optoelectronic Reservoir Computing Paquot, Y.; Duport, F.; Smerieri, A. Scientific Reports, Vol. 2, Issue 1 https://doi.org/10.1038/srep00287	journal	February 2012
Applying hardware-based machine learning to signature-based network intrusion detection Payer, Garrett; McCormick, Chris; Harang, Richard SPIE Sensing Technology + Applications, SPIE Proceedings https://doi.org/10.1117/12.2052548	conference	May 2014
Fusion, propagation, and structuring in belief networks Pearl, Judea Artificial Intelligence, Vol. 29, Issue 3 https://doi.org/10.1016/0004-3702(86)90072-X	journal	September 1986
Training and operation of an integrated neuromorphic network based on metal-oxide memristors Prezioso, M.; Merrikh-Bayat, F.; Hoskins, B. D. Nature, Vol. 521, Issue 7550 https://doi.org/10.1038/nature14441	journal	May 2015
A review and synthesis of the first 20years of PET and fMRI studies of heard speech, spoken language and reading Price, Cathy J. NeuroImage, Vol. 62, Issue 2 https://doi.org/10.1016/j.neuroimage.2012.04.062	journal	August 2012
A reconfigurable on-line learning spiking neuromorphic processor comprising 256 neurons and 128K synapses Qiao, Ning; Mostafa, Hesham; Corradi, Federico Frontiers in Neuroscience, Vol. 9 https://doi.org/10.3389/fnins.2015.00141	journal	April 2015
Induction of decision trees Quinlan, J. R. Machine Learning, Vol. 1, Issue 1, p. 81-106 https://doi.org/10.1007/BF00116251	journal	March 1986
Learning logical definitions from relations Quinlan, J. R. Machine Learning, Vol. 5, Issue 3 https://doi.org/10.1007/BF00117105	journal	August 1990
A tutorial on hidden Markov models and selected applications in speech recognition Rabiner, L. R. Proceedings of the IEEE, Vol. 77, Issue 2 https://doi.org/10.1109/5.18626	journal	January 1989
A biophysically-based neuromorphic model of spike rate- and timing-dependent plasticity Rachmuth, G.; Shouval, H. Z.; Bear, M. F. Proceedings of the National Academy of Sciences, Vol. 108, Issue 49 https://doi.org/10.1073/pnas.1106161108	journal	November 2011
A neuromorphic VLSI design for spike timing and rate based synaptic plasticity Rahimi Azghadi, Mostafa; Al-Sarawi, Said; Abbott, Derek Neural Networks, Vol. 45 https://doi.org/10.1016/j.neunet.2013.03.003	journal	September 2013
Floating Gate Synapses With Spike-Time-Dependent Plasticity Ramakrishnan, Shubha; Hasler, Paul E.; Gordon, Christal IEEE Transactions on Biomedical Circuits and Systems, Vol. 5, Issue 3 https://doi.org/10.1109/TBCAS.2011.2109000	journal	June 2011
Temporally selective contextual encoding in the dentate gyrus of the hippocampus Rangel, L. M.; Alexander, A. S.; Aimone, J. B. Nature Communications, Vol. 5, Issue 1 https://doi.org/10.1038/ncomms4181	journal	February 2014
Phase-change random access memory: A scalable technology Raoux, S.; Burr, G. W.; Breitwisch, M. J. IBM Journal of Research and Development, Vol. 52, Issue 4.5 https://doi.org/10.1147/rd.524.0465	journal	July 2008
The perceptron: A probabilistic model for information storage and organization in the brain. Rosenblatt, F. Psychological Review, Vol. 65, Issue 6 https://doi.org/10.1037/h0042519	journal	January 1958
N2A: a computational tool for modeling from neurons to algorithms Rothganger, Fredrick; Warrender, Christina E.; Trumbo, Derek Frontiers in Neural Circuits, Vol. 8 https://doi.org/10.3389/fncir.2014.00001	journal	January 2014
Learning representations by back-propagating errors Rumelhart, David E.; Hinton, Geoffrey E.; Williams, Ronald J. Nature, Vol. 323, Issue 6088 https://doi.org/10.1038/323533a0	journal	October 1986
Plasticity in memristive devices for spiking neural networks Saïghi, Sylvain; Mayr, Christian G.; Serrano-Gotarredona, Teresa Frontiers in Neuroscience, Vol. 9 https://doi.org/10.3389/fnins.2015.00051	journal	March 2015
A wafer-scale neuromorphic hardware system for large-scale neural modeling Schemmel, Johannes; Briiderle, Daniel; Griibl, Andreas 2010 IEEE International Symposium on Circuits and Systems - ISCAS 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems https://doi.org/10.1109/ISCAS.2010.5536970	conference	May 2010
Deep learning in neural networks: An overview Schmidhuber, Jürgen Neural Networks, Vol. 61 https://doi.org/10.1016/j.neunet.2014.09.003	journal	January 2015
ConvNets experiments on SpiNNaker Serrano-Gotarredona, T.; Linares-Barranco, B.; Galluppi, F. 2015 IEEE International Symposium on Circuits and Systems (ISCAS) https://doi.org/10.1109/ISCAS.2015.7169169	conference	May 2015
Non-volatile memory as hardware synapse in neuromorphic computing: A first look at reliability issues Shelby, Robert M.; Burr, Geoffrey W.; Boybat, Irem 2015 IEEE International Reliability Physics Symposium (IRPS) https://doi.org/10.1109/IRPS.2015.7112755	conference	April 2015
Mastering the game of Go with deep neural networks and tree search Silver, David; Huang, Aja; Maddison, Chris J. Nature, Vol. 529, Issue 7587 https://doi.org/10.1038/nature16961	journal	January 2016
The Rate of Establishment of a Discrimination Skinner, B. F. The Journal of General Psychology, Vol. 9, Issue 2 https://doi.org/10.1080/00221309.1933.9920939	journal	October 1933
Parvalbumin neurons and gamma rhythms enhance cortical circuit performance Sohal, Vikaas S.; Zhang, Feng; Yizhar, Ofer Nature, Vol. 459, Issue 7247 https://doi.org/10.1038/nature07991	journal	April 2009
Deep learning for real-time robust facial expression recognition on a smartphone Song, Inchul; Kim, Hyun-Jun; Jeon, Paul Barom 2014 IEEE International Conference on Consumer Electronics (ICCE) https://doi.org/10.1109/ICCE.2014.6776135	conference	January 2014
PyNCS: a microkernel for high-level definition and configuration of neuromorphic electronic systems Stefanini, Fabio; Neftci, Emre O.; Sheik, Sadique Frontiers in Neuroinformatics, Vol. 8 https://doi.org/10.3389/fninf.2014.00073	journal	August 2014
How advances in neural recording affect data analysis Stevenson, Ian H.; Kording, Konrad P. Nature Neuroscience, Vol. 14, Issue 2 https://doi.org/10.1038/nn.2731	journal	January 2011
Large-Scale Synthesis of Functional Spiking Neural Circuits Stewart, Terrence C.; Eliasmith, Chris Proceedings of the IEEE, Vol. 102, Issue 5 https://doi.org/10.1109/JPROC.2014.2306061	journal	May 2014
The missing memristor found Strukov, Dmitri B.; Snider, Gregory S.; Stewart, Duncan R. Nature, Vol. 453, Issue 7191 https://doi.org/10.1038/nature06932	journal	May 2008
Modeling meta-cognition in a cognitive architecture Sun, Ron; Zhang, Xi; Mathews, Robert Cognitive Systems Research, Vol. 7, Issue 4 https://doi.org/10.1016/j.cogsys.2005.09.001	journal	December 2006
A Real-time Scalable Object Detection System using Low-power HOG Accelerator VLSI Takagi, Kenta; Tanaka, Kotaro; Izumi, Shintaro Journal of Signal Processing Systems, Vol. 76, Issue 3 https://doi.org/10.1007/s11265-014-0870-7	journal	March 2014
Modeling otoacoustic emission and hearing threshold fine structures Talmadge, Carrick L.; Tubis, Arnold; Long, Glenis R. The Journal of the Acoustical Society of America, Vol. 104, Issue 3 https://doi.org/10.1121/1.424364	journal	September 1998
Accelerating a Random Forest Classifier: Multi-Core, GP-GPU, or FPGA? Van Essen, Brian; Macaraeg, Chris; Gokhale, Maya 2012 IEEE 20th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM), 2012 IEEE 20th International Symposium on Field-Programmable Custom Computing Machines https://doi.org/10.1109/FCCM.2012.47	conference	April 2012
Experimental demonstration of reservoir computing on a silicon photonics chip Vandoorne, Kristof; Mechet, Pauline; Van Vaerenbergh, Thomas Nature Communications, Vol. 5, Issue 1 https://doi.org/10.1038/ncomms4541	journal	March 2014
An experimental unification of reservoir computing methods Verstraeten, D.; Schrauwen, B.; D’Haene, M. Neural Networks, Vol. 20, Issue 3 https://doi.org/10.1016/j.neunet.2007.04.003	journal	April 2007
Non-invasive optical spectroscopy and imaging of human brain function Villringer, A. Trends in Neurosciences, Vol. 20, Issue 10 https://doi.org/10.1016/S0166-2236(97)01132-6	journal	October 1997
Repeated play of the SVM game as a means of adaptive classification Vineyard, Craig M.; Verzi, Stephen J.; James, Conrad D. 2015 International Joint Conference on Neural Networks (IJCNN) https://doi.org/10.1109/IJCNN.2015.7280729	conference	July 2015
Improved implementation of the silicon cochlea Watts, L.; Kerns, D. A.; Lyon, R. F. IEEE Journal of Solid-State Circuits, Vol. 27, Issue 5 https://doi.org/10.1109/4.133156	journal	May 1992
Highly reliable TaOx ReRAM and direct evidence of redox reaction mechanism Wei, Z.; Kanzawa, Y.; Arita, K. 2008 IEEE International Electron Devices Meeting (IEDM) https://doi.org/10.1109/IEDM.2008.4796676	conference	December 2008
Backpropagation through time: what it does and how to do it Werbos, P. J. Proceedings of the IEEE, Vol. 78, Issue 10 https://doi.org/10.1109/5.58337	journal	January 1990
The digi-neocognitron: a digital neocognitron neural network model for VLSI White, B. A.; Elmasry, M. I. IEEE Transactions on Neural Networks, Vol. 3, Issue 1 https://doi.org/10.1109/72.105419	journal	January 1992
MADALINE RULE II: a training algorithm for neural networks IEEE 1988 International Conference on Neural Networks https://doi.org/10.1109/ICNN.1988.23872	conference	July 1988
Phase Change Memory Wong, H. -S. Philip; Raoux, Simone; Kim, SangBum Proceedings of the IEEE, Vol. 98, Issue 12 https://doi.org/10.1109/JPROC.2010.2070050	journal	December 2010
Memory leads the way to better computing Wong, H. -S. Philip; Salahuddin, Sayeef Nature Nanotechnology, Vol. 10, Issue 3 https://doi.org/10.1038/nnano.2015.29	journal	March 2015
Autonomous target tracking of UAVs based on low-power neural network hardware Yang, Wei; Jin, Zhanpeng; Thiem, Clare SPIE Sensing Technology + Applications, SPIE Proceedings https://doi.org/10.1117/12.2054049	conference	May 2014
High-Performance and Compact Architecture for Regular Expression Matching on FPGA Yang, Yi-Hua; Prasanna, Viktor IEEE Transactions on Computers, Vol. 61, Issue 7 https://doi.org/10.1109/TC.2011.129	journal	July 2012
Plasticity in gray and white: neuroimaging changes in brain structure during learning Zatorre, Robert J.; Fields, R. Douglas; Johansen-Berg, Heidi Nature Neuroscience, Vol. 15, Issue 4 https://doi.org/10.1038/nn.3045	journal	March 2012
Brill tagging on the Micron Automata Processor Zhou, Keira; Fox, Jeffrey J.; Wang, Ke 2015 IEEE International Conference on Semantic Computing (ICSC), Proceedings of the 2015 IEEE 9th International Conference on Semantic Computing (IEEE ICSC 2015) https://doi.org/10.1109/ICOSC.2015.7050812	conference	February 2015
Activity-Dependent Synaptogenesis in the Adult Mammalian Cortex Zito, Karen; Svoboda, Karel Neuron, Vol. 35, Issue 6 https://doi.org/10.1016/S0896-6273(02)00903-0	journal	September 2002

Cited By (8)

Functional Oxides for Photoneuromorphic Engineering: Toward a Solar Brain Pérez‐Tomás, Amador Advanced Materials Interfaces, Vol. 6, Issue 15 https://doi.org/10.1002/admi.201900471	journal	June 2019
Realizing spike-timing dependent plasticity learning rule in Pt/Cu:ZnO/Nb:STO memristors for implementing single spike based denoising autoencoder Suresh, Bharathwaj; Boppidi, Pavan Kumar Reddy; Prabhakar Rao, B. V. V. S. N. Journal of Micromechanics and Microengineering, Vol. 29, Issue 8 https://doi.org/10.1088/1361-6439/ab235f	journal	June 2019
Electrolyte-gated transistors for synaptic electronics, neuromorphic computing, and adaptable biointerfacing Ling, Haifeng; Koutsouras, Dimitrios A.; Kazemzadeh, Setareh Applied Physics Reviews, Vol. 7, Issue 1 https://doi.org/10.1063/1.5122249	journal	March 2020
Neuromorphic Computing with Memristor Crossbar Zhang, Xinjiang; Huang, Anping; Hu, Qi physica status solidi (a), Vol. 215, Issue 13 https://doi.org/10.1002/pssa.201700875	journal	May 2018
Large-Scale Neuromorphic Spiking Array Processors: A Quest to Mimic the Brain Thakur, Chetan Singh; Molin, Jamal Lottier; Cauwenberghs, Gert Frontiers in Neuroscience, Vol. 12 https://doi.org/10.3389/fnins.2018.00891	journal	December 2018
Training deep neural networks for binary communication with the Whetstone method Severa, William; Vineyard, Craig M.; Dellana, Ryan Nature Machine Intelligence, Vol. 1, Issue 2 https://doi.org/10.1038/s42256-018-0015-y	journal	January 2019
Large-Scale Neuromorphic Spiking Array Processors: A Quest to Mimic the Brain Thakur, Chetan S.; Molin, Jamal L.; Cauwenberghs, Gert ETH Zurich https://doi.org/10.3929/ethz-b-000310459	text	January 2018
Large-Scale Neuromorphic Spiking Array Processors: A quest to mimic the brain Thakur, Chetan Singh; Molin, Jamal; Cauwenberghs, Gert arXiv https://doi.org/10.48550/arxiv.1805.08932	preprint	January 2018

Similar Records

Benchmarking the Performance of Neuromorphic and Spiking Neural Network Simulators

Journal Article · Thu Mar 18 00:00:00 EDT 2021 · Neurocomputing · OSTI ID:1340263

Kulkarni, Shruti; Parsa, Maryam; Mitchell, J. Parker; +1 more

Hardware Acceleration of Adaptive Neural Algorithms.

Technical Report · Wed Nov 01 00:00:00 EDT 2017 · OSTI ID:1340263

James, Conrad D.

Will Stochastic Devices Play Nice With Others in Neuromorphic Hardware?: There’s More to a Probabilistic System Than Noisy Devices

Journal Article · Fri Sep 01 00:00:00 EDT 2023 · IEEE Electron Devices Magazine · OSTI ID:1340263

Aimone, James B.; Misra, Shashank

Related Subjects

97 MATHEMATICS AND COMPUTING
neuromorphic computing
algorithms
artificial neural networks
data-driven computing
machine learning
pattern recognition

Title: A historical survey of algorithms and hardware architectures for neural-inspired and neuromorphic computing applications

Citation Formats

References (157)

Cited By (8)

Similar Records

Related Subjects