Neuromorphic scaling advantages for energy-efficient random walk computations

Smith, J. Darby; Hill, Aaron J.; Reeder, Leah E.; Franke, Brian C.; Lehoucq, Richard B.; Parekh, Ojas; Severa, William; Aimone, James B.

doi:10.1038/s41928-021-00705-7

Neuromorphic scaling advantages for energy-efficient random walk computations

Journal Article · Mon Feb 14 00:00:00 EST 2022 · Nature Electronics

DOI:https://doi.org/10.1038/s41928-021-00705-7· OSTI ID:1845387

^[1]; Hill, Aaron J. ^[1]; Reeder, Leah E. ^[1]; Franke, Brian C. ^[1]; Lehoucq, Richard B. ^[1]; Parekh, Ojas ^[1]; Severa, William ^[1]; ^[1]

Sandia National Lab. (SNL-NM), Albuquerque, NM (United States). Neural Exploration and Research Lab.

Neuromorphic computing, which aims to replicate the computational structure and architecture of the brain in synthetic hardware, has typically focused on artificial intelligence applications. What is less explored is whether such brain-inspired hardware can provide value beyond cognitive tasks. Here we show that the high degree of parallelism and configurability of spiking neuromorphic architectures makes them well suited to implement random walks via discrete-time Markov chains. Overall, these random walks are useful in Monte Carlo methods, which represent a fundamental computational tool for solving a wide range of numerical computing tasks. Using IBM’s TrueNorth and Intel’s Loihi neuromorphic computing platforms, we show that our neuromorphic computing algorithm for generating random walk approximations of diffusion offers advantages in energy-efficient computation compared with conventional approaches. We also show that our neuromorphic computing algorithm can be extended to more sophisticated jump-diffusion processes that are useful in a range of applications, including financial economics, particle physics and machine learning.

View Accepted Manuscript (DOE)

Research Organization:: Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA), Office of Defense Programs (DP)

Grant/Contract Number:: NA0003525

OSTI ID:: 1845387

Report Number(s):: SAND2022-1227J; 703214

Journal Information:: Nature Electronics, Journal Name: Nature Electronics Journal Issue: 2 Vol. 5; ISSN 2520-1131

Publisher:: Springer NatureCopyright Statement

Country of Publication:: United States

Language:: English

References (38)

Quantum mechanical computers Feynman, Richard P. Foundations of Physics, Vol. 16, Issue 6 https://doi.org/10.1007/BF01886518	journal	June 1986
Multigroup Monte Carlo on GPUs: Comparison of history- and event-based algorithms Hamilton, Steven P.; Slattery, Stuart R.; Evans, Thomas M. Annals of Nuclear Energy, Vol. 113 https://doi.org/10.1016/j.anucene.2017.11.032	journal	March 2018
Continuous-energy Monte Carlo neutron transport on GPUs in the Shift code Hamilton, Steven P.; Evans, Thomas M. Annals of Nuclear Energy, Vol. 128 https://doi.org/10.1016/j.anucene.2019.01.012	journal	June 2019
Deep learning in spiking neural networks Tavanaei, Amirhossein; Ghodrati, Masoud; Kheradpisheh, Saeed Reza Neural Networks, Vol. 111 https://doi.org/10.1016/j.neunet.2018.12.002	journal	March 2019
Structural plasticity on an accelerated analog neuromorphic hardware system Billaudelle, Sebastian; Cramer, Benjamin; Petrovici, Mihai A. Neural Networks, Vol. 133 https://doi.org/10.1016/j.neunet.2020.09.024	journal	January 2021
Random walks and diffusion on networks Masuda, Naoki; Porter, Mason A.; Lambiotte, Renaud Physics Reports, Vol. 716-717 https://doi.org/10.1016/j.physrep.2017.07.007	journal	November 2017
Quantum computational supremacy Harrow, Aram W.; Montanaro, Ashley Nature, Vol. 549, Issue 7671 https://doi.org/10.1038/nature23458	journal	September 2017
Towards quantum chemistry on a quantum computer Lanyon, B. P.; Whitfield, J. D.; Gillett, G. G. Nature Chemistry, Vol. 2, Issue 2 https://doi.org/10.1038/nchem.483	journal	January 2010
A scalable neuristor built with Mott memristors Pickett, Matthew D.; Medeiros-Ribeiro, Gilberto; Williams, R. Stanley Nature Materials, Vol. 12, Issue 2 https://doi.org/10.1038/nmat3510	journal	December 2012
Quantum supremacy using a programmable superconducting processor Arute, Frank; Arya, Kunal; Babbush, Ryan Nature, Vol. 574, Issue 7779 https://doi.org/10.1038/s41586-019-1666-5	journal	October 2019
Towards spike-based machine intelligence with neuromorphic computing Roy, Kaushik; Jaiswal, Akhilesh; Panda, Priyadarshini Nature, Vol. 575, Issue 7784 https://doi.org/10.1038/s41586-019-1677-2	journal	November 2019
Training deep neural networks for binary communication with the Whetstone method Severa, William; Vineyard, Craig M.; Dellana, Ryan Nature Machine Intelligence, Vol. 1, Issue 2 https://doi.org/10.1038/s42256-018-0015-y	journal	January 2019
Benchmarks for progress in neuromorphic computing Davies, Mike Nature Machine Intelligence, Vol. 1, Issue 9 https://doi.org/10.1038/s42256-019-0097-1	journal	September 2019
Fast and energy-efficient neuromorphic deep learning with first-spike times Göltz, J.; Kriener, L.; Baumbach, A. Nature Machine Intelligence, Vol. 3, Issue 9 https://doi.org/10.1038/s42256-021-00388-x	journal	September 2021
Integration of nanoscale memristor synapses in neuromorphic computing architectures Indiveri, Giacomo; Linares-Barranco, Bernabé; Legenstein, Robert Nanotechnology, Vol. 24, Issue 38 https://doi.org/10.1088/0957-4484/24/38/384010	journal	September 2013
No free lunch theorems for optimization Wolpert, D. H.; Macready, W. G. IEEE Transactions on Evolutionary Computation, Vol. 1, Issue 1 https://doi.org/10.1109/4235.585893	journal	April 1997
Spiking Neural Algorithms for Markov Process Random Walk Severa, William; Lehoucq, Rich; Parekh, Ojas 2018 International Joint Conference on Neural Networks (IJCNN) https://doi.org/10.1109/IJCNN.2018.8489628	conference	July 2018
A wafer-scale neuromorphic hardware system for large-scale neural modeling Schemmel, Johannes; Briiderle, Daniel; Griibl, Andreas 2010 IEEE International Symposium on Circuits and Systems - ISCAS 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems https://doi.org/10.1109/ISCAS.2010.5536970	conference	May 2010
The SpiNNaker Project Furber, Steve B.; Galluppi, Francesco; Temple, Steve Proceedings of the IEEE, Vol. 102, Issue 5 https://doi.org/10.1109/JPROC.2014.2304638	journal	May 2014
Advancing Neuromorphic Computing With Loihi: A Survey of Results and Outlook Davies, Mike; Wild, Andreas; Orchard, Garrick Proceedings of the IEEE, Vol. 109, Issue 5 https://doi.org/10.1109/JPROC.2021.3067593	journal	May 2021
Loihi: A Neuromorphic Manycore Processor with On-Chip Learning Davies, Mike; Srinivasa, Narayan; Lin, Tsung-Han IEEE Micro, Vol. 38, Issue 1 https://doi.org/10.1109/MM.2018.112130359	journal	January 2018
Special report : Can we copy the brain? - The brain as computer Meier, Karlheinz IEEE Spectrum, Vol. 54, Issue 6 https://doi.org/10.1109/MSPEC.2017.7934228	journal	June 2017
A million spiking-neuron integrated circuit with a scalable communication network and interface Merolla, P. A.; Arthur, J. V.; Alvarez-Icaza, R. Science, Vol. 345, Issue 6197 https://doi.org/10.1126/science.1254642	journal	August 2014
ROBOTIC VISION:Neuromorphic Vision Sensors Indiveri, G. Science, Vol. 288, Issue 5469 https://doi.org/10.1126/science.288.5469.1189	journal	May 2000
Polynomial-Time Algorithms for Prime Factorization and Discrete Logarithms on a Quantum Computer Shor, Peter W. SIAM Review, Vol. 41, Issue 2 https://doi.org/10.1137/S0036144598347011	journal	January 1999
Constant-Depth and Subcubic-Size Threshold Circuits for Matrix Multiplication Parekh, Ojas; Phillips, Cynthia A.; James, Conrad D. SPAA '18: 30th ACM Symposium on Parallelism in Algorithms and Architectures, Proceedings of the 30th on Symposium on Parallelism in Algorithms and Architectures https://doi.org/10.1145/3210377.3210410	conference	July 2018
Neural algorithms and computing beyond Moore's law Aimone, James B. Communications of the ACM, Vol. 62, Issue 4 https://doi.org/10.1145/3231589	journal	March 2019
Shortest Path and Neighborhood Subgraph Extraction on a Spiking Memristive Neuromorphic Implementation Schuman, Catherine D.; Hamilton, Kathleen; Mintz, Tiffany Proceedings of the 7th Annual Neuro-inspired Computational Elements Workshop on - NICE '19 https://doi.org/10.1145/3320288.3320290	conference	January 2019
Composing neural algorithms with Fugu Aimone, James B.; Severa, William; Vineyard, Craig M. ICONS '19: International Conference on Neuromorphic Systems, Proceedings of the International Conference on Neuromorphic Systems https://doi.org/10.1145/3354265.3354268	conference	July 2019
Graph Partitioning as Quadratic Unconstrained Binary Optimization (QUBO) on Spiking Neuromorphic Hardware Mniszewski, Susan M. ICONS '19: International Conference on Neuromorphic Systems, Proceedings of the International Conference on Neuromorphic Systems https://doi.org/10.1145/3354265.3354269	conference	July 2019
Dynamic Programming with Spiking Neural Computing Aimone, James B.; Parekh, Ojas; Phillips, Cynthia A. ICONS '19: International Conference on Neuromorphic Systems, Proceedings of the International Conference on Neuromorphic Systems https://doi.org/10.1145/3354265.3354285	conference	July 2019
Solving a steady-state PDE using spiking networks and neuromorphic hardware Smith, J. Darby; Severa, William; Hill, Aaron J. ICONS 2020: International Conference on Neuromorphic Systems 2020 https://doi.org/10.1145/3407197.3407202	conference	July 2020
Discrete approximations to reflected Brownian motion Burdzy, Krzysztof; Chen, Zhen-Qing The Annals of Probability, Vol. 36, Issue 2 https://doi.org/10.1214/009117907000000240	journal	March 2008
Neural Dynamics as Sampling: A Model for Stochastic Computation in Recurrent Networks of Spiking Neurons Buesing, Lars; Bill, Johannes; Nessler, Bernhard PLoS Computational Biology, Vol. 7, Issue 11 https://doi.org/10.1371/journal.pcbi.1002211	journal	November 2011
Probabilistic Inference in General Graphical Models through Sampling in Stochastic Networks of Spiking Neurons Pecevski, Dejan; Buesing, Lars; Maass, Wolfgang PLoS Computational Biology, Vol. 7, Issue 12 https://doi.org/10.1371/journal.pcbi.1002294	journal	December 2011
Using Stochastic Spiking Neural Networks on SpiNNaker to Solve Constraint Satisfaction Problems Fonseca Guerra, Gabriel A.; Furber, Steve B. Frontiers in Neuroscience, Vol. 11 https://doi.org/10.3389/fnins.2017.00714	journal	December 2017
Deep Learning With Spiking Neurons: Opportunities and Challenges Pfeiffer, Michael; Pfeil, Thomas Frontiers in Neuroscience, Vol. 12 https://doi.org/10.3389/fnins.2018.00774	journal	October 2018
Making BREAD: Biomimetic Strategies for Artificial Intelligence Now and in the Future Krichmar, Jeffrey L.; Severa, William; Khan, Muhammad S. Frontiers in Neuroscience, Vol. 13 https://doi.org/10.3389/fnins.2019.00666	journal	June 2019