Energy scaling advantages of resistive memory crossbar based computation and its application to sparse coding

Agarwal, Sapan; Quach, Tu -Thach; Parekh, Ojas; DeBenedictis, Erik P.; James, Conrad D.; Marinella, Matthew J.; Aimone, James B.

doi:10.3389/fnins.2015.00484

Title: Energy scaling advantages of resistive memory crossbar based computation and its application to sparse coding

Journal Article · Wed Jan 06 00:00:00 EST 2016 · Frontiers in Neuroscience (Online)

DOI:https://doi.org/10.3389/fnins.2015.00484· OSTI ID:1236485

Agarwal, Sapan ^[1]; Quach, Tu -Thach ^[1]; Parekh, Ojas ^[1]; DeBenedictis, Erik P. ^[1]; James, Conrad D. ^[1]; Marinella, Matthew J. ^[1]; Aimone, James B. ^[1]

Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)

In this study, the exponential increase in data over the last decade presents a significant challenge to analytics efforts that seek to process and interpret such data for various applications. Neural-inspired computing approaches are being developed in order to leverage the computational properties of the analog, low-power data processing observed in biological systems. Analog resistive memory crossbars can perform a parallel read or a vector-matrix multiplication as well as a parallel write or a rank-1 update with high computational efficiency. For an N × N crossbar, these two kernels can be O(N) more energy efficient than a conventional digital memory-based architecture. If the read operation is noise limited, the energy to read a column can be independent of the crossbar size (O(1)). These two kernels form the basis of many neuromorphic algorithms such as image, text, and speech recognition. For instance, these kernels can be applied to a neural sparse coding algorithm to give an O(N) reduction in energy for the entire algorithm when run with finite precision. Sparse coding is a rich problem with a host of applications including computer vision, object tracking, and more generally unsupervised learning.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA)

Grant/Contract Number:: AC04-94AL85000

OSTI ID:: 1236485

Report Number(s):: SAND-2015-9530J; 607777

Journal Information:: Frontiers in Neuroscience (Online), Vol. 9, Issue C; ISSN 1662-453X

Publisher:: Frontiers Research FoundationCopyright Statement

Country of Publication:: United States

Language:: English

Citation Metrics:

Cited by: 64 works

Citation information provided by
Web of Science

References (30)

Experimental Demonstration and Tolerancing of a Large-Scale Neural Network (165 000 Synapses) Using Phase-Change Memory as the Synaptic Weight Element Burr, Geoffrey W.; Shelby, Robert M.; Sidler, Severin IEEE Transactions on Electron Devices, Vol. 62, Issue 11 https://doi.org/10.1109/TED.2015.2439635	journal	November 2015
Real-Time Scalable Cortical Computing at 46 Giga-Synaptic OPS/Watt with ~100× Speedup in Time-to-Solution and ~100,000× Reduction in Energy-to-Solution Cassidy, Andrew S.; Alvarez-Icaza, Rodrigo; Akopyan, Filipp SC14: International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1109/SC.2014.8	conference	November 2014
A Comprehensive Crossbar Array Model With Solutions for Line Resistance and Nonlinear Device Characteristics Chen, An IEEE Transactions on Electron Devices, Vol. 60, Issue 4 https://doi.org/10.1109/TED.2013.2246791	journal	April 2013
A Column-Row-Parallel ASIC architecture for 3D wearable / portable medical ultrasonic imaging Chen, Kailiang; Lee, Hae-Seung; Sodini, Charles G. 2014 Symposium on VLSI Circuits Digest of Technical Papers https://doi.org/10.1109/VLSIC.2014.6858445	conference	June 2014
High performance ultra-low energy RRAM with good retention and endurance Cheng, C. H.; Tsai, C. Y.; Chin, Albert 2010 IEEE International Electron Devices Meeting (IEDM), 2010 International Electron Devices Meeting https://doi.org/10.1109/IEDM.2010.5703392	conference	December 2010
Memristor-The missing circuit element Chua, L. IEEE Transactions on Circuit Theory, Vol. 18, Issue 5 https://doi.org/10.1109/TCT.1971.1083337	journal	January 1971
CMOS low-power analog circuit design Enz, C. C.; Vittoz, E. A. Designing Low Power Digital Systems, Emerging Technologies (1996), Emerging Technologies: Designing Low Power Digital Systems https://doi.org/10.1109/ETLPDS.1996.508872	conference	January 1996
Faster Integer Multiplication Fürer, Martin SIAM Journal on Computing, Vol. 39, Issue 3 https://doi.org/10.1137/070711761	journal	January 2009
Processing in memory: the Terasys massively parallel PIM array Gokhale, M.; Holmes, B.; Iobst, K. Computer, Vol. 28, Issue 4 https://doi.org/10.1109/2.375174	journal	April 1995
Enabling back propagation training of memristor crossbar neuromorphic processors Hasan, Raqibul; Taha, Tarek M. 2014 International Joint Conference on Neural Networks (IJCNN) https://doi.org/10.1109/IJCNN.2014.6889893	conference	July 2014
Nanoscale Memristor Device as Synapse in Neuromorphic Systems Jo, Sung Hyun; Chang, Ting; Ebong, Idongesit Nano Letters, Vol. 10, Issue 4, p. 1297-1301 https://doi.org/10.1021/nl904092h	journal	April 2010
High-Density Crossbar Arrays Based on a Si Memristive System Jo, Sung Hyun; Kim, Kuk-Hwan; Lu, Wei Nano Letters, Vol. 9, Issue 2 https://doi.org/10.1021/nl8037689	journal	February 2009
A scalable neural chip with synaptic electronics using CMOS integrated memristors Cruz-Albrecht, Jose M.; Derosier, Timothy; Srinivasa, Narayan Nanotechnology, Vol. 24, Issue 38 https://doi.org/10.1088/0957-4484/24/38/384011	journal	September 2013
Parallel Architecture With Resistive Crosspoint Array for Dictionary Learning Acceleration Kadetotad, Deepak; Xu, Zihan; Mohanty, Abinash IEEE Journal on Emerging and Selected Topics in Circuits and Systems, Vol. 5, Issue 2 https://doi.org/10.1109/JETCAS.2015.2426495	journal	June 2015
A Functional Hybrid Memristor Crossbar-Array/CMOS System for Data Storage and Neuromorphic Applications Kim, Kuk-Hwan; Gaba, Siddharth; Wheeler, Dana Nano Letters, Vol. 12, Issue 1 https://doi.org/10.1021/nl203687n	journal	December 2011
A Reconfigurable Digital Neuromorphic Processor with Memristive Synaptic Crossbar for Cognitive Computing Kim, Yongtae; Zhang, Yong; Li, Peng ACM Journal on Emerging Technologies in Computing Systems, Vol. 11, Issue 4 https://doi.org/10.1145/2700234	journal	April 2015
The metabolic cost of neural information Laughlin, Simon B.; de Ruyter van Steveninck, Rob R.; Anderson, John C. Nature Neuroscience, Vol. 1, Issue 1 https://doi.org/10.1038/236	journal	May 1998
Incremental resistance programming of programmable metallization cells for use as electronic synapses Mahalanabis, D.; Barnaby, H. J.; Gonzalez-Velo, Y. Solid-State Electronics, Vol. 100 https://doi.org/10.1016/j.sse.2014.07.002	journal	October 2014
A million spiking-neuron integrated circuit with a scalable communication network and interface Merolla, P. A.; Arthur, J. V.; Alvarez-Icaza, R. Science, Vol. 345, Issue 6197 https://doi.org/10.1126/science.1254642	journal	August 2014
Device Requirements for Optical Interconnects to Silicon Chips Miller, D. Proceedings of the IEEE, Vol. 97, Issue 7 https://doi.org/10.1109/JPROC.2009.2014298	journal	July 2009
Emergence of simple-cell receptive field properties by learning a sparse code for natural images Olshausen, Bruno A.; Field, David J. Nature, Vol. 381, Issue 6583 https://doi.org/10.1038/381607a0	journal	June 1996
Sparse coding with an overcomplete basis set: A strategy employed by V1? Olshausen, Bruno A.; Field, David J. Vision Research, Vol. 37, Issue 23 https://doi.org/10.1016/S0042-6989(97)00169-7	journal	December 1997
Density of neurons and synapses in the cerebral cortex of the mouse: NEURONS AND SYNAPSES IN THE MOUSE CORTEX Schüz, Almut; Palm, Günther Journal of Comparative Neurology, Vol. 286, Issue 4 https://doi.org/10.1002/cne.902860404	journal	August 1989
Energy-Efficient Non-Boolean Computing With Spin Neurons and Resistive Memory Sharad, Mrigank; Fan, Deliang; Aitken, Kyle IEEE Transactions on Nanotechnology, Vol. 13, Issue 1 https://doi.org/10.1109/TNANO.2013.2286424	journal	January 2014
The missing memristor found Strukov, Dmitri B.; Snider, Gregory S.; Stewart, Duncan R. Nature, Vol. 453, Issue 7191 https://doi.org/10.1038/nature06932	journal	May 2008
Building Neuromorphic Circuits with Memristive Devices Chang, Ting; Yang, Yuchao; Lu, Wei IEEE Circuits and Systems Magazine, Vol. 13, Issue 2 https://doi.org/10.1109/MCAS.2013.2256260	journal	July 2013
Nanoionics-based resistive switching memories Waser, Rainer; Aono, Masakazu Nature Materials, Vol. 6, Issue 11, p. 833-840 https://doi.org/10.1038/nmat2023	journal	November 2007
Faster integer multiplication Fürer, Martin Proceedings of the thirty-ninth annual ACM symposium on Theory of computing - STOC '07 https://doi.org/10.1145/1250790.1250800	conference	January 2007
Overcoming the challenges of crossbar resistive memory architectures Xu, Cong; Niu, Dimin; Muralimanohar, Naveen 2015 IEEE 21st International Symposium on High Performance Computer Architecture (HPCA) https://doi.org/10.1109/hpca.2015.7056056	conference	February 2015
Efficient supervised learning in networks with binary synapses Baldassi, Carlo; Braunstein, Alfredo; Brunel, Nicolas arXiv https://doi.org/10.48550/arxiv.0707.1295	text	January 2007

Cited By (18)

On-Demand Reconfiguration of Nanomaterials: When Electronics Meets Ionics Lee, Jihang; Lu, Wei D. Advanced Materials, Vol. 30, Issue 1 https://doi.org/10.1002/adma.201702770	journal	October 2017
Memristor-Based Analog Computation and Neural Network Classification with a Dot Product Engine Hu, Miao; Graves, Catherine E.; Li, Can Advanced Materials, Vol. 30, Issue 9 https://doi.org/10.1002/adma.201705914	journal	January 2018
Analytical Modeling of Organic-Inorganic CH ₃ NH ₃ PbI ₃ Perovskite Resistive Switching and its Application for Neuromorphic Recognition Ren, Yanyun; Milo, Valerio; Wang, Zhongqiang Advanced Theory and Simulations, Vol. 1, Issue 4 https://doi.org/10.1002/adts.201700035	journal	February 2018
Mechanisms for Enhanced State Retention and Stability in Redox-Gated Organic Neuromorphic Devices Keene, Scott Tom; Melianas, Armantas; van de Burgt, Yoeri Advanced Electronic Materials, Vol. 5, Issue 2 https://doi.org/10.1002/aelm.201800686	journal	November 2018
Electrical AFM for the Analysis of Resistive Switching Brivio, Stefano; Frascaroli, Jacopo; Lee, Min Hwan Electrical Atomic Force Microscopy for Nanoelectronics https://doi.org/10.1007/978-3-030-15612-1_7	book	January 2019
Nanoscale resistive switching devices for memory and computing applications Lee, Seung Hwan; Zhu, Xiaojian; Lu, Wei D. Nano Research, Vol. 13, Issue 5 https://doi.org/10.1007/s12274-020-2616-0	journal	January 2020
Sparse coding with memristor networks Sheridan, Patrick M.; Cai, Fuxi; Du, Chao Nature Nanotechnology, Vol. 12, Issue 8 https://doi.org/10.1038/nnano.2017.83	journal	May 2017
Analog high resistance bilayer RRAM device for hardware acceleration of neuromorphic computation Jacobs-Gedrim, R. B.; Agarwal, S.; Goeke, R. S. Journal of Applied Physics, Vol. 124, Issue 20 https://doi.org/10.1063/1.5042432	journal	November 2018
Perspective on training fully connected networks with resistive memories: Device requirements for multiple conductances of varying significance Cristiano, Giorgio; Giordano, Massimo; Ambrogio, Stefano Journal of Applied Physics, Vol. 124, Issue 15 https://doi.org/10.1063/1.5042462	journal	October 2018
Redox-based memristive devices for new computing paradigm Dittmann, R.; Strachan, J. P. APL Materials, Vol. 7, Issue 11 https://doi.org/10.1063/1.5129101	journal	November 2019
Optimized pulsed write schemes improve linearity and write speed for low-power organic neuromorphic devices Keene, Scott T.; Melianas, Armantas; Fuller, Elliot J. Journal of Physics D: Applied Physics, Vol. 51, Issue 22 https://doi.org/10.1088/1361-6463/aabe70	journal	May 2018
Parallel programming of an ionic floating-gate memory array for scalable neuromorphic computing Fuller, Elliot J.; Keene, Scott T.; Melianas, Armantas Science, Vol. 364, Issue 6440 https://doi.org/10.1126/science.aaw5581	journal	April 2019
Training Deep Convolutional Neural Networks with Resistive Cross-Point Devices Gokmen, Tayfun; Onen, Murat; Haensch, Wilfried Frontiers in Neuroscience, Vol. 11 https://doi.org/10.3389/fnins.2017.00538	journal	October 2017
Training LSTM Networks With Resistive Cross-Point Devices Gokmen, Tayfun; Rasch, Malte J.; Haensch, Wilfried Frontiers in Neuroscience, Vol. 12 https://doi.org/10.3389/fnins.2018.00745	journal	October 2018
Training Deep Convolutional Neural Networks with Resistive Cross-Point Devices Gokmen, Tayfun; Onen, O. Murat; Haensch, Wilfried arXiv https://doi.org/10.48550/arxiv.1705.08014	preprint	January 2017
Training LSTM Networks with Resistive Cross-Point Devices Gokmen, Tayfun; Rasch, Malte; Haensch, Wilfried arXiv https://doi.org/10.48550/arxiv.1806.00166	preprint	January 2018
Lithium ion trapping mechanism of SiO2 in LiCoO2 based memristors Hu, Qi; Li, Runmiao; Zhang, Xinjiang Scientific Reports, Vol. 9, Issue 1 https://doi.org/10.1038/s41598-019-41508-3	journal	March 2019
Algorithm for Training Neural Networks on Resistive Device Arrays Gokmen, Tayfun; Haensch, Wilfried arXiv https://doi.org/10.48550/arxiv.1909.07908	preprint	January 2019

Similar Records

Energy Efficient Stochastic-Based Deep Spiking Neural Networks for Sparse Datasets

Conference · Fri Dec 01 00:00:00 EST 2017 · OSTI ID:1236485

Alawad, Mohammed; Yoon, Hong-Jun; Tourassi, Georgia

Parallel programming of an ionic floating-gate memory array for scalable neuromorphic computing

Journal Article · Thu Apr 25 00:00:00 EDT 2019 · Science · OSTI ID:1236485

Fuller, Elliot J.; Keene, Scott T.; Melianas, Armantas; +9 more