Energy scaling advantages of resistive memory crossbar based computation and its application to sparse coding
Abstract
In this study, the exponential increase in data over the last decade presents a significant challenge to analytics efforts that seek to process and interpret such data for various applications. Neural-inspired computing approaches are being developed in order to leverage the computational properties of the analog, low-power data processing observed in biological systems. Analog resistive memory crossbars can perform a parallel read or a vector-matrix multiplication as well as a parallel write or a rank-1 update with high computational efficiency. For an N × N crossbar, these two kernels can be O(N) more energy efficient than a conventional digital memory-based architecture. If the read operation is noise limited, the energy to read a column can be independent of the crossbar size (O(1)). These two kernels form the basis of many neuromorphic algorithms such as image, text, and speech recognition. For instance, these kernels can be applied to a neural sparse coding algorithm to give an O(N) reduction in energy for the entire algorithm when run with finite precision. Sparse coding is a rich problem with a host of applications including computer vision, object tracking, and more generally unsupervised learning.
- Authors:
-
- Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
- Publication Date:
- Research Org.:
- Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
- Sponsoring Org.:
- USDOE National Nuclear Security Administration (NNSA)
- OSTI Identifier:
- 1236485
- Report Number(s):
- SAND-2015-9530J
Journal ID: ISSN 1662-453X; 607777
- Grant/Contract Number:
- AC04-94AL85000
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Frontiers in Neuroscience (Online)
- Additional Journal Information:
- Journal Name: Frontiers in Neuroscience (Online); Journal Volume: 9; Journal Issue: C; Journal ID: ISSN 1662-453X
- Publisher:
- Frontiers Research Foundation
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 97 MATHEMATICS AND COMPUTING
Citation Formats
Agarwal, Sapan, Quach, Tu -Thach, Parekh, Ojas, DeBenedictis, Erik P., James, Conrad D., Marinella, Matthew J., and Aimone, James B. Energy scaling advantages of resistive memory crossbar based computation and its application to sparse coding. United States: N. p., 2016.
Web. doi:10.3389/fnins.2015.00484.
Agarwal, Sapan, Quach, Tu -Thach, Parekh, Ojas, DeBenedictis, Erik P., James, Conrad D., Marinella, Matthew J., & Aimone, James B. Energy scaling advantages of resistive memory crossbar based computation and its application to sparse coding. United States. https://doi.org/10.3389/fnins.2015.00484
Agarwal, Sapan, Quach, Tu -Thach, Parekh, Ojas, DeBenedictis, Erik P., James, Conrad D., Marinella, Matthew J., and Aimone, James B. Wed .
"Energy scaling advantages of resistive memory crossbar based computation and its application to sparse coding". United States. https://doi.org/10.3389/fnins.2015.00484. https://www.osti.gov/servlets/purl/1236485.
@article{osti_1236485,
title = {Energy scaling advantages of resistive memory crossbar based computation and its application to sparse coding},
author = {Agarwal, Sapan and Quach, Tu -Thach and Parekh, Ojas and DeBenedictis, Erik P. and James, Conrad D. and Marinella, Matthew J. and Aimone, James B.},
abstractNote = {In this study, the exponential increase in data over the last decade presents a significant challenge to analytics efforts that seek to process and interpret such data for various applications. Neural-inspired computing approaches are being developed in order to leverage the computational properties of the analog, low-power data processing observed in biological systems. Analog resistive memory crossbars can perform a parallel read or a vector-matrix multiplication as well as a parallel write or a rank-1 update with high computational efficiency. For an N × N crossbar, these two kernels can be O(N) more energy efficient than a conventional digital memory-based architecture. If the read operation is noise limited, the energy to read a column can be independent of the crossbar size (O(1)). These two kernels form the basis of many neuromorphic algorithms such as image, text, and speech recognition. For instance, these kernels can be applied to a neural sparse coding algorithm to give an O(N) reduction in energy for the entire algorithm when run with finite precision. Sparse coding is a rich problem with a host of applications including computer vision, object tracking, and more generally unsupervised learning.},
doi = {10.3389/fnins.2015.00484},
journal = {Frontiers in Neuroscience (Online)},
number = C,
volume = 9,
place = {United States},
year = {Wed Jan 06 00:00:00 EST 2016},
month = {Wed Jan 06 00:00:00 EST 2016}
}
Free Publicly Available Full Text
Publisher's Version of Record
Other availability
Search WorldCat to find libraries that may hold this journal
Cited by: 64 works
Citation information provided by
Web of Science
Web of Science
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
Experimental Demonstration and Tolerancing of a Large-Scale Neural Network (165 000 Synapses) Using Phase-Change Memory as the Synaptic Weight Element
journal, November 2015
- Burr, Geoffrey W.; Shelby, Robert M.; Sidler, Severin
- IEEE Transactions on Electron Devices, Vol. 62, Issue 11
Real-Time Scalable Cortical Computing at 46 Giga-Synaptic OPS/Watt with ~100× Speedup in Time-to-Solution and ~100,000× Reduction in Energy-to-Solution
conference, November 2014
- Cassidy, Andrew S.; Alvarez-Icaza, Rodrigo; Akopyan, Filipp
- SC14: International Conference for High Performance Computing, Networking, Storage and Analysis
A Comprehensive Crossbar Array Model With Solutions for Line Resistance and Nonlinear Device Characteristics
journal, April 2013
- Chen, An
- IEEE Transactions on Electron Devices, Vol. 60, Issue 4
A Column-Row-Parallel ASIC architecture for 3D wearable / portable medical ultrasonic imaging
conference, June 2014
- Chen, Kailiang; Lee, Hae-Seung; Sodini, Charles G.
- 2014 Symposium on VLSI Circuits Digest of Technical Papers
High performance ultra-low energy RRAM with good retention and endurance
conference, December 2010
- Cheng, C. H.; Tsai, C. Y.; Chin, Albert
- 2010 IEEE International Electron Devices Meeting (IEDM), 2010 International Electron Devices Meeting
Memristor-The missing circuit element
journal, January 1971
- Chua, L.
- IEEE Transactions on Circuit Theory, Vol. 18, Issue 5
CMOS low-power analog circuit design
conference, January 1996
- Enz, C. C.; Vittoz, E. A.
- Designing Low Power Digital Systems, Emerging Technologies (1996), Emerging Technologies: Designing Low Power Digital Systems
Faster Integer Multiplication
journal, January 2009
- Fürer, Martin
- SIAM Journal on Computing, Vol. 39, Issue 3
Processing in memory: the Terasys massively parallel PIM array
journal, April 1995
- Gokhale, M.; Holmes, B.; Iobst, K.
- Computer, Vol. 28, Issue 4
Enabling back propagation training of memristor crossbar neuromorphic processors
conference, July 2014
- Hasan, Raqibul; Taha, Tarek M.
- 2014 International Joint Conference on Neural Networks (IJCNN)
Nanoscale Memristor Device as Synapse in Neuromorphic Systems
journal, April 2010
- Jo, Sung Hyun; Chang, Ting; Ebong, Idongesit
- Nano Letters, Vol. 10, Issue 4, p. 1297-1301
High-Density Crossbar Arrays Based on a Si Memristive System
journal, February 2009
- Jo, Sung Hyun; Kim, Kuk-Hwan; Lu, Wei
- Nano Letters, Vol. 9, Issue 2
A scalable neural chip with synaptic electronics using CMOS integrated memristors
journal, September 2013
- Cruz-Albrecht, Jose M.; Derosier, Timothy; Srinivasa, Narayan
- Nanotechnology, Vol. 24, Issue 38
Parallel Architecture With Resistive Crosspoint Array for Dictionary Learning Acceleration
journal, June 2015
- Kadetotad, Deepak; Xu, Zihan; Mohanty, Abinash
- IEEE Journal on Emerging and Selected Topics in Circuits and Systems, Vol. 5, Issue 2
A Functional Hybrid Memristor Crossbar-Array/CMOS System for Data Storage and Neuromorphic Applications
journal, December 2011
- Kim, Kuk-Hwan; Gaba, Siddharth; Wheeler, Dana
- Nano Letters, Vol. 12, Issue 1
A Reconfigurable Digital Neuromorphic Processor with Memristive Synaptic Crossbar for Cognitive Computing
journal, April 2015
- Kim, Yongtae; Zhang, Yong; Li, Peng
- ACM Journal on Emerging Technologies in Computing Systems, Vol. 11, Issue 4
The metabolic cost of neural information
journal, May 1998
- Laughlin, Simon B.; de Ruyter van Steveninck, Rob R.; Anderson, John C.
- Nature Neuroscience, Vol. 1, Issue 1
Incremental resistance programming of programmable metallization cells for use as electronic synapses
journal, October 2014
- Mahalanabis, D.; Barnaby, H. J.; Gonzalez-Velo, Y.
- Solid-State Electronics, Vol. 100
A million spiking-neuron integrated circuit with a scalable communication network and interface
journal, August 2014
- Merolla, P. A.; Arthur, J. V.; Alvarez-Icaza, R.
- Science, Vol. 345, Issue 6197
Device Requirements for Optical Interconnects to Silicon Chips
journal, July 2009
- Miller, D.
- Proceedings of the IEEE, Vol. 97, Issue 7
Emergence of simple-cell receptive field properties by learning a sparse code for natural images
journal, June 1996
- Olshausen, Bruno A.; Field, David J.
- Nature, Vol. 381, Issue 6583
Sparse coding with an overcomplete basis set: A strategy employed by V1?
journal, December 1997
- Olshausen, Bruno A.; Field, David J.
- Vision Research, Vol. 37, Issue 23
Density of neurons and synapses in the cerebral cortex of the mouse: NEURONS AND SYNAPSES IN THE MOUSE CORTEX
journal, August 1989
- Schüz, Almut; Palm, Günther
- Journal of Comparative Neurology, Vol. 286, Issue 4
Energy-Efficient Non-Boolean Computing With Spin Neurons and Resistive Memory
journal, January 2014
- Sharad, Mrigank; Fan, Deliang; Aitken, Kyle
- IEEE Transactions on Nanotechnology, Vol. 13, Issue 1
The missing memristor found
journal, May 2008
- Strukov, Dmitri B.; Snider, Gregory S.; Stewart, Duncan R.
- Nature, Vol. 453, Issue 7191
Building Neuromorphic Circuits with Memristive Devices
journal, July 2013
- Chang, Ting; Yang, Yuchao; Lu, Wei
- IEEE Circuits and Systems Magazine, Vol. 13, Issue 2
Nanoionics-based resistive switching memories
journal, November 2007
- Waser, Rainer; Aono, Masakazu
- Nature Materials, Vol. 6, Issue 11, p. 833-840
Faster integer multiplication
conference, January 2007
- Fürer, Martin
- Proceedings of the thirty-ninth annual ACM symposium on Theory of computing - STOC '07
Incremental resistance programming of programmable metallization cells for use as electronic synapses
journal, October 2014
- Mahalanabis, D.; Barnaby, H. J.; Gonzalez-Velo, Y.
- Solid-State Electronics, Vol. 100
A Functional Hybrid Memristor Crossbar-Array/CMOS System for Data Storage and Neuromorphic Applications
journal, December 2011
- Kim, Kuk-Hwan; Gaba, Siddharth; Wheeler, Dana
- Nano Letters, Vol. 12, Issue 1
High-Density Crossbar Arrays Based on a Si Memristive System
journal, February 2009
- Jo, Sung Hyun; Kim, Kuk-Hwan; Lu, Wei
- Nano Letters, Vol. 9, Issue 2
Emergence of simple-cell receptive field properties by learning a sparse code for natural images
journal, June 1996
- Olshausen, Bruno A.; Field, David J.
- Nature, Vol. 381, Issue 6583
Nanoionics-based resistive switching memories
journal, November 2007
- Waser, Rainer; Aono, Masakazu
- Nature Materials, Vol. 6, Issue 11, p. 833-840
Processing in memory: the Terasys massively parallel PIM array
journal, April 1995
- Gokhale, M.; Holmes, B.; Iobst, K.
- Computer, Vol. 28, Issue 4
Overcoming the challenges of crossbar resistive memory architectures
conference, February 2015
- Xu, Cong; Niu, Dimin; Muralimanohar, Naveen
- 2015 IEEE 21st International Symposium on High Performance Computer Architecture (HPCA)
Faster Integer Multiplication
journal, January 2009
- Fürer, Martin
- SIAM Journal on Computing, Vol. 39, Issue 3
A Reconfigurable Digital Neuromorphic Processor with Memristive Synaptic Crossbar for Cognitive Computing
journal, April 2015
- Kim, Yongtae; Zhang, Yong; Li, Peng
- ACM Journal on Emerging Technologies in Computing Systems, Vol. 11, Issue 4
Efficient supervised learning in networks with binary synapses
text, January 2007
- Baldassi, Carlo; Braunstein, Alfredo; Brunel, Nicolas
- arXiv
Works referencing / citing this record:
On-Demand Reconfiguration of Nanomaterials: When Electronics Meets Ionics
journal, October 2017
- Lee, Jihang; Lu, Wei D.
- Advanced Materials, Vol. 30, Issue 1
Memristor-Based Analog Computation and Neural Network Classification with a Dot Product Engine
journal, January 2018
- Hu, Miao; Graves, Catherine E.; Li, Can
- Advanced Materials, Vol. 30, Issue 9
Analytical Modeling of Organic-Inorganic CH 3 NH 3 PbI 3 Perovskite Resistive Switching and its Application for Neuromorphic Recognition
journal, February 2018
- Ren, Yanyun; Milo, Valerio; Wang, Zhongqiang
- Advanced Theory and Simulations, Vol. 1, Issue 4
Mechanisms for Enhanced State Retention and Stability in Redox-Gated Organic Neuromorphic Devices
journal, November 2018
- Keene, Scott Tom; Melianas, Armantas; van de Burgt, Yoeri
- Advanced Electronic Materials, Vol. 5, Issue 2
Electrical AFM for the Analysis of Resistive Switching
book, January 2019
- Brivio, Stefano; Frascaroli, Jacopo; Lee, Min Hwan
- Electrical Atomic Force Microscopy for Nanoelectronics
Nanoscale resistive switching devices for memory and computing applications
journal, January 2020
- Lee, Seung Hwan; Zhu, Xiaojian; Lu, Wei D.
- Nano Research, Vol. 13, Issue 5
Sparse coding with memristor networks
journal, May 2017
- Sheridan, Patrick M.; Cai, Fuxi; Du, Chao
- Nature Nanotechnology, Vol. 12, Issue 8
Analog high resistance bilayer RRAM device for hardware acceleration of neuromorphic computation
journal, November 2018
- Jacobs-Gedrim, R. B.; Agarwal, S.; Goeke, R. S.
- Journal of Applied Physics, Vol. 124, Issue 20
Perspective on training fully connected networks with resistive memories: Device requirements for multiple conductances of varying significance
journal, October 2018
- Cristiano, Giorgio; Giordano, Massimo; Ambrogio, Stefano
- Journal of Applied Physics, Vol. 124, Issue 15
Redox-based memristive devices for new computing paradigm
journal, November 2019
- Dittmann, R.; Strachan, J. P.
- APL Materials, Vol. 7, Issue 11
Optimized pulsed write schemes improve linearity and write speed for low-power organic neuromorphic devices
journal, May 2018
- Keene, Scott T.; Melianas, Armantas; Fuller, Elliot J.
- Journal of Physics D: Applied Physics, Vol. 51, Issue 22
Parallel programming of an ionic floating-gate memory array for scalable neuromorphic computing
journal, April 2019
- Fuller, Elliot J.; Keene, Scott T.; Melianas, Armantas
- Science, Vol. 364, Issue 6440
Training Deep Convolutional Neural Networks with Resistive Cross-Point Devices
journal, October 2017
- Gokmen, Tayfun; Onen, Murat; Haensch, Wilfried
- Frontiers in Neuroscience, Vol. 11
Training LSTM Networks With Resistive Cross-Point Devices
journal, October 2018
- Gokmen, Tayfun; Rasch, Malte J.; Haensch, Wilfried
- Frontiers in Neuroscience, Vol. 12
Training Deep Convolutional Neural Networks with Resistive Cross-Point Devices
preprint, January 2017
- Gokmen, Tayfun; Onen, O. Murat; Haensch, Wilfried
- arXiv
Training LSTM Networks with Resistive Cross-Point Devices
preprint, January 2018
- Gokmen, Tayfun; Rasch, Malte; Haensch, Wilfried
- arXiv
Lithium ion trapping mechanism of SiO2 in LiCoO2 based memristors
journal, March 2019
- Hu, Qi; Li, Runmiao; Zhang, Xinjiang
- Scientific Reports, Vol. 9, Issue 1
Training LSTM Networks With Resistive Cross-Point Devices
journal, October 2018
- Gokmen, Tayfun; Rasch, Malte J.; Haensch, Wilfried
- Frontiers in Neuroscience, Vol. 12
Training Deep Convolutional Neural Networks with Resistive Cross-Point Devices
preprint, January 2017
- Gokmen, Tayfun; Onen, O. Murat; Haensch, Wilfried
- arXiv
Algorithm for Training Neural Networks on Resistive Device Arrays
preprint, January 2019
- Gokmen, Tayfun; Haensch, Wilfried
- arXiv