skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Energy scaling advantages of resistive memory crossbar based computation and its application to sparse coding

Abstract

In this study, the exponential increase in data over the last decade presents a significant challenge to analytics efforts that seek to process and interpret such data for various applications. Neural-inspired computing approaches are being developed in order to leverage the computational properties of the analog, low-power data processing observed in biological systems. Analog resistive memory crossbars can perform a parallel read or a vector-matrix multiplication as well as a parallel write or a rank-1 update with high computational efficiency. For an N × N crossbar, these two kernels can be O(N) more energy efficient than a conventional digital memory-based architecture. If the read operation is noise limited, the energy to read a column can be independent of the crossbar size (O(1)). These two kernels form the basis of many neuromorphic algorithms such as image, text, and speech recognition. For instance, these kernels can be applied to a neural sparse coding algorithm to give an O(N) reduction in energy for the entire algorithm when run with finite precision. Sparse coding is a rich problem with a host of applications including computer vision, object tracking, and more generally unsupervised learning.

Authors:
 [1];  [1];  [1];  [1];  [1];  [1];  [1]
  1. Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
Publication Date:
Research Org.:
Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
Sponsoring Org.:
USDOE National Nuclear Security Administration (NNSA)
OSTI Identifier:
1236485
Report Number(s):
SAND-2015-9530J
Journal ID: ISSN 1662-453X; 607777
Grant/Contract Number:  
AC04-94AL85000
Resource Type:
Accepted Manuscript
Journal Name:
Frontiers in Neuroscience (Online)
Additional Journal Information:
Journal Name: Frontiers in Neuroscience (Online); Journal Volume: 9; Journal Issue: C; Journal ID: ISSN 1662-453X
Publisher:
Frontiers Research Foundation
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING

Citation Formats

Agarwal, Sapan, Quach, Tu -Thach, Parekh, Ojas, DeBenedictis, Erik P., James, Conrad D., Marinella, Matthew J., and Aimone, James B. Energy scaling advantages of resistive memory crossbar based computation and its application to sparse coding. United States: N. p., 2016. Web. doi:10.3389/fnins.2015.00484.
Agarwal, Sapan, Quach, Tu -Thach, Parekh, Ojas, DeBenedictis, Erik P., James, Conrad D., Marinella, Matthew J., & Aimone, James B. Energy scaling advantages of resistive memory crossbar based computation and its application to sparse coding. United States. doi:10.3389/fnins.2015.00484.
Agarwal, Sapan, Quach, Tu -Thach, Parekh, Ojas, DeBenedictis, Erik P., James, Conrad D., Marinella, Matthew J., and Aimone, James B. Wed . "Energy scaling advantages of resistive memory crossbar based computation and its application to sparse coding". United States. doi:10.3389/fnins.2015.00484. https://www.osti.gov/servlets/purl/1236485.
@article{osti_1236485,
title = {Energy scaling advantages of resistive memory crossbar based computation and its application to sparse coding},
author = {Agarwal, Sapan and Quach, Tu -Thach and Parekh, Ojas and DeBenedictis, Erik P. and James, Conrad D. and Marinella, Matthew J. and Aimone, James B.},
abstractNote = {In this study, the exponential increase in data over the last decade presents a significant challenge to analytics efforts that seek to process and interpret such data for various applications. Neural-inspired computing approaches are being developed in order to leverage the computational properties of the analog, low-power data processing observed in biological systems. Analog resistive memory crossbars can perform a parallel read or a vector-matrix multiplication as well as a parallel write or a rank-1 update with high computational efficiency. For an N × N crossbar, these two kernels can be O(N) more energy efficient than a conventional digital memory-based architecture. If the read operation is noise limited, the energy to read a column can be independent of the crossbar size (O(1)). These two kernels form the basis of many neuromorphic algorithms such as image, text, and speech recognition. For instance, these kernels can be applied to a neural sparse coding algorithm to give an O(N) reduction in energy for the entire algorithm when run with finite precision. Sparse coding is a rich problem with a host of applications including computer vision, object tracking, and more generally unsupervised learning.},
doi = {10.3389/fnins.2015.00484},
journal = {Frontiers in Neuroscience (Online)},
number = C,
volume = 9,
place = {United States},
year = {2016},
month = {1}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Citation Metrics:
Cited by: 4 works
Citation information provided by
Web of Science

Save / Share:

Works referenced in this record:

Experimental Demonstration and Tolerancing of a Large-Scale Neural Network (165 000 Synapses) Using Phase-Change Memory as the Synaptic Weight Element
journal, November 2015

  • Burr, Geoffrey W.; Shelby, Robert M.; Sidler, Severin
  • IEEE Transactions on Electron Devices, Vol. 62, Issue 11
  • DOI: 10.1109/TED.2015.2439635

Real-Time Scalable Cortical Computing at 46 Giga-Synaptic OPS/Watt with ~100× Speedup in Time-to-Solution and ~100,000× Reduction in Energy-to-Solution
conference, November 2014

  • Cassidy, Andrew S.; Alvarez-Icaza, Rodrigo; Akopyan, Filipp
  • SC14: International Conference for High Performance Computing, Networking, Storage and Analysis
  • DOI: 10.1109/SC.2014.8

A Comprehensive Crossbar Array Model With Solutions for Line Resistance and Nonlinear Device Characteristics
journal, April 2013


High performance ultra-low energy RRAM with good retention and endurance
conference, December 2010

  • Cheng, C. H.; Tsai, C. Y.; Chin, Albert
  • 2010 IEEE International Electron Devices Meeting (IEDM), 2010 International Electron Devices Meeting
  • DOI: 10.1109/IEDM.2010.5703392

Memristor-The missing circuit element
journal, January 1971


CMOS low-power analog circuit design
conference, January 1996

  • Enz, C. C.; Vittoz, E. A.
  • Designing Low Power Digital Systems, Emerging Technologies (1996), Emerging Technologies: Designing Low Power Digital Systems
  • DOI: 10.1109/ETLPDS.1996.508872

Faster Integer Multiplication
journal, January 2009


Processing in memory: the Terasys massively parallel PIM array
journal, April 1995

  • Gokhale, M.; Holmes, B.; Iobst, K.
  • Computer, Vol. 28, Issue 4
  • DOI: 10.1109/2.375174

Enabling back propagation training of memristor crossbar neuromorphic processors
conference, July 2014


Nanoscale Memristor Device as Synapse in Neuromorphic Systems
journal, April 2010

  • Jo, Sung Hyun; Chang, Ting; Ebong, Idongesit
  • Nano Letters, Vol. 10, Issue 4, p. 1297-1301
  • DOI: 10.1021/nl904092h

High-Density Crossbar Arrays Based on a Si Memristive System
journal, February 2009

  • Jo, Sung Hyun; Kim, Kuk-Hwan; Lu, Wei
  • Nano Letters, Vol. 9, Issue 2
  • DOI: 10.1021/nl8037689

A scalable neural chip with synaptic electronics using CMOS integrated memristors
journal, September 2013


Parallel Architecture With Resistive Crosspoint Array for Dictionary Learning Acceleration
journal, June 2015

  • Kadetotad, Deepak; Xu, Zihan; Mohanty, Abinash
  • IEEE Journal on Emerging and Selected Topics in Circuits and Systems, Vol. 5, Issue 2
  • DOI: 10.1109/JETCAS.2015.2426495

A Functional Hybrid Memristor Crossbar-Array/CMOS System for Data Storage and Neuromorphic Applications
journal, December 2011

  • Kim, Kuk-Hwan; Gaba, Siddharth; Wheeler, Dana
  • Nano Letters, Vol. 12, Issue 1
  • DOI: 10.1021/nl203687n

A Reconfigurable Digital Neuromorphic Processor with Memristive Synaptic Crossbar for Cognitive Computing
journal, April 2015

  • Kim, Yongtae; Zhang, Yong; Li, Peng
  • ACM Journal on Emerging Technologies in Computing Systems, Vol. 11, Issue 4
  • DOI: 10.1145/2700234

The metabolic cost of neural information
journal, May 1998

  • Laughlin, Simon B.; de Ruyter van Steveninck, Rob R.; Anderson, John C.
  • Nature Neuroscience, Vol. 1, Issue 1
  • DOI: 10.1038/236

Incremental resistance programming of programmable metallization cells for use as electronic synapses
journal, October 2014


A million spiking-neuron integrated circuit with a scalable communication network and interface
journal, August 2014


Device Requirements for Optical Interconnects to Silicon Chips
journal, July 2009


Emergence of simple-cell receptive field properties by learning a sparse code for natural images
journal, June 1996

  • Olshausen, Bruno A.; Field, David J.
  • Nature, Vol. 381, Issue 6583
  • DOI: 10.1038/381607a0

Sparse coding with an overcomplete basis set: A strategy employed by V1?
journal, December 1997


Density of neurons and synapses in the cerebral cortex of the mouse: NEURONS AND SYNAPSES IN THE MOUSE CORTEX
journal, August 1989

  • Schüz, Almut; Palm, Günther
  • Journal of Comparative Neurology, Vol. 286, Issue 4
  • DOI: 10.1002/cne.902860404

Energy-Efficient Non-Boolean Computing With Spin Neurons and Resistive Memory
journal, January 2014

  • Sharad, Mrigank; Fan, Deliang; Aitken, Kyle
  • IEEE Transactions on Nanotechnology, Vol. 13, Issue 1
  • DOI: 10.1109/TNANO.2013.2286424

The missing memristor found
journal, May 2008

  • Strukov, Dmitri B.; Snider, Gregory S.; Stewart, Duncan R.
  • Nature, Vol. 453, Issue 7191
  • DOI: 10.1038/nature06932

In Quest of the “Next Switch”: Prospects for Greatly Reduced Power Dissipation in a Successor to the Silicon Field-Effect Transistor
journal, December 2010


Building Neuromorphic Circuits with Memristive Devices
journal, July 2013


Nanoionics-based resistive switching memories
journal, November 2007

  • Waser, Rainer; Aono, Masakazu
  • Nature Materials, Vol. 6, Issue 11, p. 833-840
  • DOI: 10.1038/nmat2023

    Works referencing / citing this record:

    On-Demand Reconfiguration of Nanomaterials: When Electronics Meets Ionics
    journal, October 2017


    Memristor-Based Analog Computation and Neural Network Classification with a Dot Product Engine
    journal, January 2018


    Analytical Modeling of Organic-Inorganic CH 3 NH 3 PbI 3 Perovskite Resistive Switching and its Application for Neuromorphic Recognition
    journal, February 2018

    • Ren, Yanyun; Milo, Valerio; Wang, Zhongqiang
    • Advanced Theory and Simulations, Vol. 1, Issue 4
    • DOI: 10.1002/adts.201700035

    Mechanisms for Enhanced State Retention and Stability in Redox-Gated Organic Neuromorphic Devices
    journal, November 2018

    • Keene, Scott Tom; Melianas, Armantas; van de Burgt, Yoeri
    • Advanced Electronic Materials, Vol. 5, Issue 2
    • DOI: 10.1002/aelm.201800686

    Nanoscale resistive switching devices for memory and computing applications
    journal, January 2020


    Sparse coding with memristor networks
    journal, May 2017

    • Sheridan, Patrick M.; Cai, Fuxi; Du, Chao
    • Nature Nanotechnology, Vol. 12, Issue 8
    • DOI: 10.1038/nnano.2017.83

    Analog high resistance bilayer RRAM device for hardware acceleration of neuromorphic computation
    journal, November 2018

    • Jacobs-Gedrim, R. B.; Agarwal, S.; Goeke, R. S.
    • Journal of Applied Physics, Vol. 124, Issue 20
    • DOI: 10.1063/1.5042432

    Optimized pulsed write schemes improve linearity and write speed for low-power organic neuromorphic devices
    journal, May 2018

    • Keene, Scott T.; Melianas, Armantas; Fuller, Elliot J.
    • Journal of Physics D: Applied Physics, Vol. 51, Issue 22
    • DOI: 10.1088/1361-6463/aabe70

    Parallel programming of an ionic floating-gate memory array for scalable neuromorphic computing
    journal, April 2019

    • Fuller, Elliot J.; Keene, Scott T.; Melianas, Armantas
    • Science, Vol. 364, Issue 6440
    • DOI: 10.1126/science.aaw5581

    Training LSTM Networks With Resistive Cross-Point Devices
    journal, October 2018