Integration of Ag-CBRAM crossbars and Mott ReLU neurons for efficient implementation of deep neural networks in hardware
Abstract
Abstract In-memory computing with emerging non-volatile memory devices (eNVMs) has shown promising results in accelerating matrix-vector multiplications (MVMs). However, activation function calculations are still being implemented with general processors or large and complex neuron peripheral circuits. Here, we present the integration of Ag-based conductive bridge random access memory (Ag-CBRAM) crossbar arrays with Mott ReLU activation neurons for scalable, energy and area-efficient hardware implementation of deep neural networks (DNNs). We develop Ag-CBRAM devices that can achieve a high ON/OFF ratio and multi-level programmability. Compact and energy-efficient Mott ReLU neuron devices implementing rectified linear unit (ReLU) activation function are directly connected to the columns of Ag-CBRAM crossbars to compute the output from the weighted sum current. We implement convolution filters and activations for VGG-16 using our integrated hardware and demonstrate the successful generation of feature maps for CIFAR-10 images in hardware. Our approach paves a new way toward building a highly compact and energy-efficient eNVMs-based in-memory computing system.
- Publication Date:
- Sponsoring Org.:
- USDOE
- OSTI Identifier:
- 1997290
- Alternate Identifier(s):
- OSTI ID: 1994436
- Grant/Contract Number:
- #DE-SC0019273
- Resource Type:
- Published Article
- Journal Name:
- Neuromorphic Computing and Engineering
- Additional Journal Information:
- Journal Name: Neuromorphic Computing and Engineering Journal Volume: 3 Journal Issue: 3; Journal ID: ISSN 2634-4386
- Publisher:
- IOP Publishing
- Country of Publication:
- United Kingdom
- Language:
- English
Citation Formats
Shi, Yuhan, Oh, Sangheon, Park, Jaeseoung, Valle, Javier del, Salev, Pavel, Schuller, Ivan K., and Kuzum, Duygu. Integration of Ag-CBRAM crossbars and Mott ReLU neurons for efficient implementation of deep neural networks in hardware. United Kingdom: N. p., 2023.
Web. doi:10.1088/2634-4386/aceea9.
Shi, Yuhan, Oh, Sangheon, Park, Jaeseoung, Valle, Javier del, Salev, Pavel, Schuller, Ivan K., & Kuzum, Duygu. Integration of Ag-CBRAM crossbars and Mott ReLU neurons for efficient implementation of deep neural networks in hardware. United Kingdom. https://doi.org/10.1088/2634-4386/aceea9
Shi, Yuhan, Oh, Sangheon, Park, Jaeseoung, Valle, Javier del, Salev, Pavel, Schuller, Ivan K., and Kuzum, Duygu. Tue .
"Integration of Ag-CBRAM crossbars and Mott ReLU neurons for efficient implementation of deep neural networks in hardware". United Kingdom. https://doi.org/10.1088/2634-4386/aceea9.
@article{osti_1997290,
title = {Integration of Ag-CBRAM crossbars and Mott ReLU neurons for efficient implementation of deep neural networks in hardware},
author = {Shi, Yuhan and Oh, Sangheon and Park, Jaeseoung and Valle, Javier del and Salev, Pavel and Schuller, Ivan K. and Kuzum, Duygu},
abstractNote = {Abstract In-memory computing with emerging non-volatile memory devices (eNVMs) has shown promising results in accelerating matrix-vector multiplications (MVMs). However, activation function calculations are still being implemented with general processors or large and complex neuron peripheral circuits. Here, we present the integration of Ag-based conductive bridge random access memory (Ag-CBRAM) crossbar arrays with Mott ReLU activation neurons for scalable, energy and area-efficient hardware implementation of deep neural networks (DNNs). We develop Ag-CBRAM devices that can achieve a high ON/OFF ratio and multi-level programmability. Compact and energy-efficient Mott ReLU neuron devices implementing rectified linear unit (ReLU) activation function are directly connected to the columns of Ag-CBRAM crossbars to compute the output from the weighted sum current. We implement convolution filters and activations for VGG-16 using our integrated hardware and demonstrate the successful generation of feature maps for CIFAR-10 images in hardware. Our approach paves a new way toward building a highly compact and energy-efficient eNVMs-based in-memory computing system.},
doi = {10.1088/2634-4386/aceea9},
journal = {Neuromorphic Computing and Engineering},
number = 3,
volume = 3,
place = {United Kingdom},
year = {Tue Aug 29 00:00:00 EDT 2023},
month = {Tue Aug 29 00:00:00 EDT 2023}
}
Free Publicly Available Full Text
Publisher's Version of Record
https://doi.org/10.1088/2634-4386/aceea9
https://doi.org/10.1088/2634-4386/aceea9
Other availability
Search WorldCat to find libraries that may hold this journal
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
Wafer-scale growth of VO2 thin films using a combinatorial approach
journal, October 2015
- Zhang, Hai-Tian; Zhang, Lei; Mukherjee, Debangshu
- Nature Communications, Vol. 6, Issue 1
Metal-Insulator Transition in ALD VO 2 Ultrathin Films and Nanoparticles: Morphological Control
journal, December 2014
- Peter, Antony P.; Martens, Koen; Rampelberg, Geert
- Advanced Functional Materials, Vol. 25, Issue 5
Deep learning
journal, May 2015
- LeCun, Yann; Bengio, Yoshua; Hinton, Geoffrey
- Nature, Vol. 521, Issue 7553
A comparison of deep networks with ReLU activation function and linear spline-type methods
journal, February 2019
- Eckle, Konstantin; Schmidt-Hieber, Johannes
- Neural Networks, Vol. 110
Learning in Memristive Neural Network Architectures Using Analog Backpropagation Circuits
journal, February 2019
- Krestinskaya, Olga; Salama, Khaled Nabil; James, Alex Pappachen
- IEEE Transactions on Circuits and Systems I: Regular Papers, Vol. 66, Issue 2
Analog-to-Digital Conversion With Reconfigurable Function Mapping for Neural Networks Activation Function Acceleration
journal, June 2019
- Giordano, Massimo; Cristiano, Giorgio; Ishibashi, Koji
- IEEE Journal on Emerging and Selected Topics in Circuits and Systems, Vol. 9, Issue 2
Heterogeneous integration of single-crystalline rutile nanomembranes with steep phase transition on silicon substrates
journal, August 2021
- Lee, Dong Kyu; Park, Yunkyu; Sim, Hyeji
- Nature Communications, Vol. 12, Issue 1
In-memory computing with resistive switching devices
journal, June 2018
- Ielmini, Daniele; Wong, H. -S. Philip
- Nature Electronics, Vol. 1, Issue 6
Compact Modeling of RRAM Devices and Its Applications in 1T1R and 1S1R Array Design
journal, December 2015
- Chen, Pai-Yu; Yu, Shimeng
- IEEE Transactions on Electron Devices, Vol. 62, Issue 12
Monolithically Integrated RRAM- and CMOS-Based In-Memory Computing Optimizations for Efficient Deep Learning
journal, November 2019
- Yin, Shihui; Seo, Jae-sun; Kim, Yulhwa
- IEEE Micro, Vol. 39, Issue 6
Oxide-based filamentary RRAM for deep learning
journal, December 2020
- Zhang, Yizhou; Huang, Peng; Gao, Bin
- Journal of Physics D: Applied Physics, Vol. 54, Issue 8
Research progress on solutions to the sneak path issue in memristor crossbar arrays
journal, January 2020
- Shi, Lingyun; Zheng, Guohao; Tian, Bobo
- Nanoscale Advances, Vol. 2, Issue 5
Energy-efficient Mott activation neuron for full-hardware implementation of neural networks
journal, March 2021
- Oh, Sangheon; Shi, Yuhan; del Valle, Javier
- Nature Nanotechnology, Vol. 16, Issue 6
Reconfigurable anisotropy and functional transformations with -based metamaterial electric circuits
journal, April 2015
- Savo, Salvatore; Zhou, You; Castaldi, Giuseppe
- Physical Review B, Vol. 91, Issue 13
Conduction mechanism of a TaO x -based selector and its application in crossbar memory arrays
journal, January 2015
- Wang, Ming; Zhou, Jiantao; Yang, Yuchao
- Nanoscale, Vol. 7, Issue 11