Analysis and mitigation of parasitic resistance effects for analog in-memory neural network acceleration

Xiao, T. Patrick; Feinberg, Ben; Rohan, Jacob N.; Bennett, Christopher H.; Agarwal, Sapan; Marinella, Matthew J.

doi:10.1088/1361-6641/ac271a

Title: Analysis and mitigation of parasitic resistance effects for analog in-memory neural network acceleration

Journal Article · Thu Oct 14 00:00:00 EDT 2021 · Semiconductor Science and Technology

DOI:https://doi.org/10.1088/1361-6641/ac271a· OSTI ID:1828786

^[1];

^[1]; Rohan, Jacob N. ^[2];

^[1];

^[1]

Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
Sandia National Lab. (SNL-NM), Albuquerque, NM (United States); Univ. of Texas, Austin, TX (United States)

To support the increasing demands for efficient deep neural network processing, accelerators based on analog in-memory computation of matrix multiplication have recently gained significant attention for reducing the energy of neural network inference. However, analog processing within memory arrays must contend with the issue of parasitic voltage drops across the metal interconnects, which distort the results of the computation and limit the array size. This work analyzes how parasitic resistance affects the end-to-end inference accuracy of state-of-the-art convolutional neural networks, and comprehensively studies how various design decisions at the device, circuit, architecture, and algorithm levels affect the system's sensitivity to parasitic resistance effects. Here, a set of guidelines are provided for how to design analog accelerator hardware that is intrinsically robust to parasitic resistance, without any explicit compensation or re-training of the network parameters.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA)

Grant/Contract Number:: NA0003525

OSTI ID:: 1828786

Report Number(s):: SAND-2021-12073J; 700487

Journal Information:: Semiconductor Science and Technology, Vol. 36, Issue 11; ISSN 0268-1242

Publisher:: IOP PublishingCopyright Statement

Country of Publication:: United States

Language:: English

References (25)

Neuro-Inspired Computing With Emerging Nonvolatile Memorys Yu, Shimeng Proceedings of the IEEE, Vol. 106, Issue 2 https://doi.org/10.1109/JPROC.2018.2790840	journal	February 2018
ISAAC: a convolutional neural network accelerator with in-situ analog arithmetic in crossbars Shafiee, Ali; Nag, Anirban; Muralimanohar, Naveen ACM SIGARCH Computer Architecture News, Vol. 44, Issue 3 https://doi.org/10.1145/3007787.3001139	journal	June 2016
Analogue signal and image processing with large memristor crossbars Li, Can; Hu, Miao; Li, Yunning Nature Electronics, Vol. 1, Issue 1 https://doi.org/10.1038/s41928-017-0002-z	journal	December 2017
Analog architectures for neural network acceleration based on non-volatile memory Xiao, T. Patrick; Bennett, Christopher H.; Feinberg, Ben Applied Physics Reviews, Vol. 7, Issue 3 https://doi.org/10.1063/1.5143815	journal	September 2020
Training and operation of an integrated neuromorphic network based on metal-oxide memristors Prezioso, M.; Merrikh-Bayat, F.; Hoskins, B. D. Nature, Vol. 521, Issue 7550 https://doi.org/10.1038/nature14441	journal	May 2015
Access devices for 3D crosspoint memory Burr, Geoffrey W.; Shenoy, Rohit S.; Virwani, Kumar Journal of Vacuum Science & Technology B, Nanotechnology and Microelectronics: Materials, Processing, Measurement, and Phenomena, Vol. 32, Issue 4 https://doi.org/10.1116/1.4889999	journal	July 2014
MLPerf Inference Benchmark Reddi, Vijay Janapa; Cheng, Christine; Kanter, David 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA) https://doi.org/10.1109/ISCA45697.2020.00045	conference	May 2020
Mitigate Parasitic Resistance in Resistive Crossbar-based Convolutional Neural Networks Zhang, Fan; Hu, Miao ACM Journal on Emerging Technologies in Computing Systems, Vol. 16, Issue 3 https://doi.org/10.1145/3371277	journal	July 2020
Recent progress in analog memory-based accelerators for deep learning Tsai, Hsinyu; Ambrogio, Stefano; Narayanan, Pritish Journal of Physics D: Applied Physics, Vol. 51, Issue 28 https://doi.org/10.1088/1361-6463/aac8a5	journal	June 2018
Memristive Boltzmann machine: A hardware accelerator for combinatorial optimization and deep learning Bojnordi, Mahdi Nazm; Ipek, Engin 2016 IEEE International Symposium on High Performance Computer Architecture (HPCA) https://doi.org/10.1109/HPCA.2016.7446049	conference	March 2016
Memory devices and applications for in-memory computing Sebastian, Abu; Le Gallo, Manuel; Khaddam-Aljameh, Riduan Nature Nanotechnology, Vol. 15, Issue 7 https://doi.org/10.1038/s41565-020-0655-z	journal	March 2020
ImageNet: A large-scale hierarchical image database Deng, Jia; Dong, Wei; Socher, Richard 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPR Workshops), 2009 IEEE Conference on Computer Vision and Pattern Recognition https://doi.org/10.1109/CVPR.2009.5206848	conference	June 2009
Circuit-Level Benchmarking of Access Devices for Resistive Nonvolatile Memory Arrays Narayanan, Pritish; Burr, Geoffrey W.; Virwani, Kumar IEEE Journal on Emerging and Selected Topics in Circuits and Systems, Vol. 6, Issue 3 https://doi.org/10.1109/JETCAS.2016.2547744	journal	September 2016
A Floating-Gate-Based Field-Programmable Analog Array Basu, Arindam; Brink, Stephen; Schlottmann, Craig IEEE Journal of Solid-State Circuits, Vol. 45, Issue 9 https://doi.org/10.1109/JSSC.2010.2056832	journal	September 2010
CxDNN: Hardware-software Compensation Methods for Deep Neural Networks on Resistive Crossbar Systems Jain, Shubham; Raghunathan, Anand ACM Transactions on Embedded Computing Systems, Vol. 18, Issue 6 https://doi.org/10.1145/3362035	journal	January 2020
Multiscale Co-Design Analysis of Energy, Latency, Area, and Accuracy of a ReRAM Analog Neural Training Accelerator Marinella, Matthew J.; Agarwal, Sapan; Hsia, Alexander IEEE Journal on Emerging and Selected Topics in Circuits and Systems, Vol. 8, Issue 1 https://doi.org/10.1109/JETCAS.2018.2796379	journal	March 2018
Accurate deep neural network inference using computational phase-change memory Joshi, Vinay; Le Gallo, Manuel; Haefeli, Simon Nature Communications, Vol. 11, Issue 1 https://doi.org/10.1038/s41467-020-16108-9	journal	May 2020
Fully hardware-implemented memristor convolutional neural network Yao, Peng; Wu, Huaqiang; Gao, Bin Nature, Vol. 577, Issue 7792 https://doi.org/10.1038/s41586-020-1942-4	journal	January 2020
Parallel programming of an ionic floating-gate memory array for scalable neuromorphic computing Fuller, Elliot J.; Keene, Scott T.; Melianas, Armantas Science, Vol. 364, Issue 6440 https://doi.org/10.1126/science.aaw5581	journal	April 2019
Efficient Mixed-Signal Neurocomputing Via Successive Integration and Rescaling Bavandpour, Mohammad; Sahay, Shubham; Mahmoodi, Mohammad R. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, Vol. 28, Issue 3 https://doi.org/10.1109/TVLSI.2019.2946516	journal	March 2020
Efficient Processing of Deep Neural Networks: A Tutorial and Survey Sze, Vivienne; Chen, Yu-Hsin; Yang, Tien-Ju Proceedings of the IEEE, Vol. 105, Issue 12 https://doi.org/10.1109/JPROC.2017.2761740	journal	December 2017
Exploring the Design Space for Crossbar Arrays Built With Mixed-Ionic-Electronic-Conduction (MIEC) Access Devices Narayanan, Pritish; Burr, Geoffrey W.; Shenoy, Rohit S. IEEE Journal of the Electron Devices Society, Vol. 3, Issue 5 https://doi.org/10.1109/JEDS.2015.2442242	journal	September 2015
Parasitic Effect Analysis in Memristor-Array-Based Neuromorphic Systems Jeong, YeonJoo; Zidan, Mohammed A.; Lu, Wei D. IEEE Transactions on Nanotechnology, Vol. 17, Issue 1 https://doi.org/10.1109/TNANO.2017.2784364	journal	January 2018
RxNN: A Framework for Evaluating Deep Neural Networks on Resistive Crossbars Jain, Shubham; Sengupta, Abhronil; Roy, Kaushik IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, Vol. 40, Issue 2 https://doi.org/10.1109/TCAD.2020.3000185	journal	February 2021
Charge-mode parallel architecture for matrix-vector multiplication Genov, R.; Cauwenberghs, G. Proceedings of the 43rd IEEE Midwest Symposium on Circuits and Systems (Cat.No.CH37144) https://doi.org/10.1109/MWSCAS.2000.951694	conference	January 2000

Similar Records

An Accurate, Error-Tolerant, and Energy-Efficient Neural Network Inference Engine Based on SONOS Analog Memory

Journal Article · Tue Jan 04 00:00:00 EST 2022 · IEEE Transactions on Circuits and Systems I: Regular Papers · OSTI ID:1828786

Xiao, Tianyao Patrick; Feinberg, Benjamin; Bennett, Christopher H.; +9 more

Analog architectures for neural network acceleration based on non-volatile memory

Journal Article · Thu Jul 09 00:00:00 EDT 2020 · Applied Physics Reviews · OSTI ID:1828786

Xiao, T. Patrick; Bennett, Christopher H.; Feinberg, Ben; +2 more

In situ Parallel Training of Analog Neural Network Using Electrochemical Random-Access Memory

Journal Article · Thu Apr 08 00:00:00 EDT 2021 · Frontiers in Neuroscience (Online) · OSTI ID:1828786

Li, Yiyang; Xiao, T. Patrick; Bennett, Christopher H.; +7 more

Related Subjects

97 MATHEMATICS AND COMPUTING

Title: Analysis and mitigation of parasitic resistance effects for analog in-memory neural network acceleration

Citation Formats

References (25)

Similar Records

Related Subjects