An Accurate, Error-Tolerant, and Energy-Efficient Neural Network Inference Engine Based on SONOS Analog Memory

Xiao, Tianyao Patrick; Feinberg, Benjamin; Bennett, Christopher H.; Agrawal, Vineet; Saxena, Prashant; Prabhakar, Venkatraman; Ramkumar, Krishnaswamy; Medu, Harsha; Raghavan, Vijay; Chettuvetty, Ramesh; Agarwal, Sapan; Marinella, Matthew J.

doi:10.1109/tcsi.2021.3134313

Title: An Accurate, Error-Tolerant, and Energy-Efficient Neural Network Inference Engine Based on SONOS Analog Memory

Journal Article · Tue Jan 04 00:00:00 EST 2022 · IEEE Transactions on Circuits and Systems I: Regular Papers

DOI:https://doi.org/10.1109/tcsi.2021.3134313· OSTI ID:1842837

^[1];

^[1]; Agrawal, Vineet ^[2]; Saxena, Prashant ^[2]; Prabhakar, Venkatraman ^[2]; Ramkumar, Krishnaswamy ^[2]; Medu, Harsha ^[2]; Raghavan, Vijay ^[2]; Chettuvetty, Ramesh ^[2];

^[1];

^[3]

Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
Infineon Technologies LLC, San Jose, CA (United States)
Sandia National Lab. (SNL-NM), Albuquerque, NM (United States); Arizona State Univ., Tempe, AZ (United States)

In this work, we demonstrate SONOS (silicon-oxide-nitrideoxide- silicon) analog memory arrays that are optimized for neural network inference. The devices are fabricated in a 40nm process and operated in the subthreshold regime for in-memory matrix multiplication. Subthreshold operation enables low conductances to be implemented with low error, which matches the typical weight distribution of neural networks, which is heavily skewed toward near-zero values. This leads to high accuracy in the presence of programming errors and process variations. We simulate the end-to-end neural network inference accuracy, accounting for the measured programming error, read noise, and retention loss in a fabricated SONOS array. Evaluated on the ImageNet dataset using ResNet50, the accuracy using a SONOS system is within 2.16% of floating-point accuracy without any retraining. The unique error properties and high On/Off ratio of the SONOS device allow scaling to large arrays without bit slicing, and enable an inference architecture that achieves 20 TOPS/W on ResNet50, a >10× gain in energy efficiency over state-of-the-art digital and analog inference accelerators.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA); USDOE Laboratory Directed Research and Development (LDRD) Program; Defense Threat Reduction Agency (DTRA)

Grant/Contract Number:: NA0003525; HDTRA1-17-1-0038

OSTI ID:: 1842837

Report Number(s):: SAND-2022-0047J; 703075

Journal Information:: IEEE Transactions on Circuits and Systems I: Regular Papers, Vol. 69, Issue 4; ISSN 1549-8328

Publisher:: IEEECopyright Statement

Country of Publication:: United States

Language:: English

References (52)

Memristive devices and systems Chua, L. O. Proceedings of the IEEE, Vol. 64, Issue 2 https://doi.org/10.1109/PROC.1976.10092	journal	January 1976
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference Jacob, Benoit; Kligys, Skirmantas; Chen, Bo 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2018.00286	conference	June 2018
Sparse ReRAM engine Yang, Tzu-Hsien; Cheng, Hsiang-Yun; Yang, Chia-Lin Proceedings of the 46th International Symposium on Computer Architecture https://doi.org/10.1145/3307650.3322271	conference	June 2019
PUMA: A Programmable Ultra-efficient Memristor-based Accelerator for Machine Learning Inference Ankit, Aayush; Hajj, Izzat El; Chalamalasetti, Sai Rahul ASPLOS '19: Architectural Support for Programming Languages and Operating Systems, Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems https://doi.org/10.1145/3297858.3304049	conference	April 2019
Newton: Gravitating Towards the Physical Limits of Crossbar Acceleration Nag, Anirban; Balasubramonian, Rajeev; Srikumar, Vivek IEEE Micro, Vol. 38, Issue 5 https://doi.org/10.1109/MM.2018.053631140	journal	September 2018
A Ferroelectric FET-Based Processing-in-Memory Architecture for DNN Acceleration Long, Yun; Kim, Daehyun; Lee, Edward IEEE Journal on Exploratory Solid-State Computational Devices and Circuits, Vol. 5, Issue 2 https://doi.org/10.1109/JXCDC.2019.2923745	journal	December 2019
Timely: Pushing Data Movements And Interfaces In Pim Accelerators Towards Local And In Time Domain Li, Weitao; Xu, Pengfei; Zhao, Yang 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA) https://doi.org/10.1109/ISCA45697.2020.00073	conference	May 2020
Efficient Mixed-Signal Neurocomputing Via Successive Integration and Rescaling Bavandpour, Mohammad; Sahay, Shubham; Mahmoodi, Mohammad R. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, Vol. 28, Issue 3 https://doi.org/10.1109/TVLSI.2019.2946516	journal	March 2020
A 3.1 mW 8b 1.2 GS/s Single-Channel Asynchronous SAR ADC With Alternate Comparators for Enhanced Speed in 32 nm Digital SOI CMOS Kull, Lukas; Toifl, Thomas; Schmatz, Martin IEEE Journal of Solid-State Circuits, Vol. 48, Issue 12 https://doi.org/10.1109/JSSC.2013.2279571	journal	December 2013
Analog in-memory subthreshold deep neural network accelerator Fick, L.; Blaauw, D.; Sylvester, D. 2017 IEEE Custom Integrated Circuits Conference (CICC) https://doi.org/10.1109/CICC.2017.7993629	conference	April 2017
CACTI 7: New Tools for Interconnect Exploration in Innovative Off-Chip Memories Balasubramonian, Rajeev; Kahng, Andrew B.; Muralimanohar, Naveen ACM Transactions on Architecture and Code Optimization, Vol. 14, Issue 2 https://doi.org/10.1145/3085572	journal	July 2017
Three-Dimensional nand Flash for Vector–Matrix Multiplication Wang, Panni; Xu, Feng; Wang, Bo IEEE Transactions on Very Large Scale Integration (VLSI) Systems, Vol. 27, Issue 4 https://doi.org/10.1109/TVLSI.2018.2882194	journal	April 2019
Improved on-chip router analytical power and area modeling Kahng, Andrew B.; Lin, Bill; Samadi, Kambiz 2010 15th Asia and South Pacific Design Automation Conference (ASP-DAC) https://doi.org/10.1109/ASPDAC.2010.5419887	conference	January 2010
Explicit modeling of control and data for improved NoC router estimation Kahng, Andrew B.; Lin, Bill; Nath, Siddhartha Proceedings of the 49th Annual Design Automation Conference on - DAC '12 https://doi.org/10.1145/2228360.2228430	conference	January 2012
An Analog Neural Network Computing Engine Using CMOS-Compatible Charge-Trap-Transistor (CTT) Du, Yuan; Du, Li; Gu, Xuefeng IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, Vol. 38, Issue 10 https://doi.org/10.1109/TCAD.2018.2859237	journal	October 2019
Hanguang 800 NPU – The Ultimate AI Inference Solution for Data Centers Jiao, Yang; Han, Liang; Long, Xin 2020 IEEE Hot Chips 32 Symposium (HCS) https://doi.org/10.1109/HCS49909.2020.9220619	conference	August 2020
Serving DNNs in Real Time at Datacenter Scale with Project Brainwave Chung, Eric; Haselman, Michael; Abeydeera, Maleen IEEE Micro, Vol. 38, Issue 2 https://doi.org/10.1109/MM.2018.022071131	journal	March 2018
Eyeriss: a spatial architecture for energy-efficient dataflow for convolutional neural networks Chen, Yu-Hsin; Emer, Joel; Sze, Vivienne ACM SIGARCH Computer Architecture News, Vol. 44, Issue 3 https://doi.org/10.1145/3007787.3001177	journal	October 2016
Dot-product engine for neuromorphic computing: programming 1T1M crossbar to accelerate matrix-vector multiplication Hu, Miao; Williams, R. Stanley; Strachan, John Paul Proceedings of the 53rd Annual Design Automation Conference on - DAC '16 https://doi.org/10.1145/2897937.2898010	conference	January 2016
Efficient Processing of Deep Neural Networks: A Tutorial and Survey Sze, Vivienne; Chen, Yu-Hsin; Yang, Tien-Ju Proceedings of the IEEE, Vol. 105, Issue 12 https://doi.org/10.1109/JPROC.2017.2761740	journal	December 2017
Fully hardware-implemented memristor convolutional neural network Yao, Peng; Wu, Huaqiang; Gao, Bin Nature, Vol. 577, Issue 7792 https://doi.org/10.1038/s41586-020-1942-4	journal	January 2020
Analogue signal and image processing with large memristor crossbars Li, Can; Hu, Miao; Li, Yunning Nature Electronics, Vol. 1, Issue 1 https://doi.org/10.1038/s41928-017-0002-z	journal	December 2017
Fast, energy-efficient, robust, and reproducible mixed-signal neuromorphic classifier based on embedded NOR flash memory technology Guo, X.; Bayat, F. Merrikh; Bavandpour, M. 2017 IEEE International Electron Devices Meeting (IEDM) https://doi.org/10.1109/IEDM.2017.8268341	conference	December 2017
Reducing the Impact of Phase-Change Memory Conductance Drift on the Inference of large-scale Hardware Neural Networks Ambrogio, S.; Kumar, A.; Chen, A. 2019 IEEE International Electron Devices Meeting (IEDM) https://doi.org/10.1109/IEDM19573.2019.8993482	conference	December 2019
New second generation current conveyor with reduced parasitic resistance and bandpass filter application Sequin, F.; Fabre, A. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, Vol. 48, Issue 6 https://doi.org/10.1109/81.928161	journal	June 2001
Performance and Reliability Features of Advanced Nonvolatile Memories Based on Discrete Traps (Silicon Nanocrystals, SONOS) DeSalvo, B.; Gerardi, C.; vanSchaijk, R. IEEE Transactions on Device and Materials Reliability, Vol. 4, Issue 3 https://doi.org/10.1109/TDMR.2004.837209	journal	September 2004
Analysis and mitigation of parasitic resistance effects for analog in-memory neural network acceleration Xiao, T. Patrick; Feinberg, Ben; Rohan, Jacob N. Semiconductor Science and Technology, Vol. 36, Issue 11 https://doi.org/10.1088/1361-6641/ac271a	journal	October 2021
Access devices for 3D crosspoint memory Burr, Geoffrey W.; Shenoy, Rohit S.; Virwani, Kumar Journal of Vacuum Science & Technology B, Nanotechnology and Microelectronics: Materials, Processing, Measurement, and Phenomena, Vol. 32, Issue 4 https://doi.org/10.1116/1.4889999	journal	July 2014
Device-aware inference operations in SONOS nonvolatile memory arrays Bennett, Christopher H.; Xiao, T. Patrick; Dellana, Ryan 2020 IEEE International Reliability Physics Symposium (IRPS) https://doi.org/10.1109/IRPS45951.2020.9129313	conference	April 2020
Optimized programming algorithms for multilevel RRAM in hardware neural networks Milo, Valerio; Anzalone, Francesco; Zambelli, Cristian 2021 IEEE International Reliability Physics Symposium (IRPS) https://doi.org/10.1109/IRPS46558.2021.9405119	conference	March 2021
Analog architectures for neural network acceleration based on non-volatile memory Xiao, T. Patrick; Bennett, Christopher H.; Feinberg, Ben Applied Physics Reviews, Vol. 7, Issue 3 https://doi.org/10.1063/1.5143815	journal	September 2020
The missing memristor found Strukov, Dmitri B.; Snider, Gregory S.; Stewart, Duncan R. Nature, Vol. 453, Issue 7191 https://doi.org/10.1038/nature06932	journal	May 2008
ImageNet: A large-scale hierarchical image database Deng, Jia; Dong, Wei; Socher, Richard 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPR Workshops), 2009 IEEE Conference on Computer Vision and Pattern Recognition https://doi.org/10.1109/CVPR.2009.5206848	conference	June 2009
Accurate deep neural network inference using computational phase-change memory Joshi, Vinay; Le Gallo, Manuel; Haefeli, Simon Nature Communications, Vol. 11, Issue 1 https://doi.org/10.1038/s41467-020-16108-9	journal	May 2020
Noise Injection Adaption He, Zhezhi; Lin, Jie; Ewetz, Rickard Proceedings of the 56th Annual Design Automation Conference 2019 https://doi.org/10.1145/3316781.3317870	conference	June 2019
CxDNN: Hardware-software Compensation Methods for Deep Neural Networks on Resistive Crossbar Systems Jain, Shubham; Raghunathan, Anand ACM Transactions on Embedded Computing Systems, Vol. 18, Issue 6 https://doi.org/10.1145/3362035	journal	January 2020
40 nm Ultralow-Power Charge-Trap Embedded NVM Technology for IoT Applications Kouznetsov, Igor; Ramkumar, Krishnaswamy; Prabhakar, Venkatraman 2018 IEEE International Memory Workshop (IMW) https://doi.org/10.1109/IMW.2018.8388777	conference	May 2018
Deep Residual Learning for Image Recognition He, Kaiming; Zhang, Xiangyu; Ren, Shaoqing 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2016.90	conference	June 2016
MLPerf Inference Benchmark Reddi, Vijay Janapa; Cheng, Christine; Kanter, David 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA) https://doi.org/10.1109/ISCA45697.2020.00045	conference	May 2020
In-Datacenter Performance Analysis of a Tensor Processing Unit Jouppi, Norman P.; Borchers, Al; Boyle, Rick Proceedings of the 44th Annual International Symposium on Computer Architecture - ISCA '17 https://doi.org/10.1145/3079856.3080246	conference	January 2017
Xilinx Adaptive Compute Acceleration Platform Gaide, Brian; Gaitonde, Dinesh; Ravishankar, Chirag Proceedings of the 2019 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays https://doi.org/10.1145/3289602.3293906	conference	February 2019
Memristive Boltzmann machine: A hardware accelerator for combinatorial optimization and deep learning Bojnordi, Mahdi Nazm; Ipek, Engin 2016 IEEE International Symposium on High Performance Computer Architecture (HPCA) https://doi.org/10.1109/HPCA.2016.7446049	conference	March 2016
A domain-specific supercomputer for training deep neural networks Jouppi, Norman P.; Yoon, Doe Hyun; Kurian, George Communications of the ACM, Vol. 63, Issue 7 https://doi.org/10.1145/3360307	journal	June 2020
PRIME: a novel processing-in-memory architecture for neural network computation in ReRAM-based main memory Chi, Ping; Li, Shuangchen; Xu, Cong ACM SIGARCH Computer Architecture News, Vol. 44, Issue 3 https://doi.org/10.1145/3007787.3001140	journal	October 2016
A unified model for the flicker noise in metal-oxide-semiconductor field-effect transistors Hung, K. K.; Ko, P. K.; Hu, C. IEEE Transactions on Electron Devices, Vol. 37, Issue 3 https://doi.org/10.1109/16.47770	journal	March 1990
ISAAC: a convolutional neural network accelerator with in-situ analog arithmetic in crossbars Shafiee, Ali; Nag, Anirban; Muralimanohar, Naveen ACM SIGARCH Computer Architecture News, Vol. 44, Issue 3 https://doi.org/10.1145/3007787.3001139	journal	June 2016
Memory devices and applications for in-memory computing Sebastian, Abu; Le Gallo, Manuel; Khaddam-Aljameh, Riduan Nature Nanotechnology, Vol. 15, Issue 7 https://doi.org/10.1038/s41565-020-0655-z	journal	March 2020
Study of Charge Loss Mechanism of SONOS-Type Devices using Hot-Hole Erase and Methods to Improve the Charge Retention Lue, Hang-ting; Hsiao, Yi-hsuan; Shih, Yen-hao 2006 IEEE International Reliability Physics Symposium Proceedings https://doi.org/10.1109/RELPHY.2006.251273	conference	January 2006
Data retention in MLC NAND flash memory: Characterization, optimization, and recovery Cai, Yu; Luo, Yixin; Haratsch, Erich F. 2015 IEEE 21st International Symposium on High Performance Computer Architecture (HPCA) https://doi.org/10.1109/HPCA.2015.7056062	conference	February 2015
Physics of Semiconductor Devices Sze, S. M.; Ng, Kwok K. https://doi.org/10.1002/0470068329	book	January 2007
Resistive switching memories based on metal oxides: mechanisms, reliability and scaling Ielmini, Daniele Semiconductor Science and Technology, Vol. 31, Issue 6 https://doi.org/10.1088/0268-1242/31/6/063002	journal	May 2016
A Highly Dense, Low Power, Programmable Analog Vector-Matrix Multiplier: The FPAA Implementation Schlottmann, Craig R.; Hasler, Paul E. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, Vol. 1, Issue 3 https://doi.org/10.1109/JETCAS.2011.2165755	journal	September 2011

Similar Records

Ionizing radiation effects in SONOS-based neuromorphic inference accelerators

Journal Article · Thu Feb 11 00:00:00 EST 2021 · IEEE Transactions on Nuclear Science · OSTI ID:1842837

Patrick Xiao, T.; Bennett, Christopher H.; Agarwal, Sapan; +6 more

Using Floating-Gate Memory to Train Ideal Accuracy Neural Networks

Journal Article · Sat Jun 01 00:00:00 EDT 2019 · IEEE Journal on Exploratory Solid-State Computational Devices and Circuits · OSTI ID:1842837

Agarwal, Sapan; Garland, Diana; Niroula, John; +6 more

Analysis and mitigation of parasitic resistance effects for analog in-memory neural network acceleration

Journal Article · Thu Oct 14 00:00:00 EDT 2021 · Semiconductor Science and Technology · OSTI ID:1842837

Xiao, T. Patrick; Feinberg, Ben; Rohan, Jacob N.; +3 more

Related Subjects

97 MATHEMATICS AND COMPUTING
SONOS
charge trap memory
neuromorphic
neural network
analog
in-memory computing
inference accelerator.

Title: An Accurate, Error-Tolerant, and Energy-Efficient Neural Network Inference Engine Based on SONOS Analog Memory

Citation Formats

References (52)

Similar Records

Related Subjects