Analog hardware accelerators, which perform computation within a dense memory array, have the potential to overcome the major bottlenecks faced by digital hardware for data-heavy workloads such as deep learning. Exploiting the intrinsic computational advantages of memory arrays, however, has proven to be challenging principally due to the overhead imposed by the peripheral circuitry and due to the non-ideal properties of memory devices that play the role of the synapse. In this paper, we review the existing implementations of these accelerators for deep supervised learning, organizing our discussion around the different levels of the accelerator design hierarchy, with an emphasis on circuits and architecture. We also explore and consolidate the various approaches that have been proposed to address the critical challenges faced by analog accelerators, for both neural network inference and training, and highlight the key design tradeoffs underlying these techniques.
Xiao, T. Patrick, et al. "Analog architectures for neural network acceleration based on non-volatile memory." Applied Physics Reviews, vol. 7, no. 3, Jul. 2020. https://doi.org/10.1063/1.5143815
Xiao, T. Patrick, Bennett, Christopher H., Feinberg, Ben, Agarwal, Sapan, & Marinella, Matthew J. (2020). Analog architectures for neural network acceleration based on non-volatile memory. Applied Physics Reviews, 7(3). https://doi.org/10.1063/1.5143815
Xiao, T. Patrick, Bennett, Christopher H., Feinberg, Ben, et al., "Analog architectures for neural network acceleration based on non-volatile memory," Applied Physics Reviews 7, no. 3 (2020), https://doi.org/10.1063/1.5143815
@article{osti_2564777,
author = {Xiao, T. Patrick and Bennett, Christopher H. and Feinberg, Ben and Agarwal, Sapan and Marinella, Matthew J.},
title = {Analog architectures for neural network acceleration based on non-volatile memory},
annote = {Analog hardware accelerators, which perform computation within a dense memory array, have the potential to overcome the major bottlenecks faced by digital hardware for data-heavy workloads such as deep learning. Exploiting the intrinsic computational advantages of memory arrays, however, has proven to be challenging principally due to the overhead imposed by the peripheral circuitry and due to the non-ideal properties of memory devices that play the role of the synapse. In this paper, we review the existing implementations of these accelerators for deep supervised learning, organizing our discussion around the different levels of the accelerator design hierarchy, with an emphasis on circuits and architecture. We also explore and consolidate the various approaches that have been proposed to address the critical challenges faced by analog accelerators, for both neural network inference and training, and highlight the key design tradeoffs underlying these techniques.},
doi = {10.1063/1.5143815},
url = {https://www.osti.gov/biblio/2564777},
journal = {Applied Physics Reviews},
issn = {ISSN 1931-9401},
number = {3},
volume = {7},
place = {United States},
publisher = {American Institute of Physics (AIP)},
year = {2020},
month = {07}}
Ankit, Aayush; Hajj, Izzat El; Chalamalasetti, Sai Rahul
ASPLOS '19: Architectural Support for Programming Languages and Operating Systems, Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systemshttps://doi.org/10.1145/3297858.3304049
Proceedings of the 19th international conference on Architectural support for programming languages and operating systems - ASPLOS '14https://doi.org/10.1145/2541940.2541967
ASPDAC '19: 24th Asia and South Pacific Design Automation Conference, Proceedings of the 24th Asia and South Pacific Design Automation Conferencehttps://doi.org/10.1145/3287624.3287715