Analog hardware accelerators, which perform computation within a dense memory array, have the potential to overcome the major bottlenecks faced by digital hardware for data-heavy workloads such as deep learning. Exploiting the intrinsic computational advantages of memory arrays, however, has proven to be challenging principally due to the overhead imposed by the peripheral circuitry and due to the non-ideal properties of memory devices that play the role of the synapse. We review the existing implementations of these accelerators for deep supervised learning, organizing our discussion around the different levels of the accelerator design hierarchy, with an emphasis on circuits and architecture. We explore and consolidate the various approaches that have been proposed to address the critical challenges faced by analog accelerators, for both neural network inference and training, and highlight the key design trade-offs underlying these techniques.
Xiao, T. Patrick, et al. "Analog architectures for neural network acceleration based on non-volatile memory." Applied Physics Reviews, vol. 7, no. 3, Jul. 2020. https://doi.org/10.1063/1.5143815
Xiao, T. Patrick, Bennett, Christopher H., Feinberg, Ben, Agarwal, Sapan, & Marinella, Matthew J. (2020). Analog architectures for neural network acceleration based on non-volatile memory. Applied Physics Reviews, 7(3). https://doi.org/10.1063/1.5143815
Xiao, T. Patrick, Bennett, Christopher H., Feinberg, Ben, et al., "Analog architectures for neural network acceleration based on non-volatile memory," Applied Physics Reviews 7, no. 3 (2020), https://doi.org/10.1063/1.5143815
@article{osti_2564777,
author = {Xiao, T. Patrick and Bennett, Christopher H. and Feinberg, Ben and Agarwal, Sapan and Marinella, Matthew J.},
title = {Analog architectures for neural network acceleration based on non-volatile memory},
annote = {Analog hardware accelerators, which perform computation within a dense memory array, have the potential to overcome the major bottlenecks faced by digital hardware for data-heavy workloads such as deep learning. Exploiting the intrinsic computational advantages of memory arrays, however, has proven to be challenging principally due to the overhead imposed by the peripheral circuitry and due to the non-ideal properties of memory devices that play the role of the synapse. We review the existing implementations of these accelerators for deep supervised learning, organizing our discussion around the different levels of the accelerator design hierarchy, with an emphasis on circuits and architecture. We explore and consolidate the various approaches that have been proposed to address the critical challenges faced by analog accelerators, for both neural network inference and training, and highlight the key design trade-offs underlying these techniques.},
doi = {10.1063/1.5143815},
url = {https://www.osti.gov/biblio/2564777},
journal = {Applied Physics Reviews},
issn = {ISSN 1931-9401},
number = {3},
volume = {7},
place = {United States},
publisher = {American Institute of Physics},
year = {2020},
month = {07}}