Decoding behavior, perception or cognitive state directly from neural signals is critical for brain–computer interface research and an important tool for systems neuroscience. In the last decade, deep learning has become the state-of-the-art method in many machine learning tasks ranging from speech recognition to image segmentation. The success of deep networks in other domains has led to a new wave of applications in neuroscience. In this article, we review deep learning approaches to neural decoding. Here, we describe the architectures used for extracting useful features from neural recording modalities ranging from spikes to functional magnetic resonance imaging. Furthermore, we explore how deep learning has been leveraged to predict common outputs including movement, speech and vision, with a focus on how pretrained deep networks can be incorporated as priors for complex decoding targets like acoustic speech or images. Deep learning has been shown to be a useful tool for improving the accuracy and flexibility of neural decoding across a wide range of tasks, and we point out areas for future scientific development.
Livezey, Jesse A. and Glaser, Joshua I.. "Deep learning approaches for neural decoding across architectures and recording modalities." Briefings in Bioinformatics, vol. 22, no. 2, Dec. 2020. https://doi.org/10.1093/bib/bbaa355
Livezey, Jesse A., & Glaser, Joshua I. (2020). Deep learning approaches for neural decoding across architectures and recording modalities. Briefings in Bioinformatics, 22(2). https://doi.org/10.1093/bib/bbaa355
Livezey, Jesse A., and Glaser, Joshua I., "Deep learning approaches for neural decoding across architectures and recording modalities," Briefings in Bioinformatics 22, no. 2 (2020), https://doi.org/10.1093/bib/bbaa355
@article{osti_1826323,
author = {Livezey, Jesse A. and Glaser, Joshua I.},
title = {Deep learning approaches for neural decoding across architectures and recording modalities},
annote = {Decoding behavior, perception or cognitive state directly from neural signals is critical for brain–computer interface research and an important tool for systems neuroscience. In the last decade, deep learning has become the state-of-the-art method in many machine learning tasks ranging from speech recognition to image segmentation. The success of deep networks in other domains has led to a new wave of applications in neuroscience. In this article, we review deep learning approaches to neural decoding. Here, we describe the architectures used for extracting useful features from neural recording modalities ranging from spikes to functional magnetic resonance imaging. Furthermore, we explore how deep learning has been leveraged to predict common outputs including movement, speech and vision, with a focus on how pretrained deep networks can be incorporated as priors for complex decoding targets like acoustic speech or images. Deep learning has been shown to be a useful tool for improving the accuracy and flexibility of neural decoding across a wide range of tasks, and we point out areas for future scientific development.},
doi = {10.1093/bib/bbaa355},
url = {https://www.osti.gov/biblio/1826323},
journal = {Briefings in Bioinformatics},
issn = {ISSN 1467-5463},
number = {2},
volume = {22},
place = {United States},
publisher = {Oxford University Press},
year = {2020},
month = {12}}
Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)
Sponsoring Organization:
USDOE Office of Science (SC), High Energy Physics (HEP) (SC-25); USDOE Laboratory Directed Research and Development (LDRD) Program; National Science Foundation (NSF); Gatsby Foundation
Grant/Contract Number:
AC02-05CH11231
OSTI ID:
1826323
Journal Information:
Briefings in Bioinformatics, Journal Name: Briefings in Bioinformatics Journal Issue: 2 Vol. 22; ISSN 1467-5463