Deep learning as a tool for neural data analysis: Speech classification and cross-frequency coupling in human sensorimotor cortex

Journal Article · 16 September 2019 · PLoS Computational Biology (Online)

DOI: https://doi.org/10.1371/journal.pcbi.1007091 · OSTI ID:1572056

Livezey, Jesse A.  ^[1]; Bouchard, Kristofer E.  ^[1]; Chang, Edward F.  ^[2]

---

1. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States); Univ. of California, Berkeley, CA (United States)
2. Univ. of California, San Francisco, CA (United States)

A fundamental challenge in neuroscience is to understand what structure in the world is represented in spatially distributed patterns of neural activity from multiple single-trial measurements. This is often accomplished by learning a simple, linear transformations between neural features and features of the sensory stimuli or motor task. While successful in some early sensory processing areas, linear mappings are unlikely to be ideal tools for elucidating nonlinear, hierarchical representations of higher-order brain areas during complex tasks, such as the production of speech by humans. Here, we apply deep networks to predict produced speech syllables from a dataset of high gamma cortical surface electric potentials recorded from human sensorimotor cortex. We find that deep networks had higher decoding prediction accuracy compared to baseline models. Having established that deep networks extract more task relevant information from neural data sets relative to linear models (i.e., higher predictive accuracy), we next sought to demonstrate their utility as a data analysis tool for neuroscience. We first show that deep network's confusions revealed hierarchical latent structure in the neural data, which recapitulated the underlying articulatory nature of speech motor control. We next broadened the frequency features beyond high-gamma and identified a novel high-gamma-to-beta coupling during speech production. Finally, we used deep networks to compare task-relevant information in different neural frequency bands, and found that the high-gamma band contains the vast majority of information relevant for the speech prediction task, with little-to-no additional contribution from lower-frequency amplitudes. Together, these results demonstrate the utility of deep networks as a data analysis tool for basic and applied neuroscience.

• Research Organization: Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)
• Sponsoring Organization: USDOE Office of Science (SC)
• Grant/Contract Number: AC02-05CH11231
• OSTI ID: 1572056
• Journal Information: PLoS Computational Biology (Online), Journal Name: PLoS Computational Biology (Online) Journal Issue: 9 Vol. 15; ISSN 1553-7358
• Publisher: Public Library of ScienceCopyright Statement
• Country of Publication: United States
• Language: English