Title: Deep learning as a tool for neural data analysis: Speech classification and cross-frequency coupling in human sensorimotor cortex

Abstract

A fundamental challenge in neuroscience is to understand what structure in the world is represented in spatially distributed patterns of neural activity from multiple single-trial measurements. This is often accomplished by learning a simple, linear transformations between neural features and features of the sensory stimuli or motor task. While successful in some early sensory processing areas, linear mappings are unlikely to be ideal tools for elucidating nonlinear, hierarchical representations of higher-order brain areas during complex tasks, such as the production of speech by humans. Here, we apply deep networks to predict produced speech syllables from a dataset of high gamma cortical surface electric potentials recorded from human sensorimotor cortex. We find that deep networks had higher decoding prediction accuracy compared to baseline models. Having established that deep networks extract more task relevant information from neural data sets relative to linear models (i.e., higher predictive accuracy), we next sought to demonstrate their utility as a data analysis tool for neuroscience. We first show that deep network's confusions revealed hierarchical latent structure in the neural data, which recapitulated the underlying articulatory nature of speech motor control. We next broadened the frequency features beyond high-gamma and identified a novel high-gamma-to-beta coupling during speechmore » production. Finally, we used deep networks to compare task-relevant information in different neural frequency bands, and found that the high-gamma band contains the vast majority of information relevant for the speech prediction task, with little-to-no additional contribution from lower-frequency amplitudes. Together, these results demonstrate the utility of deep networks as a data analysis tool for basic and applied neuroscience.« less

Authors:

Livezey, Jesse A.  ^[1];

• Search DOE PAGES for author "Livezey, Jesse A."
• Search DOE PAGES for ORCID "0000-0003-0494-8758"
• Search orcid.org for ORCID "0000-0003-0494-8758"

Bouchard, Kristofer E.  ^[1];

• Search DOE PAGES for author "Bouchard, Kristofer E."
• Search DOE PAGES for ORCID "0000-0002-1974-4603"
• Search orcid.org for ORCID "0000-0002-1974-4603"

Chang, Edward F.  ^[2]

• Search DOE PAGES for author "Chang, Edward F."
• Search DOE PAGES for ORCID "0000-0003-2480-4700"
• Search orcid.org for ORCID "0000-0003-2480-4700"

---

+ Show Author Affiliations

1. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States); Univ. of California, Berkeley, CA (United States)
2. Univ. of California, San Francisco, CA (United States)

Publication Date:

2019-09-16

Research Org.:

Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)

Sponsoring Org.:

USDOE Office of Science (SC)

OSTI Identifier:

1572056

Grant/Contract Number:  

AC02-05CH11231

Resource Type:

Accepted Manuscript

Journal Name:

PLoS Computational Biology (Online)

Additional Journal Information:

Journal Name: PLoS Computational Biology (Online); Journal Volume: 15; Journal Issue: 9; Journal ID: ISSN 1553-7358

Publisher:

Public Library of Science

Country of Publication:

United States

Language:

English

Subject:

60 APPLIED LIFE SCIENCES