skip to main content


Title: High-Resolution, Non-Invasive Imaging of Upper Vocal Tract Articulators Compatible with Human Brain Recordings

A complete neurobiological understanding of speech motor control requires determination of the relationship between simultaneously recorded neural activity and the kinematics of the lips, jaw, tongue, and larynx. Many speech articulators are internal to the vocal tract, and therefore simultaneously tracking the kinematics of all articulators is nontrivial-especially in the context of human electrophysiology recordings. Here, we describe a noninvasive, multi-modal imaging system to monitor vocal tract kinematics, demonstrate this system in six speakers during production of nine American English vowels, and provide new analysis of such data. Classification and regression analysis revealed considerable variability in the articulator-to-acoustic relationship across speakers. Non-negative matrix factorization extracted basis sets capturing vocal tract shapes allowing for higher vowel classification accuracy than traditional methods. Statistical speech synthesis generated speech from vocal tract measurements, and we demonstrate perceptual identification. We demonstrate the capacity to predict lip kinematics from ventral sensorimotor cortical activity. These results demonstrate a multi-modal system to non-invasively monitor articulator kinematics during speech production, describe novel analytic methods for relating kinematic data to speech acoustics, and provide the first decoding of speech kinematics from electrocorticography. These advances will be critical for understanding the cortical basis of speech production and the creation of vocalmore » prosthetics.« less
 [1] ;  [2] ;  [2] ;  [2] ;  [2] ;  [3] ;  [2] ;  [4]
  1. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States); Univ. of California San Francisco (UCSF), San Francisco, CA (United States)
  2. Univ. of California San Francisco (UCSF), San Francisco, CA (United States)
  3. Univ. of California, Berkeley, CA (United States)
  4. The Univ. of Western Ontario, Ontario (Canada)
Publication Date:
Grant/Contract Number:
Accepted Manuscript
Journal Name:
Additional Journal Information:
Journal Volume: 11; Journal Issue: 3; Journal ID: ISSN 1932-6203
Public Library of Science
Research Org:
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Sponsoring Org:
USDOE Office of Science (SC)
Country of Publication:
United States
60 APPLIED LIFE SCIENCES; vowels; speech; tongue; acoustics; kinematics; speech signal processing; bioacoustics; lips
OSTI Identifier: