DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Phonetic acquisition in cortical dynamics, a computational approach

Abstract

Many computational theories have been developed to improve artificial phonetic classification performance from linguistic auditory streams. However, less attention has been given to psycholinguistic data and neurophysiological features recently found in cortical tissue. We focus on a context in which basic linguistic units-such as phonemes-are extracted and robustly classified by humans and other animals from complex acoustic streams in speech data. We are especially motivated by the fact that 8-month-old human infants can accomplish segmentation of words from fluent audio streams based exclusively on the statistical relationships between neighboring speech sounds without any kind of supervision. In this paper, we introduce a biologically inspired and fully unsupervised neurocomputational approach that incorporates key neurophysiological and anatomical cortical properties, including columnar organization, spontaneous micro-columnar formation, adaptation to contextual activations and Sparse Distributed Representations (SDRs) produced by means of partial N-Methyl-D-aspartic acid (NMDA) depolarization. Its feature abstraction capabilities show promising phonetic invariance and generalization attributes. Our model improves the performance of a Support Vector Machine (SVM) classifier for monosyllabic, disyllabic and trisyllabic word classification tasks in the presence of environmental disturbances such as white noise, reverberation, and pitch and voice variations. Furthermore, our approach emphasizes potential self-organizing cortical principles achieving improvement without anymore » kind of optimization guidance which could minimize hypothetical loss functions by means of-for example-backpropagation. Thus, our computational model outperforms multiresolution spectro-temporal auditory feature representations using only the statistical sequential structure immerse in the phonotactic rules of the input stream.« less

Authors:
ORCiD logo [1]; ORCiD logo [2];  [3];  [4];  [5];  [6]
  1. Univ. de Buenos Aires, Buenos Aires (Argentina)
  2. Argonne National Lab. (ANL), Lemont, IL (United States)
  3. Argonne National Lab. (ANL), Lemont, IL (United States); Loyola Univ. Chicago, Chicago, IL (United States)
  4. Inst. de Ciencias Humanas, Mendoza (Argentina)
  5. Univ. de Buenos Aires, Buenos Aires (Argentina); Inst. de Biología y Medicina Experimental-CONICET, Ciudad Autónoma de Buenos Aires (Argentina)
  6. Fraunhofer-Inst. fur Nachrichtentechnik Heinrich-Hertz-Inst. (Germany)
Publication Date:
Research Org.:
Argonne National Laboratory (ANL), Argonne, IL (United States)
Sponsoring Org.:
Argonne National Laboratory - Argonne Leadership Computing Facility; USDOE
OSTI Identifier:
1559970
Grant/Contract Number:  
AC02-06CH11357
Resource Type:
Accepted Manuscript
Journal Name:
PLoS ONE
Additional Journal Information:
Journal Volume: 14; Journal Issue: 6; Journal ID: ISSN 1932-6203
Publisher:
Public Library of Science
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; 60 APPLIED LIFE SCIENCES

Citation Formats

Dematties, Dario, Rizzi, Silvio, Thiruvathukal, George K., Wainselboim, Alejandro, Zanutto, B. Silvano, and Samek, Wojciech. Phonetic acquisition in cortical dynamics, a computational approach. United States: N. p., 2019. Web. doi:10.1371/journal.pone.0217966.
Dematties, Dario, Rizzi, Silvio, Thiruvathukal, George K., Wainselboim, Alejandro, Zanutto, B. Silvano, & Samek, Wojciech. Phonetic acquisition in cortical dynamics, a computational approach. United States. https://doi.org/10.1371/journal.pone.0217966
Dematties, Dario, Rizzi, Silvio, Thiruvathukal, George K., Wainselboim, Alejandro, Zanutto, B. Silvano, and Samek, Wojciech. Fri . "Phonetic acquisition in cortical dynamics, a computational approach". United States. https://doi.org/10.1371/journal.pone.0217966. https://www.osti.gov/servlets/purl/1559970.
@article{osti_1559970,
title = {Phonetic acquisition in cortical dynamics, a computational approach},
author = {Dematties, Dario and Rizzi, Silvio and Thiruvathukal, George K. and Wainselboim, Alejandro and Zanutto, B. Silvano and Samek, Wojciech},
abstractNote = {Many computational theories have been developed to improve artificial phonetic classification performance from linguistic auditory streams. However, less attention has been given to psycholinguistic data and neurophysiological features recently found in cortical tissue. We focus on a context in which basic linguistic units-such as phonemes-are extracted and robustly classified by humans and other animals from complex acoustic streams in speech data. We are especially motivated by the fact that 8-month-old human infants can accomplish segmentation of words from fluent audio streams based exclusively on the statistical relationships between neighboring speech sounds without any kind of supervision. In this paper, we introduce a biologically inspired and fully unsupervised neurocomputational approach that incorporates key neurophysiological and anatomical cortical properties, including columnar organization, spontaneous micro-columnar formation, adaptation to contextual activations and Sparse Distributed Representations (SDRs) produced by means of partial N-Methyl-D-aspartic acid (NMDA) depolarization. Its feature abstraction capabilities show promising phonetic invariance and generalization attributes. Our model improves the performance of a Support Vector Machine (SVM) classifier for monosyllabic, disyllabic and trisyllabic word classification tasks in the presence of environmental disturbances such as white noise, reverberation, and pitch and voice variations. Furthermore, our approach emphasizes potential self-organizing cortical principles achieving improvement without any kind of optimization guidance which could minimize hypothetical loss functions by means of-for example-backpropagation. Thus, our computational model outperforms multiresolution spectro-temporal auditory feature representations using only the statistical sequential structure immerse in the phonotactic rules of the input stream.},
doi = {10.1371/journal.pone.0217966},
journal = {PLoS ONE},
number = 6,
volume = 14,
place = {United States},
year = {Fri Jun 07 00:00:00 EDT 2019},
month = {Fri Jun 07 00:00:00 EDT 2019}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Citation Metrics:
Cited by: 2 works
Citation information provided by
Web of Science

Save / Share:

Works referenced in this record:

Perception of synthetic /ba/–/wa/ speech continuum by budgerigars ( Melopsittacus undulatus )
journal, September 1997

  • Dent, Micheal L.; Brittan-Powell, Elizabeth F.; Dooling, Robert J.
  • The Journal of the Acoustical Society of America, Vol. 102, Issue 3
  • DOI: 10.1121/1.420111

Active Properties of Neocortical Pyramidal Neuron Dendrites
journal, July 2013


Distributional regularity and phonotactic constraints are useful for segmentation
journal, October 1996


Specificity and timescales of cortical adaptation as inferences about natural movie statistics
journal, October 2016

  • Snow, Michoel; Coen-Cagli, Ruben; Schwartz, Odelia
  • Journal of Vision, Vol. 16, Issue 13
  • DOI: 10.1167/16.13.1

Enhanced discriminability at the phonetic boundaries for the place feature in macaques
journal, March 1983

  • Kuhl, Patricia K.; Padden, Denise M.
  • The Journal of the Acoustical Society of America, Vol. 73, Issue 3
  • DOI: 10.1121/1.389148

Speech perception by the chinchilla: voiced-voiceless distinction in alveolar plosive consonants
journal, October 1975


Self-organized formation of topologically correct feature maps
journal, January 1982


Columnar Transformations in Auditory Cortex? A Comparison to Visual and Somatosensory Cortices
journal, January 2003


Phoneme representation and classification in primary auditory cortex
journal, February 2008

  • Mesgarani, Nima; David, Stephen V.; Fritz, Jonathan B.
  • The Journal of the Acoustical Society of America, Vol. 123, Issue 2
  • DOI: 10.1121/1.2816572

Adaptive shift in the domain of negative stiffness during spontaneous oscillation by hair bundles from the internal ear
journal, November 2005

  • Le Goff, L.; Bozovic, D.; Hudspeth, A. J.
  • Proceedings of the National Academy of Sciences, Vol. 102, Issue 47
  • DOI: 10.1073/pnas.0508731102

The decade of the dendritic NMDA spike
journal, June 2010

  • Antic, Srdjan D.; Zhou, Wen-Liang; Moore, Anna R.
  • Journal of Neuroscience Research, Vol. 88, Issue 14
  • DOI: 10.1002/jnr.22444

The effects of distributional learning on rats' sensitivity to phonetic information.
journal, January 2006

  • Pons, Ferran
  • Journal of Experimental Psychology: Animal Behavior Processes, Vol. 32, Issue 1
  • DOI: 10.1037/0097-7403.32.1.97

Spectral shape analysis in the central auditory system
journal, January 1995

  • Kuansan Wang, ; Shamma, S. A.
  • IEEE Transactions on Speech and Audio Processing, Vol. 3, Issue 5
  • DOI: 10.1109/89.466657

Adaptation in Hair Cells
journal, March 2000


Role of experience for language-specific functional mappings of vowel sounds
journal, December 1998

  • Kluender, Keith R.; Lotto, Andrew J.; Holt, Lori L.
  • The Journal of the Acoustical Society of America, Vol. 104, Issue 6
  • DOI: 10.1121/1.423939

Organization of response areas in ferret primary auditory cortex
journal, February 1993

  • Shamma, S. A.; Fleshman, J. W.; Wiser, P. R.
  • Journal of Neurophysiology, Vol. 69, Issue 2
  • DOI: 10.1152/jn.1993.69.2.367

Two mechanisms for transducer adaptation in vertebrate hair cells
journal, October 2000

  • Holt, J. R.; Corey, D. P.
  • Proceedings of the National Academy of Sciences, Vol. 97, Issue 22
  • DOI: 10.1073/pnas.97.22.11730

Random synaptic feedback weights support error backpropagation for deep learning
journal, November 2016

  • Lillicrap, Timothy P.; Cownden, Daniel; Tweed, Douglas B.
  • Nature Communications, Vol. 7, Issue 1
  • DOI: 10.1038/ncomms13276

Receptive fields, binocular interaction and functional architecture in the cat's visual cortex
journal, January 1962


Vowel discrimination in cats: Acquisition, effects of stimulus level, and performance in noise
journal, June 1996

  • Hienz, Robert D.; Aleszczyk, Cynthia M.; May, Bradford J.
  • The Journal of the Acoustical Society of America, Vol. 99, Issue 6
  • DOI: 10.1121/1.414980

Cellular organization of cortical barrel columns is whisker-specific
journal, October 2013

  • Meyer, H. S.; Egger, R.; Guest, J. M.
  • Proceedings of the National Academy of Sciences, Vol. 110, Issue 47
  • DOI: 10.1073/pnas.1312691110

The columnar organization of the neocortex
journal, April 1997


Receptive fields and functional architecture of monkey striate cortex
journal, March 1968


Sensitivity of cat primary auditory cortex (Al) neurons to the direction and rate of frequency modulation
journal, February 1985


Modality and Topographic Properties of Single Neurons of Cat'S Somatic Sensory Cortex
journal, July 1957


Top-down knowledge supports the retrieval of lexical information from degraded speech
journal, June 2007


Role of cortical N-methyl-D-aspartate receptors in auditory sensory memory and mismatch negativity generation: implications for schizophrenia.
journal, October 1996

  • Javitt, D. C.; Steinschneider, M.; Schroeder, C. E.
  • Proceedings of the National Academy of Sciences, Vol. 93, Issue 21
  • DOI: 10.1073/pnas.93.21.11962

Experimentally induced visual projections into auditory thalamus and cortex
journal, December 1988


Perceptual compensation for coarticulation by Japanese quail ( Coturnix coturnix japonica )
journal, August 1997

  • Lotto, Andrew J.; Kluender, Keith R.; Holt, Lori L.
  • The Journal of the Acoustical Society of America, Vol. 102, Issue 2
  • DOI: 10.1121/1.419865

Why Neurons Have Thousands of Synapses, a Theory of Sequence Memory in Neocortex
journal, March 2016


Integration of iconic gestures and speech in left superior temporal areas boosts speech comprehension under adverse listening conditions
journal, January 2010


Expectancy Constraints in Degraded Speech Modulate the Language Comprehension Network
journal, June 2009


Functional topography of cat primary auditory cortex: distribution of integrated excitation
journal, November 1990


Statistical Learning by 8-Month-Old Infants
journal, December 1996


Experimental evidence for sparse firing in the neocortex
journal, June 2012


Contextual modulation and stimulus selectivity in extrastriate cortex
journal, November 2014


A stagewise rejective multiple test procedure based on a modified Bonferroni test
journal, January 1988


A map of visual space induced in primary auditory cortex
journal, November 1990


LIBSVM: A library for support vector machines
journal, April 2011

  • Chang, Chih-Chung; Lin, Chih-Jen
  • ACM Transactions on Intelligent Systems and Technology, Vol. 2, Issue 3
  • DOI: 10.1145/1961189.1961199

Toward an Integration of Deep Learning and Neuroscience
journal, September 2016

  • Marblestone, Adam H.; Wayne, Greg; Kording, Konrad P.
  • Frontiers in Computational Neuroscience, Vol. 10
  • DOI: 10.3389/fncom.2016.00094

Induction of visual orientation modules in auditory cortex
journal, April 2000

  • Sharma, Jitendra; Angelucci, Alessandra; Sur, Mriganka
  • Nature, Vol. 404, Issue 6780
  • DOI: 10.1038/35009043

Multiresolution spectrotemporal analysis of complex sounds
journal, August 2005

  • Chi, Taishih; Ru, Powen; Shamma, Shihab A.
  • The Journal of the Acoustical Society of America, Vol. 118, Issue 2
  • DOI: 10.1121/1.1945807

Continuous Online Sequence Learning with an Unsupervised Neural Network Model
journal, November 2016

  • Cui, Yuwei; Ahmad, Subutai; Hawkins, Jeff
  • Neural Computation, Vol. 28, Issue 11
  • DOI: 10.1162/NECO_a_00893

The Design and Implementation of FFTW3
journal, February 2005


Multiple brain signatures of integration in the comprehension of degraded speech
journal, March 2011


Experimental evidence for sparse firing in the neocortex.
text, January 2012

  • Barth, Alison L.; Poulet, James F. A.
  • Carnegie Mellon University
  • DOI: 10.1184/r1/6098609

Perceptual Compensation for Coarticulation by Japanese Quail (Coturnix coturnix japonica)
text, January 1997


Role of Experience for Language-Specific Functional Mappings of Vowel Sounds
text, January 2018


Role of experience for language-specific functional mappings of vowel sounds
text, January 1998


Experimental evidence for sparse firing in the neocortex.
text, January 2018


Perceptual compensation for coarticulation by Japanese quail (Coturnix coturnix japonica)
text, January 1997


Datasets used to train and test the Cortical Spectro-Temporal Model (CSTM).
dataset, January 2019


Towards an integration of deep learning and neuroscience
text, January 2016


Brainstem inputs to the ferret medial geniculate nucleus and the effect of early deafferentation on novel retinal projections to the auditory thalamus
journal, October 1998


The decade of the dendritic NMDA spike
journal, June 2010

  • Antic, Srdjan D.; Zhou, Wen-Liang; Moore, Anna R.
  • Journal of Neuroscience Research, Vol. 88, Issue 14
  • DOI: 10.1002/jnr.22444

Sensitivity of cat primary auditory cortex (Al) neurons to the direction and rate of frequency modulation
journal, February 1985


Sensitivity of neurons in cat primary auditory cortex to tones and frequency-modulated stimuli. I: Effects of variation of stimulus parameters
journal, November 1992


Top-down knowledge supports the retrieval of lexical information from degraded speech
journal, June 2007


Integration of iconic gestures and speech in left superior temporal areas boosts speech comprehension under adverse listening conditions
journal, January 2010


Multiple brain signatures of integration in the comprehension of degraded speech
journal, March 2011


Contextual modulation and stimulus selectivity in extrastriate cortex
journal, November 2014


The effects of distributional learning on rats' sensitivity to phonetic information.
journal, January 2006

  • Pons, Ferran
  • Journal of Experimental Psychology: Animal Behavior Processes, Vol. 32, Issue 1
  • DOI: 10.1037/0097-7403.32.1.97

Induction of visual orientation modules in auditory cortex
journal, April 2000

  • Sharma, Jitendra; Angelucci, Alessandra; Sur, Mriganka
  • Nature, Vol. 404, Issue 6780
  • DOI: 10.1038/35009043

The functional organization of cortical feedback inputs to primary visual cortex
journal, April 2018


Adaptive shift in the domain of negative stiffness during spontaneous oscillation by hair bundles from the internal ear
journal, November 2005

  • Le Goff, L.; Bozovic, D.; Hudspeth, A. J.
  • Proceedings of the National Academy of Sciences, Vol. 102, Issue 47
  • DOI: 10.1073/pnas.0508731102

Mechanisms of noise robust representation of speech in primary auditory cortex
journal, April 2014

  • Mesgarani, N.; David, S. V.; Fritz, J. B.
  • Proceedings of the National Academy of Sciences, Vol. 111, Issue 18
  • DOI: 10.1073/pnas.1318017111

Role of cortical N-methyl-D-aspartate receptors in auditory sensory memory and mismatch negativity generation: implications for schizophrenia.
journal, October 1996

  • Javitt, D. C.; Steinschneider, M.; Schroeder, C. E.
  • Proceedings of the National Academy of Sciences, Vol. 93, Issue 21
  • DOI: 10.1073/pnas.93.21.11962

Two mechanisms for transducer adaptation in vertebrate hair cells
journal, October 2000

  • Holt, J. R.; Corey, D. P.
  • Proceedings of the National Academy of Sciences, Vol. 97, Issue 22
  • DOI: 10.1073/pnas.97.22.11730

A stagewise rejective multiple test procedure based on a modified Bonferroni test
journal, January 1988


Columnar Transformations in Auditory Cortex? A Comparison to Visual and Somatosensory Cortices
journal, January 2003


Expectancy Constraints in Degraded Speech Modulate the Language Comprehension Network
journal, June 2009


Spectral shape analysis in the central auditory system
journal, January 1995

  • Kuansan Wang, ; Shamma, S. A.
  • IEEE Transactions on Speech and Audio Processing, Vol. 3, Issue 5
  • DOI: 10.1109/89.466657

Receptive fields, binocular interaction and functional architecture in the cat's visual cortex
journal, January 1962


Receptive fields and functional architecture of monkey striate cortex
journal, March 1968


Multiresolution spectrotemporal analysis of complex sounds
journal, August 2005

  • Chi, Taishih; Ru, Powen; Shamma, Shihab A.
  • The Journal of the Acoustical Society of America, Vol. 118, Issue 2
  • DOI: 10.1121/1.1945807

Enhanced discriminability at the phonetic boundaries for the place feature in macaques
journal, March 1983

  • Kuhl, Patricia K.; Padden, Denise M.
  • The Journal of the Acoustical Society of America, Vol. 73, Issue 3
  • DOI: 10.1121/1.389148

Vowel discrimination in cats: Acquisition, effects of stimulus level, and performance in noise
journal, June 1996

  • Hienz, Robert D.; Aleszczyk, Cynthia M.; May, Bradford J.
  • The Journal of the Acoustical Society of America, Vol. 99, Issue 6
  • DOI: 10.1121/1.414980

Perceptual compensation for coarticulation by Japanese quail ( Coturnix coturnix japonica )
journal, August 1997

  • Lotto, Andrew J.; Kluender, Keith R.; Holt, Lori L.
  • The Journal of the Acoustical Society of America, Vol. 102, Issue 2
  • DOI: 10.1121/1.419865

Role of experience for language-specific functional mappings of vowel sounds
journal, December 1998

  • Kluender, Keith R.; Lotto, Andrew J.; Holt, Lori L.
  • The Journal of the Acoustical Society of America, Vol. 104, Issue 6
  • DOI: 10.1121/1.423939

Speech perception by the chinchilla: voiced-voiceless distinction in alveolar plosive consonants
journal, October 1975


A map of visual space induced in primary auditory cortex
journal, November 1990


Experimentally induced visual projections into auditory thalamus and cortex
journal, December 1988


Active Properties of Neocortical Pyramidal Neuron Dendrites
journal, July 2013


Adaptation in Hair Cells
journal, March 2000


Modality and Topographic Properties of Single Neurons of Cat'S Somatic Sensory Cortex
journal, July 1957


Functional topography of cat primary auditory cortex: distribution of integrated excitation
journal, November 1990


Organization of response areas in ferret primary auditory cortex
journal, February 1993

  • Shamma, S. A.; Fleshman, J. W.; Wiser, P. R.
  • Journal of Neurophysiology, Vol. 69, Issue 2
  • DOI: 10.1152/jn.1993.69.2.367

Experimental evidence for sparse firing in the neocortex.
text, January 2012

  • Barth, Alison L.; Poulet, James F. A.
  • Carnegie Mellon University
  • DOI: 10.1184/r1/6098609

Visual projections routed to the auditory pathway in ferrets: receptive fields of visual neurons in primary auditory cortex
journal, September 1992


Functional Integration across Brain Regions Improves Speech Perception under Adverse Listening Conditions
journal, February 2007


Continuous online sequence learning with an unsupervised neural network model
text, January 2015


Complementary control of sensory adaptation by two types of cortical interneurons
journal, October 2015

  • Natan, Ryan G.; Briguglio, John J.; Mwilambwe-Tshilobo, Laetitia
  • eLife, Vol. 4
  • DOI: 10.7554/elife.09868

Towards deep learning with segregated dendrites
journal, December 2017

  • Guerguiev, Jordan; Lillicrap, Timothy P.; Richards, Blake A.
  • eLife, Vol. 6
  • DOI: 10.7554/elife.22901

Datasets used to train and test the Cortical Spectro-Temporal Model (CSTM).
dataset, January 2019


Experimental Results and Appendices: Cortical Spectro-Temporal Model (CSTM).
dataset, January 2019