Phonetic acquisition in cortical dynamics, a computational approach

Dematties, Dario; Rizzi, Silvio; Thiruvathukal, George K.; Wainselboim, Alejandro; Zanutto, B. Silvano; Samek, Wojciech

doi:10.1371/journal.pone.0217966

Title: Phonetic acquisition in cortical dynamics, a computational approach

Journal Article · Fri Jun 07 00:00:00 EDT 2019 · PLoS ONE

DOI:https://doi.org/10.1371/journal.pone.0217966· OSTI ID:1559970

^[1];

^[2]; Thiruvathukal, George K. ^[3]; Wainselboim, Alejandro ^[4]; Zanutto, B. Silvano ^[5]; Samek, Wojciech ^[6]

Univ. de Buenos Aires, Buenos Aires (Argentina)
Argonne National Lab. (ANL), Lemont, IL (United States)
Argonne National Lab. (ANL), Lemont, IL (United States); Loyola Univ. Chicago, Chicago, IL (United States)
Inst. de Ciencias Humanas, Mendoza (Argentina)
Univ. de Buenos Aires, Buenos Aires (Argentina); Inst. de Biología y Medicina Experimental-CONICET, Ciudad Autónoma de Buenos Aires (Argentina)
Fraunhofer-Inst. fur Nachrichtentechnik Heinrich-Hertz-Inst. (Germany)

Many computational theories have been developed to improve artificial phonetic classification performance from linguistic auditory streams. However, less attention has been given to psycholinguistic data and neurophysiological features recently found in cortical tissue. We focus on a context in which basic linguistic units-such as phonemes-are extracted and robustly classified by humans and other animals from complex acoustic streams in speech data. We are especially motivated by the fact that 8-month-old human infants can accomplish segmentation of words from fluent audio streams based exclusively on the statistical relationships between neighboring speech sounds without any kind of supervision. In this paper, we introduce a biologically inspired and fully unsupervised neurocomputational approach that incorporates key neurophysiological and anatomical cortical properties, including columnar organization, spontaneous micro-columnar formation, adaptation to contextual activations and Sparse Distributed Representations (SDRs) produced by means of partial N-Methyl-D-aspartic acid (NMDA) depolarization. Its feature abstraction capabilities show promising phonetic invariance and generalization attributes. Our model improves the performance of a Support Vector Machine (SVM) classifier for monosyllabic, disyllabic and trisyllabic word classification tasks in the presence of environmental disturbances such as white noise, reverberation, and pitch and voice variations. Furthermore, our approach emphasizes potential self-organizing cortical principles achieving improvement without any kind of optimization guidance which could minimize hypothetical loss functions by means of-for example-backpropagation. Thus, our computational model outperforms multiresolution spectro-temporal auditory feature representations using only the statistical sequential structure immerse in the phonotactic rules of the input stream.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Argonne National Laboratory (ANL), Argonne, IL (United States)

Sponsoring Organization:: Argonne National Laboratory - Argonne Leadership Computing Facility; USDOE

Grant/Contract Number:: AC02-06CH11357

OSTI ID:: 1559970

Journal Information:: PLoS ONE, Vol. 14, Issue 6; ISSN 1932-6203

Publisher:: Public Library of ScienceCopyright Statement

Country of Publication:: United States

Language:: English

Citation Metrics:

Cited by: 2 works

Citation information provided by
Web of Science

References (67)

Perception of synthetic /ba/–/wa/ speech continuum by budgerigars ( Melopsittacus undulatus ) Dent, Micheal L.; Brittan-Powell, Elizabeth F.; Dooling, Robert J. The Journal of the Acoustical Society of America, Vol. 102, Issue 3 https://doi.org/10.1121/1.420111	journal	September 1997
Active Properties of Neocortical Pyramidal Neuron Dendrites Major, Guy; Larkum, Matthew E.; Schiller, Jackie Annual Review of Neuroscience, Vol. 36, Issue 1 https://doi.org/10.1146/annurev-neuro-062111-150343	journal	July 2013
Distributional regularity and phonotactic constraints are useful for segmentation Brent, Michael R.; Cartwright, Timothy A. Cognition, Vol. 61, Issue 1-2 https://doi.org/10.1016/S0010-0277(96)00719-6	journal	October 1996
Specificity and timescales of cortical adaptation as inferences about natural movie statistics Snow, Michoel; Coen-Cagli, Ruben; Schwartz, Odelia Journal of Vision, Vol. 16, Issue 13 https://doi.org/10.1167/16.13.1	journal	October 2016
Enhanced discriminability at the phonetic boundaries for the place feature in macaques Kuhl, Patricia K.; Padden, Denise M. The Journal of the Acoustical Society of America, Vol. 73, Issue 3 https://doi.org/10.1121/1.389148	journal	March 1983
Speech perception by the chinchilla: voiced-voiceless distinction in alveolar plosive consonants Kuhl, P.; Miller, J. Science, Vol. 190, Issue 4209 https://doi.org/10.1126/science.1166301	journal	October 1975
Self-organized formation of topologically correct feature maps Kohonen, Teuvo Biological Cybernetics, Vol. 43, Issue 1 https://doi.org/10.1007/BF00337288	journal	January 1982
Columnar Transformations in Auditory Cortex? A Comparison to Visual and Somatosensory Cortices Linden, J. F. Cerebral Cortex, Vol. 13, Issue 1 https://doi.org/10.1093/cercor/13.1.83	journal	January 2003
Phoneme representation and classification in primary auditory cortex Mesgarani, Nima; David, Stephen V.; Fritz, Jonathan B. The Journal of the Acoustical Society of America, Vol. 123, Issue 2 https://doi.org/10.1121/1.2816572	journal	February 2008
Adaptive shift in the domain of negative stiffness during spontaneous oscillation by hair bundles from the internal ear Le Goff, L.; Bozovic, D.; Hudspeth, A. J. Proceedings of the National Academy of Sciences, Vol. 102, Issue 47 https://doi.org/10.1073/pnas.0508731102	journal	November 2005
The decade of the dendritic NMDA spike Antic, Srdjan D.; Zhou, Wen-Liang; Moore, Anna R. Journal of Neuroscience Research, Vol. 88, Issue 14 https://doi.org/10.1002/jnr.22444	journal	June 2010
The effects of distributional learning on rats' sensitivity to phonetic information. Pons, Ferran Journal of Experimental Psychology: Animal Behavior Processes, Vol. 32, Issue 1 https://doi.org/10.1037/0097-7403.32.1.97	journal	January 2006
Spectral shape analysis in the central auditory system IEEE Transactions on Speech and Audio Processing, Vol. 3, Issue 5 https://doi.org/10.1109/89.466657	journal	January 1995
Adaptation in Hair Cells Eatock, Ruth Anne Annual Review of Neuroscience, Vol. 23, Issue 1 https://doi.org/10.1146/annurev.neuro.23.1.285	journal	March 2000
Role of experience for language-specific functional mappings of vowel sounds Kluender, Keith R.; Lotto, Andrew J.; Holt, Lori L. The Journal of the Acoustical Society of America, Vol. 104, Issue 6 https://doi.org/10.1121/1.423939	journal	December 1998
Organization of response areas in ferret primary auditory cortex Shamma, S. A.; Fleshman, J. W.; Wiser, P. R. Journal of Neurophysiology, Vol. 69, Issue 2 https://doi.org/10.1152/jn.1993.69.2.367	journal	February 1993
Two mechanisms for transducer adaptation in vertebrate hair cells Holt, J. R.; Corey, D. P. Proceedings of the National Academy of Sciences, Vol. 97, Issue 22 https://doi.org/10.1073/pnas.97.22.11730	journal	October 2000
Random synaptic feedback weights support error backpropagation for deep learning Lillicrap, Timothy P.; Cownden, Daniel; Tweed, Douglas B. Nature Communications, Vol. 7, Issue 1 https://doi.org/10.1038/ncomms13276	journal	November 2016
Receptive fields, binocular interaction and functional architecture in the cat's visual cortex Hubel, D. H.; Wiesel, T. N. The Journal of Physiology, Vol. 160, Issue 1 https://doi.org/10.1113/jphysiol.1962.sp006837	journal	January 1962
Vowel discrimination in cats: Acquisition, effects of stimulus level, and performance in noise Hienz, Robert D.; Aleszczyk, Cynthia M.; May, Bradford J. The Journal of the Acoustical Society of America, Vol. 99, Issue 6 https://doi.org/10.1121/1.414980	journal	June 1996
Cellular organization of cortical barrel columns is whisker-specific Meyer, H. S.; Egger, R.; Guest, J. M. Proceedings of the National Academy of Sciences, Vol. 110, Issue 47 https://doi.org/10.1073/pnas.1312691110	journal	October 2013
The columnar organization of the neocortex Mountcastle, V. Brain, Vol. 120, Issue 4 https://doi.org/10.1093/brain/120.4.701	journal	April 1997
Receptive fields and functional architecture of monkey striate cortex Hubel, D. H.; Wiesel, T. N. The Journal of Physiology, Vol. 195, Issue 1 https://doi.org/10.1113/jphysiol.1968.sp008455	journal	March 1968
Sensitivity of cat primary auditory cortex (Al) neurons to the direction and rate of frequency modulation Mendelson, J. R.; Cynader, M. S. Brain Research, Vol. 327, Issue 1-2 https://doi.org/10.1016/0006-8993(85)91530-6	journal	February 1985
Modality and Topographic Properties of Single Neurons of Cat'S Somatic Sensory Cortex Mountcastle, Vernon B. Journal of Neurophysiology, Vol. 20, Issue 4 https://doi.org/10.1152/jn.1957.20.4.408	journal	July 1957
Top-down knowledge supports the retrieval of lexical information from degraded speech Hannemann, R.; Obleser, J.; Eulitz, C. Brain Research, Vol. 1153 https://doi.org/10.1016/j.brainres.2007.03.069	journal	June 2007
Role of cortical N-methyl-D-aspartate receptors in auditory sensory memory and mismatch negativity generation: implications for schizophrenia. Javitt, D. C.; Steinschneider, M.; Schroeder, C. E. Proceedings of the National Academy of Sciences, Vol. 93, Issue 21 https://doi.org/10.1073/pnas.93.21.11962	journal	October 1996
Experimentally induced visual projections into auditory thalamus and cortex Sur, M.; Garraghty, P.; Roe, A. Science, Vol. 242, Issue 4884 https://doi.org/10.1126/science.2462279	journal	December 1988
Perceptual compensation for coarticulation by Japanese quail ( Coturnix coturnix japonica ) Lotto, Andrew J.; Kluender, Keith R.; Holt, Lori L. The Journal of the Acoustical Society of America, Vol. 102, Issue 2 https://doi.org/10.1121/1.419865	journal	August 1997
Why Neurons Have Thousands of Synapses, a Theory of Sequence Memory in Neocortex Hawkins, Jeff; Ahmad, Subutai Frontiers in Neural Circuits, Vol. 10 https://doi.org/10.3389/fncir.2016.00023	journal	March 2016
Integration of iconic gestures and speech in left superior temporal areas boosts speech comprehension under adverse listening conditions Holle, Henning; Obleser, Jonas; Rueschemeyer, Shirley-Ann NeuroImage, Vol. 49, Issue 1 https://doi.org/10.1016/j.neuroimage.2009.08.058	journal	January 2010
Expectancy Constraints in Degraded Speech Modulate the Language Comprehension Network Obleser, J.; Kotz, S. A. Cerebral Cortex, Vol. 20, Issue 3 https://doi.org/10.1093/cercor/bhp128	journal	June 2009
Functional topography of cat primary auditory cortex: distribution of integrated excitation Schreiner, C. E.; Mendelson, J. R. Journal of Neurophysiology, Vol. 64, Issue 5 https://doi.org/10.1152/jn.1990.64.5.1442	journal	November 1990
Statistical Learning by 8-Month-Old Infants Saffran, J. R.; Aslin, R. N.; Newport, E. L. Science, Vol. 274, Issue 5294 https://doi.org/10.1126/science.274.5294.1926	journal	December 1996
Experimental evidence for sparse firing in the neocortex Barth, Alison L.; Poulet, James F. A. Trends in Neurosciences, Vol. 35, Issue 6 https://doi.org/10.1016/j.tins.2012.03.008	journal	June 2012
Contextual modulation and stimulus selectivity in extrastriate cortex Krause, Matthew R.; Pack, Christopher C. Vision Research, Vol. 104 https://doi.org/10.1016/j.visres.2014.10.006	journal	November 2014
A stagewise rejective multiple test procedure based on a modified Bonferroni test Hommel, G. Biometrika, Vol. 75, Issue 2 https://doi.org/10.1093/biomet/75.2.383	journal	January 1988
A map of visual space induced in primary auditory cortex Roe, A.; Pallas, S.; Hahm, J. Science, Vol. 250, Issue 4982 https://doi.org/10.1126/science.2237432	journal	November 1990
LIBSVM: A library for support vector machines Chang, Chih-Chung; Lin, Chih-Jen ACM Transactions on Intelligent Systems and Technology, Vol. 2, Issue 3 https://doi.org/10.1145/1961189.1961199	journal	April 2011
Computational modeling of phonetic and lexical learning in early language acquisition: Existing models and future directions Räsänen, Okko Speech Communication, Vol. 54, Issue 9 https://doi.org/10.1016/j.specom.2012.05.001	journal	November 2012
Toward an Integration of Deep Learning and Neuroscience Marblestone, Adam H.; Wayne, Greg; Kording, Konrad P. Frontiers in Computational Neuroscience, Vol. 10 https://doi.org/10.3389/fncom.2016.00094	journal	September 2016
Induction of visual orientation modules in auditory cortex Sharma, Jitendra; Angelucci, Alessandra; Sur, Mriganka Nature, Vol. 404, Issue 6780 https://doi.org/10.1038/35009043	journal	April 2000
Sensitivity of neurons in cat primary auditory cortex to tones and frequency-modulated stimuli. II: Organization of response properties along the ‘isofrequency’ dimension Heil, Peter; Rajan, R.; Irvine, Dexter R. F. Hearing Research, Vol. 63, Issue 1-2 https://doi.org/10.1016/0378-5955(92)90081-W	journal	November 1992
Multiresolution spectrotemporal analysis of complex sounds Chi, Taishih; Ru, Powen; Shamma, Shihab A. The Journal of the Acoustical Society of America, Vol. 118, Issue 2 https://doi.org/10.1121/1.1945807	journal	August 2005
Continuous Online Sequence Learning with an Unsupervised Neural Network Model Cui, Yuwei; Ahmad, Subutai; Hawkins, Jeff Neural Computation, Vol. 28, Issue 11 https://doi.org/10.1162/NECO_a_00893	journal	November 2016
The Design and Implementation of FFTW3 Frigo, M.; Johnson, S. G. Proceedings of the IEEE, Vol. 93, Issue 2 https://doi.org/10.1109/JPROC.2004.840301	journal	February 2005
Multiple brain signatures of integration in the comprehension of degraded speech Obleser, Jonas; Kotz, Sonja A. NeuroImage, Vol. 55, Issue 2 https://doi.org/10.1016/j.neuroimage.2010.12.020	journal	March 2011
Experimental evidence for sparse firing in the neocortex. Barth, Alison L.; Poulet, James F. A. Carnegie Mellon University https://doi.org/10.1184/r1/6098609	text	January 2012
Perceptual Compensation for Coarticulation by Japanese Quail (Coturnix coturnix japonica) Lotto, Andrew J.; Kluender, Keith R.; Holt, Lori Carnegie Mellon University https://doi.org/10.1184/r1/6617513.v1	text	January 1997
Role of Experience for Language-Specific Functional Mappings of Vowel Sounds Kluender, Keith R.; Lotto, Andrew J.; Holt, Lori Figshare https://doi.org/10.1184/r1/6617975.v1	text	January 2018
Role of experience for language-specific functional mappings of vowel sounds Kluender, K. R.; Lotto, A. J.; Holt, Lori Carnegie Mellon University https://doi.org/10.1184/r1/6617978.v1	text	January 1998
Experimental evidence for sparse firing in the neocortex. Barth, Alison L.; Poulet, James F. A. Figshare https://doi.org/10.1184/r1/6098609.v1	text	January 2018
Perceptual compensation for coarticulation by Japanese quail (Coturnix coturnix japonica) Lotto, A. J.; Kluender, K. R.; Holt, Lori Carnegie Mellon University https://doi.org/10.1184/r1/6617516.v1	text	January 1997
Datasets used to train and test the Cortical Spectro-Temporal Model (CSTM). Dematties, Dario; Thiruvathukal, George K.; Rizzi, Silvio Zenodo https://doi.org/10.5281/zenodo.2576129	dataset	January 2019
Why Neurons Have Thousands of Synapses, A Theory of Sequence Memory in Neocortex Hawkins, Jeff; Ahmad, Subutai arXiv https://doi.org/10.48550/arxiv.1511.00083	text	January 2015
Towards an integration of deep learning and neuroscience Marblestone, Adam; Wayne, Greg; Kording, Konrad arXiv https://doi.org/10.48550/arxiv.1606.03813	text	January 2016
Brainstem inputs to the ferret medial geniculate nucleus and the effect of early deafferentation on novel retinal projections to the auditory thalamus Angelucci, Alessandra; Clasc�, Francisco; Sur, Mriganka The Journal of Comparative Neurology, Vol. 400, Issue 3 https://doi.org/10.1002/(sici)1096-9861(19981026)400:3<417::aid-cne10>3.0.co;2-o	journal	October 1998
Sensitivity of neurons in cat primary auditory cortex to tones and frequency-modulated stimuli. I: Effects of variation of stimulus parameters Heil, Peter; Rajan, R.; Irvine, Dexter R. F. Hearing Research, Vol. 63, Issue 1-2 https://doi.org/10.1016/0378-5955(92)90080-7	journal	November 1992
The functional organization of cortical feedback inputs to primary visual cortex Marques, Tiago; Nguyen, Julia; Fioreze, Gabriela Nature Neuroscience, Vol. 21, Issue 5 https://doi.org/10.1038/s41593-018-0135-z	journal	April 2018
Mechanisms of noise robust representation of speech in primary auditory cortex Mesgarani, N.; David, S. V.; Fritz, J. B. Proceedings of the National Academy of Sciences, Vol. 111, Issue 18 https://doi.org/10.1073/pnas.1318017111	journal	April 2014
Visual projections routed to the auditory pathway in ferrets: receptive fields of visual neurons in primary auditory cortex Roe, Aw; Pallas, Sl; Kwon, Yh The Journal of Neuroscience, Vol. 12, Issue 9 https://doi.org/10.1523/jneurosci.12-09-03651.1992	journal	September 1992
Functional Integration across Brain Regions Improves Speech Perception under Adverse Listening Conditions Obleser, J.; Wise, R. J. S.; Alex Dresner, M. Journal of Neuroscience, Vol. 27, Issue 9 https://doi.org/10.1523/jneurosci.4663-06.2007	journal	February 2007
Continuous online sequence learning with an unsupervised neural network model Cui, Yuwei; Ahmad, Subutai; Hawkins, Jeff arXiv https://doi.org/10.48550/arxiv.1512.05463	text	January 2015
Complementary control of sensory adaptation by two types of cortical interneurons Natan, Ryan G.; Briguglio, John J.; Mwilambwe-Tshilobo, Laetitia eLife, Vol. 4 https://doi.org/10.7554/elife.09868	journal	October 2015
Towards deep learning with segregated dendrites Guerguiev, Jordan; Lillicrap, Timothy P.; Richards, Blake A. eLife, Vol. 6 https://doi.org/10.7554/elife.22901	journal	December 2017
Datasets used to train and test the Cortical Spectro-Temporal Model (CSTM). Dematties, Dario; Thiruvathukal, George K.; Rizzi, Silvio Zenodo https://doi.org/10.5281/zenodo.2576130	dataset	January 2019
Experimental Results and Appendices: Cortical Spectro-Temporal Model (CSTM). Dematties, Dario; Thiruvathukal, George K.; Rizzi, Silvio Zenodo https://doi.org/10.5281/zenodo.2581550	dataset	January 2019

Similar Records

A Computational Theory for the Emergence of Grammatical Categories in Cortical Dynamics

Journal Article · Thu Apr 16 00:00:00 EDT 2020 · Frontiers in Neural Circuits · OSTI ID:1559970

Dematties, Dario; Rizzi, Silvio; Thiruvathukal, George K.; +3 more

The ''neural'' phonetic typewriter

Journal Article · Tue Mar 01 00:00:00 EST 1988 · Computer; (United States) · OSTI ID:1559970

Kohonen, T

Neural speech recognition: continuous phoneme decoding using spatiotemporal representations of human cortical activity

Journal Article · Wed Aug 03 00:00:00 EDT 2016 · Journal of Neural Engineering · OSTI ID:1559970

Moses, David A.; Mesgarani, Nima; Leonard, Matthew K.; +1 more

Related Subjects

97 MATHEMATICS AND COMPUTING
60 APPLIED LIFE SCIENCES

Title: Phonetic acquisition in cortical dynamics, a computational approach

Citation Formats

References (67)

Similar Records

Related Subjects