Automated detection of mild cognitive impairment and dementia from voice recordings: A natural language processing approach

Amini, Samad; Hao, Boran; Zhang, Lifu; Song, Mengting; Gupta, Aman; Karjadi, Cody; Kolachalama, Vijaya B.; Au, Rhoda; Paschalidis, Ioannis Ch.

doi:10.1002/alz.12721

Automated detection of mild cognitive impairment and dementia from voice recordings: A natural language processing approach

Journal Article · Thu Jul 07 00:00:00 EDT 2022 · Alzheimer's & Dementia

DOI:https://doi.org/10.1002/alz.12721· OSTI ID:1976265

Amini, Samad ^[1]; Hao, Boran ^[2]; Zhang, Lifu ^[2]; Song, Mengting ^[2]; Gupta, Aman ^[2]; Karjadi, Cody ^[2]; Kolachalama, Vijaya B. ^[3]; Au, Rhoda ^[4]; Paschalidis, Ioannis Ch. ^[2]

Boston University, MA (United States); OSTI
Boston University, MA (United States)
Boston University, MA (United States). School of Medicine
Boston University, MA (United States). School of Medicine; Boston University, MA (United States). Framingham Heart Study

Automated computational assessment of neuropsychological tests would enable widespread, cost-effective screening for dementia. A novel natural language processing approach is developed and validated to identify different stages of dementia based on automated transcription of digital voice recordings of subjects’ neuropsychological tests conducted by the Framingham Heart Study (n = 1084). Transcribed sentences from the test were encoded into quantitative data and several models were trained and tested using these data and the participants’ demographic characteristics. Average area under the curve (AUC) on the held-out test data reached 92.6%, 88.0%, and 74.4% for differentiating Normal cognition from Dementia, Normal or Mild Cognitive Impairment (MCI) from Dementia, and Normal from MCI, respectively. The proposed approach offers a fully automated identification of MCI and dementia based on a recorded neuropsychological test, providing an opportunity to develop a remote screening tool that could be adapted easily to any language.

View Accepted Manuscript (DOE)

Research Organization:: Boston University, MA (United States); Fraunhofer USA Inc., Plymouth, MI (United States)

Sponsoring Organization:: Alzheimer's Association; American Heart Association; Boston University; Framingham Heart Study's National Heart, Lung, and Blood Institute; Karen Toffler Charitable Trust; National Institutes of Health (NIH); National Science Foundation (NSF); Office of Naval Research (ONR); USDOE; USDOE Advanced Research Projects Agency - Energy (ARPA-E)

Grant/Contract Number:: AR0001282; EE0009696

OSTI ID:: 1976265

Alternate ID(s):: OSTI ID: 1875204

Journal Information:: Alzheimer's & Dementia, Journal Name: Alzheimer's & Dementia Journal Issue: 3 Vol. 19; ISSN 1552-5260

Publisher:: WileyCopyright Statement

Country of Publication:: United States

Language:: English

References (36)

The Inevitable Application of Big Data to Health Care Murdoch, Travis B.; Detsky, Allan S. JAMA, Vol. 309, Issue 13 https://doi.org/10.1001/jama.2013.393	journal	April 2013
“Mini-mental state” Folstein, Marshal F.; Folstein, Susan E.; McHugh, Paul R. Journal of Psychiatric Research, Vol. 12, Issue 3 https://doi.org/10.1016/0022-3956(75)90026-6	journal	November 1975
Alzheimer's disease Scheltens, Philip; Blennow, Kaj; Breteler, Monique M. B. The Lancet, Vol. 388, Issue 10043 https://doi.org/10.1016/S0140-6736(15)01124-1	journal	July 2016
Neural transfer learning for assigning diagnosis codes to EMRs Rios, Anthony; Kavuluru, Ramakanth Artificial Intelligence in Medicine, Vol. 96 https://doi.org/10.1016/j.artmed.2019.04.002	journal	May 2019
Automatic speech analysis for the assessment of patients with predementia and Alzheimer's disease König, Alexandra; Satt, Aharon; Sorin, Alexander Alzheimer's & Dementia: Diagnosis, Assessment & Disease Monitoring, Vol. 1, Issue 1 https://doi.org/10.1016/j.dadm.2014.11.012	journal	March 2015
Computer‐based evaluation of Alzheimer's disease and mild cognitive impairment patients during a picture description task Hernández‐Domínguez, Laura; Ratté, Sylvie; Sierra‐Martínez, Gerardo Alzheimer's & Dementia: Diagnosis, Assessment & Disease Monitoring, Vol. 10, Issue 1 https://doi.org/10.1016/j.dadm.2018.02.004	journal	January 2018
Alzheimer's disease and automatic speech analysis: A review Pulido, María Luisa Barragán; Hernández, Jesús Bernardino Alonso; Ballester, Miguel Ángel Ferrer Expert Systems with Applications, Vol. 150 https://doi.org/10.1016/j.eswa.2020.113213	journal	July 2020
P2‐267: Novel Digital Voice Biomarkers of Dementia from the Framingham Study Stück, David; Signorini, Alessio; Al Hanai, Tuka Alzheimer's & Dementia, Vol. 14, Issue 7S_Part_14 https://doi.org/10.1016/j.jalz.2018.06.956	journal	July 2006
A new machine learning method for identifying Alzheimer's disease Liu, Lin; Zhao, Shenghui; Chen, Haibao Simulation Modelling Practice and Theory, Vol. 99 https://doi.org/10.1016/j.simpat.2019.102023	journal	February 2020
Predicting mild cognitive impairment from spontaneous spoken utterances Asgari, Meysam; Kaye, Jeffrey; Dodge, Hiroko Alzheimer's & Dementia: Translational Research & Clinical Interventions, Vol. 3, Issue 2 https://doi.org/10.1016/j.trci.2017.01.006	journal	February 2017
Neuropsychological Criteria for Mild Cognitive Impairment and Dementia Risk in the Framingham Heart Study Jak, Amy J.; Preis, Sarah R.; Beiser, Alexa S. Journal of the International Neuropsychological Society, Vol. 22, Issue 9 https://doi.org/10.1017/S1355617716000199	journal	March 2016
Cross-Validation of the Newly-Normed SKT for the Detection of MCI and Dementia Hessler, Johannes Baltasar; Stemmler, Mark; Bickel, Horst GeroPsych, Vol. 30, Issue 1 https://doi.org/10.1024/1662-9647/a000154	journal	March 2017
How technology is reshaping cognitive assessment: Lessons from the Framingham Heart Study. Au, Rhoda; Piers, Ryan J.; Devine, Sherral Neuropsychology, Vol. 31, Issue 8 https://doi.org/10.1037/neu0000411	journal	November 2017
70-year legacy of the Framingham Heart Study Andersson, Charlotte; Johnson, Andrew D.; Benjamin, Emelia J. Nature Reviews Cardiology, Vol. 16, Issue 11 https://doi.org/10.1038/s41569-019-0202-5	journal	May 2019
Digital biomarkers for Alzheimer’s disease: the mobile/wearable devices opportunity Kourtis, Lampros C.; Regele, Oliver B.; Wright, Justin M. npj Digital Medicine, Vol. 2, Issue 1 https://doi.org/10.1038/s41746-019-0084-2	journal	February 2019
Digital medicine and the curse of dimensionality Berisha, Visar; Krantsevich, Chelsea; Hahn, P. Richard npj Digital Medicine, Vol. 4, Issue 1 https://doi.org/10.1038/s41746-021-00521-5	journal	October 2021
Incidence of Dementia over Three Decades in the Framingham Heart Study Satizabal, Claudia L.; Beiser, Alexa S.; Chouraki, Vincent New England Journal of Medicine, Vol. 374, Issue 6 https://doi.org/10.1056/NEJMoa1504327	journal	February 2016
The Kolmogorov-Smirnov Test for Goodness of Fit Massey, Frank J. Journal of the American Statistical Association, Vol. 46, Issue 253 https://doi.org/10.1080/01621459.1951.10500769	journal	March 1951
Connected speech assessment in the early detection of Alzheimer’s disease and mild cognitive impairment: a scoping review Filiou, Renée-Pier; Bier, Nathalie; Slegers, Antoine Aphasiology, Vol. 34, Issue 6 https://doi.org/10.1080/02687038.2019.1608502	journal	April 2019
Test–retest stability on the WRAT-3 reading subtest in geriatric cognitive evaluations Ashendorf, Lee; Jefferson, Angela L.; Green, Robert C. Journal of Clinical and Experimental Neuropsychology, Vol. 31, Issue 5 https://doi.org/10.1080/13803390802375557	journal	May 2009
Detecting emotional valence using time-domain analysis of speech signals Deshpande, Gauri; Viraraghavan, Venkata Subramanian; Duggirala, Mayuri 2019 41st Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) https://doi.org/10.1109/EMBC.2019.8857691	conference	July 2019
2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text Uzuner, Özlem; South, Brett R.; Shen, Shuying Journal of the American Medical Informatics Association, Vol. 18, Issue 5 https://doi.org/10.1136/amiajnl-2011-000203	journal	June 2011
The meaning and use of the area under a receiver operating characteristic (ROC) curve. Hanley, J. A.; McNeil, B. J. Radiology, Vol. 143, Issue 1 https://doi.org/10.1148/radiology.143.1.7063747	journal	April 1982
The Chi-square test of independence McHugh, Mary L. Biochemia Medica https://doi.org/10.11613/BM.2013.018	journal	January 2013
A Summary Score for the Framingham Heart Study Neuropsychological Battery Downer, Brian; Fardo, David W.; Schmitt, Frederick A. Journal of Aging and Health, Vol. 27, Issue 7 https://doi.org/10.1177/0898264315577590	journal	March 2015
Detection of dementia on voice recordings using deep learning: a Framingham Heart Study Xue, Chonghua; Karjadi, Cody; Paschalidis, Ioannis Ch. Alzheimer's Research & Therapy, Vol. 13, Issue 1 https://doi.org/10.1186/s13195-021-00888-3	journal	August 2021
Blood pressure from mid‐ to late life and risk of incident dementia McGrath, Emer R.; Beiser, Alexa S.; DeCarli, Charles Neurology, Vol. 89, Issue 24 https://doi.org/10.1212/WNL.0000000000004741	journal	November 2017
Enhancing Clinical BERT Embedding using a Biomedical Knowledge Base Hao, Boran; Zhu, Henghui; Paschalidis, Ioannis Proceedings of the 28th International Conference on Computational Linguistics https://doi.org/10.18653/v1/2020.coling-main.57	conference	January 2020
Fully automated neuropsychological assessment for detecting mild cognitive impairment Lehr, Maider; Prud'hommeaux, Emily; Shafran, Izhak Interspeech 2012 https://doi.org/10.21437/Interspeech.2012-306	conference	September 2012
Tracking Discourse Complexity Preceding Alzheimer's Disease Diagnosis: A Case Study Comparing the Press Conferences of Presidents Ronald Reagan and George Herbert Walker Bush Berisha, Visar; Wang, Shuai; LaCross, Amy Journal of Alzheimer's Disease, Vol. 45, Issue 3 https://doi.org/10.3233/JAD-142763	journal	March 2015
Linguistic Features Identify Alzheimer’s Disease in Narrative Speech Fraser, Kathleen C.; Meltzer, Jed A.; Rudzicz, Frank Journal of Alzheimer's Disease, Vol. 49, Issue 2 https://doi.org/10.3233/JAD-150520	journal	October 2015
Assessing the Utility of Language and Voice Biomarkers to Predict Cognitive Impairment in the Framingham Heart Study Cognitive Aging Cohort Data Thomas, Jason A.; Burkhardt, Hannah A.; Chaudhry, Safina Journal of Alzheimer's Disease, Vol. 76, Issue 3 https://doi.org/10.3233/JAD-190783	journal	August 2020
Potential New Approaches for Diagnosis of Alzheimer's Disease and Related Dementias Turner, R. Scott; Stubbs, Terry; Davies, Don A. Frontiers in Neurology, Vol. 11 https://doi.org/10.3389/fneur.2020.00496	journal	June 2020
Connected Speech in Neurodegenerative Language Disorders: A Review Boschi, Veronica; Catricalà, Eleonora; Consonni, Monica Frontiers in Psychology, Vol. 8 https://doi.org/10.3389/fpsyg.2017.00269	journal	March 2017
Identification of digital voice biomarkers for cognitive health Lin, Honghuang; Karjadi, Cody; Ang, Ting F. A. Exploration of Medicine, Vol. 1, Issue 6 https://doi.org/10.37349/emed.2020.00028	journal	December 2020
Early prediction of level-of-care requirements in patients with COVID-19 Hao, Boran; Sotudian, Shahabeddin; Wang, Taiyao eLife, Vol. 9 https://doi.org/10.7554/eLife.60519	journal	October 2020

Cited By (1)

Additional file 1 of Unveiling the sound of the cognitive status: Machine Learning-based speech analysis in the Alzheimer’s disease spectrum García-Gutiérrez, Fernando; Alegret, Montserrat; Marquié, Marta figshare https://doi.org/10.6084/m9.figshare.25137955.v1	dataset	January 2024

Similar Records

Tau Positron Emission Tomography for Predicting Dementia in Individuals With Mild Cognitive Impairment

Journal Article · Sun Jun 09 20:00:00 EDT 2024 · JAMA Neurology · OSTI ID:2470703

Memory Concerns, Memory Performance and Risk of Dementia in Patients with Mild Cognitive Impairment

Journal Article · Sun Jul 13 20:00:00 EDT 2014 · PLoS ONE · OSTI ID:1627709

Related Subjects

neuropsychological
60 APPLIED LIFE SCIENCES
Alzheimer‘s disease
cognitive impairment
natural language processing

Automated detection of mild cognitive impairment and dementia from voice recordings: A natural language processing approach

Citation Formats

References (36)

Cited By (1)

Similar Records

Related Subjects