Retrofitting Word Embeddings with the UMLS Metathesaurus for Clinical Information Extraction

Alawad, Mohammed; Hasan, S M Shamimul; Christian, Blair; Tourassi, Georgia

doi:10.1109/BigData.2018.8621999

Title: Retrofitting Word Embeddings with the UMLS Metathesaurus for Clinical Information Extraction

Abstract

Deep learning has surged in popularity and proven to be effective for various artificial intelligence appli- cations including information extraction from cancer pathol- ogy reports. Since word representation is a core unit that enables deep learning algorithms to understand words and be able to perform NLP, this representation must include as much information as possible to help these algorithms achieve high classification performance. Therefore, in this work in addition to the distributional information of words in large sized corpora, we use UMLS vocabulary resources to enrich the vector space representation of words with the semantic relations between words. These resources provide many terminologies pertaining to cancer. The refined word embeddings are used with a convolutional neural (CNN) model to extract four data elements from cancer pathology reports; ICD-O-3 tumor topography codes, tumor laterality, behavior, and histological grade. We observed that using UMLS vocabulary resources to enrich word embeddings of CNN models consistently outperformed CNN models without pre- training word embeddings and even with pre-trained word embeddings on a domain specific corpus across all four tasks. The results show marginal improvement on the laterality task, but a significant improvement on the other tasks, especially for the macro-f score. Specifically, the improvementsmore »« less

Authors:

^[1]; Hasan, S M Shamimul ^[1];

^[1];

^[1]

ORNL

Publication Date:: 2018-12-01

Research Org.:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Org.:: USDOE

OSTI Identifier:: 1491322

DOE Contract Number:: AC05-00OR22725

Resource Type:: Conference

Resource Relation:: Conference: IEEE International Conference on Big Data - Seatttle, Washington, United States of America - 12/10/2018 10:00:00 AM-12/13/2018 10:00:00 AM

Country of Publication:: United States

Language:: English

Citation Formats


                    Alawad, Mohammed, Hasan, S M Shamimul, Christian, Blair, and Tourassi, Georgia. Retrofitting Word Embeddings with the UMLS Metathesaurus for Clinical Information Extraction.  United States: N. p., 2018. 
        Web.  doi:10.1109/BigData.2018.8621999.

Copy to clipboard


                    Alawad, Mohammed, Hasan, S M Shamimul, Christian, Blair, & Tourassi, Georgia. Retrofitting Word Embeddings with the UMLS Metathesaurus for Clinical Information Extraction.  United States.  https://doi.org/10.1109/BigData.2018.8621999

Copy to clipboard


                    Alawad, Mohammed, Hasan, S M Shamimul, Christian, Blair, and Tourassi, Georgia. 2018.  
        "Retrofitting Word Embeddings with the UMLS Metathesaurus for Clinical Information Extraction".  United States.  https://doi.org/10.1109/BigData.2018.8621999.  https://www.osti.gov/servlets/purl/1491322.

Copy to clipboard


                    
@article{osti_1491322,

  title        = {Retrofitting Word Embeddings with the UMLS Metathesaurus for Clinical Information Extraction},

  author       = {Alawad, Mohammed and Hasan, S M Shamimul and Christian, Blair and Tourassi, Georgia},

  abstractNote = {Deep learning has surged in popularity and proven to be effective for various artificial intelligence appli- cations including information extraction from cancer pathol- ogy reports. Since word representation is a core unit that enables deep learning algorithms to understand words and be able to perform NLP, this representation must include as much information as possible to help these algorithms achieve high classification performance. Therefore, in this work in addition to the distributional information of words in large sized corpora, we use UMLS vocabulary resources to enrich the vector space representation of words with the semantic relations between words. These resources provide many terminologies pertaining to cancer. The refined word embeddings are used with a convolutional neural (CNN) model to extract four data elements from cancer pathology reports; ICD-O-3 tumor topography codes, tumor laterality, behavior, and histological grade. We observed that using UMLS vocabulary resources to enrich word embeddings of CNN models consistently outperformed CNN models without pre- training word embeddings and even with pre-trained word embeddings on a domain specific corpus across all four tasks. The results show marginal improvement on the laterality task, but a significant improvement on the other tasks, especially for the macro-f score. Specifically, the improvements are 3%, 13%, and 15% for tumor site, histological grade, and behavior tasks, respectively. This approach is encouraging to enrich word embeddings with more clinical data resources to be used for information abstraction tasks from clinical pathology reports.},

  doi          = {10.1109/BigData.2018.8621999},

  url          = {https://www.osti.gov/biblio/1491322},
  journal      = {},
number       = ,

  volume       = ,

  place        = {United States},

  year         = {2018},

  month        = {12}

}

Copy to clipboard

Conference:

View Conference (0.33 MB)

https://doi.org/10.1109/BigData.2018.8621999

Other availability

Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share:

Export Metadata

Save to My Library

Works referenced in this record:

Retrofitting Word Vectors to Semantic Lexicons
conference, January 2015

Faruqui, Manaal; Dodge, Jesse; Jauhar, Sujay Kumar
Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
https://doi.org/10.3115/v1/N15-1184

Automatic lymphoma classification with sentence subgraph mining from pathology reports
journal, January 2014

Luo, Yuan; Sohani, Aliyah R.; Hochberg, Ephraim P.
Journal of the American Medical Informatics Association, Vol. 21, Issue 5
https://doi.org/10.1136/amiajnl-2013-002443

Automatic ICD-10 classification of cancers from free-text death certificates
journal, November 2015

Koopman, Bevan; Zuccon, Guido; Nguyen, Anthony
International Journal of Medical Informatics, Vol. 84, Issue 11
https://doi.org/10.1016/j.ijmedinf.2015.08.004

Automatically extracting cancer disease characteristics from pathology reports into a Disease Knowledge Representation Model
journal, October 2009

Coden, Anni; Savova, Guergana; Sominsky, Igor
Journal of Biomedical Informatics, Vol. 42, Issue 5
https://doi.org/10.1016/j.jbi.2008.12.005

Text mining of cancer-related information: Review of current status and future directions
journal, September 2014

Spasić, Irena; Livsey, Jacqueline; Keane, John A.
International Journal of Medical Informatics, Vol. 83, Issue 9
https://doi.org/10.1016/j.ijmedinf.2014.06.009

Neural Machine Translation of Rare Words with Subword Units
conference, January 2016

Sennrich, Rico; Haddow, Barry; Birch, Alexandra
Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
https://doi.org/10.18653/v1/P16-1162

Improving Lexical Embeddings with Semantic Knowledge
conference, January 2014

Yu, Mo; Dredze, Mark
Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
https://doi.org/10.3115/v1/P14-2089

Convolutional Neural Networks for Sentence Classification
conference, January 2014

Kim, Yoon
Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)
https://doi.org/10.3115/v1/D14-1181

Deep learning for stock market prediction from financial news articles
conference, June 2017

Vargas, Manuel R.; de Lima, Beatriz S. L. P.; Evsukoff, Alexandre G.
2017 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications (CIVEMSA)
https://doi.org/10.1109/CIVEMSA.2017.7995302

The Unified Medical Language System
journal, January 1993

Lindberg, D. A. B.; Humphreys, B. L.; McCray, A. T.
Methods of Information in Medicine, Vol. 32, Issue 04
https://doi.org/10.1055/s-0038-1634945

Similar records in OSTI.GOV collections:

Retrofitting Word Embeddings with the UMLS Metathesaurus for Clinical Information Extraction

Conference Alawad, Mohammed; Hasan, S.M.; Blair Christian, J.; ... - 2018 IEEE International Conference on Big Data (Big Data)

Deep learning has surged in popularity and proven to be effective for various artificial intelligence applications including information extraction from cancer pathology reports. Since word representation is a core unit that enables deep learning algorithms to understand words and be able to perform NLP, this representation must include as much information as possible to help these algorithms achieve high classification performance. Therefore, in this work in addition to the distributional information of words in large sized corpora, we use UMLS vocabulary resources to enrich the vector space representation of words with the semantic relations between words. These resources provide manymore »« less
https://doi.org/10.1109/BigData.2018.8621999
Automatic extraction of cancer registry reportable information from free-text pathology reports using multitask convolutional neural networks

Journal Article Alawad, Mohammed; Gao, Shang; Qiu, John; ... - Journal of the American Medical Informatics Association

Objective: In this work we implement 2 different multitask learning (MTL) techniques, hard parameter sharing and cross-stitch, to train a word-level convolutional neural network (CNN) specifically designed for automatic extraction of cancer data from unstructured text in pathology reports. We show the importance of learning related information extraction (IE) tasks leveraging shared representations across the tasks to achieve state-of-the-art performance in classification accuracy and computational efficiency. Materials and Methods: Multitask CNN (MTCNN) attempts to tackle document information extraction by learning to extract multiple key cancer characteristics simultaneously. We trained our MTCNN to perform 5 information extraction tasks: (1) primary cancermore »« less
Cited by 22
https://doi.org/10.1093/jamia/ocz153
Privacy-Preserving Deep Learning NLP Models for Cancer Registries

Journal Article Alawad, Mohammed; Yoon, Hong-Jun; Gao, Shang; ... - IEEE Transactions on Emerging Topics in Computing

Population cancer registries can benefit from Deep Learning (DL) to automatically extract cancer characteristics from the high volume of unstructured pathology text reports they process annually. The success of DL to tackle this and other real-world problems is proportional to the availability of large labeled datasets for model training. Although collaboration among cancer registries is essential to fully exploit the promise of DL, privacy and confidentiality concerns are main obstacles for data sharing across cancer registries. Moreover, DL for natural language processing (NLP) requires sharing a vocabulary dictionary for the embedding layer which may contain patient identifiers. Thus, even distributingmore »« less
https://doi.org/10.1109/TETC.2020.2983404
Coarse-to-Fine Multi-Task Training of Convolutional Neural Networks for Automated Information Extraction from Cancer Pathology Reports

Conference Alawad, Mohammed; Yoon, Hong-Jun; Tourassi, Georgia

Information extraction and coding of free-text pathology reports is an important activity for cancer registries to support national cancer surveillance. Cancer registrars must process high volumes of pathology reports on an annual basis. In this study, we investigated an automated approach using a coarse-to-fine training of convolutional neural networks (CNNs) for extracting the primary site, histological grade and laterality from unstructured cancer pathology text reports. Our proposed training scheme consists of two stages. In the first stage, the multi-task learning (MTL) with hard parameter sharing approach is used to train a multi-task MT-CNN model for all the tasks. Then, themore »« less
https://doi.org/10.1109/BHI.2018.8333408

Full Text Available
Multimodal Data Representation with Deep Learning for Extracting Cancer Characteristics from Clinical Text

Conference Alawad, Mohammed; Gao, Shang; Alamudun, Folami; ...

This paper presents a multimodal data representation to improve the performance of deep learning models for extracting cancer key characteristics from unstructured text in pathology reports. Specifically, in addition to using the text as the input to deep learning models, we use concept unique identifiers (CUIs) as another source of information to the models. We analyze the performance of different text and CUI data representations, including word embeddings and bag of embeddings (BOE), with a convolutional neural network (CNN) and a fully connected multilayer perceptron neural network (MLP-NN). The high level document embeddings from text and CUI inputs are combinedmore »« less
Full Text Available

Similar Records

Title: Retrofitting Word Embeddings with the UMLS Metathesaurus for Clinical Information Extraction

Abstract

Citation Formats

Retrofitting Word Vectors to Semantic Lexicons conference, January 2015

Automatic lymphoma classification with sentence subgraph mining from pathology reports journal, January 2014

Automatic ICD-10 classification of cancers from free-text death certificates journal, November 2015

Automatically extracting cancer disease characteristics from pathology reports into a Disease Knowledge Representation Model journal, October 2009

Text mining of cancer-related information: Review of current status and future directions journal, September 2014

Neural Machine Translation of Rare Words with Subword Units conference, January 2016

Improving Lexical Embeddings with Semantic Knowledge conference, January 2014

Convolutional Neural Networks for Sentence Classification conference, January 2014

Deep learning for stock market prediction from financial news articles conference, June 2017

The Unified Medical Language System journal, January 1993

Retrofitting Word Vectors to Semantic Lexicons
conference, January 2015

Automatic lymphoma classification with sentence subgraph mining from pathology reports
journal, January 2014

Automatic ICD-10 classification of cancers from free-text death certificates
journal, November 2015

Automatically extracting cancer disease characteristics from pathology reports into a Disease Knowledge Representation Model
journal, October 2009

Text mining of cancer-related information: Review of current status and future directions
journal, September 2014

Neural Machine Translation of Rare Words with Subword Units
conference, January 2016

Improving Lexical Embeddings with Semantic Knowledge
conference, January 2014

Convolutional Neural Networks for Sentence Classification
conference, January 2014

Deep learning for stock market prediction from financial news articles
conference, June 2017

The Unified Medical Language System
journal, January 1993