skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Retrofitting Word Embeddings with the UMLS Metathesaurus for Clinical Information Extraction

Conference ·

Deep learning has surged in popularity and proven to be effective for various artificial intelligence appli- cations including information extraction from cancer pathol- ogy reports. Since word representation is a core unit that enables deep learning algorithms to understand words and be able to perform NLP, this representation must include as much information as possible to help these algorithms achieve high classification performance. Therefore, in this work in addition to the distributional information of words in large sized corpora, we use UMLS vocabulary resources to enrich the vector space representation of words with the semantic relations between words. These resources provide many terminologies pertaining to cancer. The refined word embeddings are used with a convolutional neural (CNN) model to extract four data elements from cancer pathology reports; ICD-O-3 tumor topography codes, tumor laterality, behavior, and histological grade. We observed that using UMLS vocabulary resources to enrich word embeddings of CNN models consistently outperformed CNN models without pre- training word embeddings and even with pre-trained word embeddings on a domain specific corpus across all four tasks. The results show marginal improvement on the laterality task, but a significant improvement on the other tasks, especially for the macro-f score. Specifically, the improvements are 3%, 13%, and 15% for tumor site, histological grade, and behavior tasks, respectively. This approach is encouraging to enrich word embeddings with more clinical data resources to be used for information abstraction tasks from clinical pathology reports.

Research Organization:
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
Sponsoring Organization:
DOE Contract Number:
Resource Relation:
Conference: IEEE International Conference on Big Data - Seatttle, Washington, United States of America - 12/10/2018 10:00:00 AM-12/13/2018 10:00:00 AM
Country of Publication:
United States

References (12)

An Introduction to the Bootstrap book May 1994
Retrofitting Word Vectors to Semantic Lexicons
  • Faruqui, Manaal; Dodge, Jesse; Jauhar, Sujay Kumar
  • Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
conference January 2015
Automatic lymphoma classification with sentence subgraph mining from pathology reports journal January 2014
Automatic ICD-10 classification of cancers from free-text death certificates journal November 2015
Automatically extracting cancer disease characteristics from pathology reports into a Disease Knowledge Representation Model journal October 2009
Text mining of cancer-related information: Review of current status and future directions journal September 2014
Neural Machine Translation of Rare Words with Subword Units
  • Sennrich, Rico; Haddow, Barry; Birch, Alexandra
  • Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
conference January 2016
Deep Learning for Automated Extraction of Primary Sites From Cancer Pathology Reports journal January 2018
Improving Lexical Embeddings with Semantic Knowledge conference January 2014
Convolutional Neural Networks for Sentence Classification conference January 2014
Deep learning for stock market prediction from financial news articles
  • Vargas, Manuel R.; de Lima, Beatriz S. L. P.; Evsukoff, Alexandre G.
  • 2017 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications (CIVEMSA)
conference June 2017
The Unified Medical Language System journal January 1993

Similar Records

Retrofitting Word Embeddings with the UMLS Metathesaurus for Clinical Information Extraction
Conference · 2018 · 2018 IEEE International Conference on Big Data (Big Data) · OSTI ID:1491322

Automatic extraction of cancer registry reportable information from free-text pathology reports using multitask convolutional neural networks
Journal Article · 2019 · Journal of the American Medical Informatics Association · OSTI ID:1491322

Privacy-Preserving Deep Learning NLP Models for Cancer Registries
Journal Article · 2021 · IEEE Transactions on Emerging Topics in Computing · OSTI ID:1491322

Related Subjects