Deformable phrase level attention: A flexible approach for improving AI based medical coding

Metzner, Christoph; Gao, Shang; Herrmannova, Drahomira; Gounley, John; Hanson, Heidi A.

doi:10.1016/j.artmed.2025.103299

Deformable phrase level attention: A flexible approach for improving AI based medical coding

Journal Article · Sat Nov 15 00:00:00 EST 2025 · Artificial Intelligence in Medicine

DOI:https://doi.org/10.1016/j.artmed.2025.103299· OSTI ID:3020940

^[1]; ^[2]; ^[3]; ^[4]; ^[4]

Univ. of Tennessee, Knoxville, TN (United States). Bredesen Center for Interdisciplinary Research and Graduate Education; Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
Thomson Reuters, Eagan, MN (United States)
Elsevier, Philadelphia, PA (United States)
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Objective: Improving the AI-driven automated medical encoding of clinical text plays a vital role in gathering information on the occurrence of diseases to improve population-level health. This work presents a novel attention mechanism designed to enhance text classification models and ensure appropriate classification of medical concepts in unstructured electronic health records. Materials and Methods: We developed a deformable, phrase-level attention mechanism to identify important lexical word-level and contextual phrase-level information from clinical text documents. We evaluated conventional and transformer-based deep learning models that we extended with our attention mechanism on the extraction of critical cancer information (e.g., site, subsite, laterality, histology, behavior) from 629,908 electronic pathology reports and on the automated medical encoding of 52,722 hospital discharge summaries. Results: Transformer-based models with the deformable, phrase-level attention mechanism achieved the best performance on the extraction of critical cancer information from pathology reports. Conventional- and transformer-based models show similar or better performance than their baseline counterparts on the automated medical encoding of clinical documents. Discussion: The addition of phrase-level information allowed models extended with our proposed method to outperform standard word-level attention. Our method showed favorable properties for the real-world application in terms of model robustness and phenotyping. These results indicate that our method is promising for automated data harmonization for common data models. Conclusion: This work proposes a novel deformable, phrase-level attention mechanism that enhances text classification models in the extraction of medical concepts from clinical text documents. We demonstrate strong performances on two clinical text datasets and showcase real-world deployability of our method.

View Accepted Manuscript (DOE)

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Organization:: National Institutes of Health (NIH); USDOE

Grant/Contract Number:: AC02-06CH11357; AC05-00OR22725; AC52-06NA25396; AC52-07NA27344

OSTI ID:: 3020940

Journal Information:: Artificial Intelligence in Medicine, Journal Name: Artificial Intelligence in Medicine Vol. 171; ISSN 0933-3657

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

References (31)

Natural Language Processing in Oncology: A Review Yim, Wen-wai; Yetisgen, Meliha; Harris, William P. JAMA Oncology, Vol. 2, Issue 6 https://doi.org/10.1001/jamaoncol.2016.0213	journal	June 2016
Challenges and opportunities beyond structured data in analysis of electronic health records Tayefi, Maryam; Ngo, Phuong; Chomutare, Taridzo WIREs Computational Statistics, Vol. 13, Issue 6 https://doi.org/10.1002/wics.1549	journal	February 2021
RETRACTED ARTICLE: Artificial intelligence and machine learning in precision and genomic medicine Quazi, Sameer Medical Oncology, Vol. 39, Issue 8 https://doi.org/10.1007/s12032-022-01711-1	journal	June 2022
Racial disparities in guideline-concordant cancer care and mortality in the United States Fang, Penny; He, Weiguo; Gomez, Daniel Advances in Radiation Oncology, Vol. 3, Issue 3 https://doi.org/10.1016/j.adro.2018.04.013	journal	July 2018
Machine learning classification of surgical pathology reports and chunk recognition for information extraction noise reduction Napolitano, Giulio; Marshall, Adele; Hamilton, Peter Artificial Intelligence in Medicine, Vol. 70 https://doi.org/10.1016/j.artmed.2016.06.001	journal	June 2016
Classifying cancer pathology reports with hierarchical self-attention networks Gao, Shang; Qiu, John X.; Alawad, Mohammed Artificial Intelligence in Medicine, Vol. 101 https://doi.org/10.1016/j.artmed.2019.101726	journal	November 2019
Phrase2Vec: Phrase embedding based on parsing Wu, Yongliang; Zhao, Shuliang; Li, Wenbin Information Sciences, Vol. 517 https://doi.org/10.1016/j.ins.2019.12.031	journal	May 2020
Natural language processing: State of the art and prospects for significant progress, a workshop sponsored by the National Library of Medicine Friedman, Carol; Rindflesch, Thomas C.; Corn, Milton Journal of Biomedical Informatics, Vol. 46, Issue 5 https://doi.org/10.1016/j.jbi.2013.06.004	journal	October 2013
Hierarchical label-wise attention transformer model for explainable ICD coding Liu, Leibo; Perez-Concha, Oscar; Nguyen, Anthony Journal of Biomedical Informatics, Vol. 133 https://doi.org/10.1016/j.jbi.2022.104161	journal	September 2022
Explainable artificial intelligence (XAI) in deep learning-based medical image analysis van der Velden, Bas H. M.; Kuijf, Hugo J.; Gilhuijs, Kenneth G. A. Medical Image Analysis, Vol. 79 https://doi.org/10.1016/j.media.2022.102470	journal	July 2022
Predicting infectious disease for biopreparedness and response: A systematic review of machine learning and deep learning approaches Keshavamurthy, Ravikiran; Dixon, Samuel; Pazdernik, Karl T. One Health, Vol. 15 https://doi.org/10.1016/j.onehlt.2022.100439	journal	December 2022
MIMIC-III, a freely accessible critical care database Johnson, Alistair E. W.; Pollard, Tom J.; Shen, Lu Scientific Data, Vol. 3, Issue 1 https://doi.org/10.1038/sdata.2016.35	journal	May 2016
Using informatics to improve cancer surveillance Blumenthal, Wendy; Alimi, Temitope O.; Jones, Sandra F. Journal of the American Medical Informatics Association, Vol. 27, Issue 9 https://doi.org/10.1093/jamia/ocaa149	journal	September 2020
A comparative study of large language model-based zero-shot inference and task-specific supervised classification of breast cancer pathology reports Sushil, Madhumita; Zack, Travis; Mandair, Divneet Journal of the American Medical Informatics Association, Vol. 31, Issue 10 https://doi.org/10.1093/jamia/ocae146	journal	June 2024
Evaluating large language models for health-related text classification tasks with public social media data Guo, Yuting; Ovadje, Anthony; Al-Garadi, Mohammed Ali Journal of the American Medical Informatics Association, Vol. 31, Issue 10 https://doi.org/10.1093/jamia/ocae210	journal	August 2024
Using ensembles and distillation to optimize the deployment of deep learning models for the classification of electronic cancer pathology reports De Angeli, Kevin; Gao, Shang; Blanchard, Andrew JAMIA Open, Vol. 5, Issue 3 https://doi.org/10.1093/jamiaopen/ooac075	journal	July 2022
Boosting Crowd Counting via Multifaceted Attention Lin, Hui; Ma, Zhiheng; Ji, Rongrong 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR52688.2022.01901	conference	June 2022
Measuring Domain Shift for Deep Learning in Histopathology Stacke, Karin; Eilertsen, Gabriel; Unger, Jonas IEEE Journal of Biomedical and Health Informatics, Vol. 25, Issue 2 https://doi.org/10.1109/JBHI.2020.3032060	journal	February 2021
Limitations of Transformers on Clinical Text Classification Gao, Shang; Alawad, Mohammed; Young, M. Todd IEEE Journal of Biomedical and Health Informatics, Vol. 25, Issue 9 https://doi.org/10.1109/JBHI.2021.3062322	journal	September 2021
Attention Mechanisms in Clinical Text Classification: A Comparative Evaluation Metzner, Christoph S.; Gao, Shang; Herrmannova, Drahomira IEEE Journal of Biomedical and Health Informatics, Vol. 28, Issue 4 https://doi.org/10.1109/JBHI.2024.3355951	journal	April 2024
Deformable Self-Attention for Text Classification Ma, Qianli; Yan, Jiangyue; Lin, Zhenxi IEEE/ACM Transactions on Audio, Speech, and Language Processing, Vol. 29 https://doi.org/10.1109/TASLP.2021.3067210	journal	January 2021
A systematic literature review of automated clinical coding and classification systems Stanfill, Mary H.; Williams, Margaret; Fenton, Susan H. Journal of the American Medical Informatics Association, Vol. 17, Issue 6 https://doi.org/10.1136/jamia.2009.001024	journal	November 2010
Deep Learning--based Text Classification: A Comprehensive Review Minaee, Shervin; Kalchbrenner, Nal; Cambria, Erik ACM Computing Surveys, Vol. 54, Issue 3 https://doi.org/10.1145/3439726	journal	June 2021
A comparative study on deep learning models for text classification of unstructured medical notes with various levels of class imbalance Lu, Hongxia; Ehwerhemuepha, Louis; Rakovski, Cyril BMC Medical Research Methodology, Vol. 22, Issue 1 https://doi.org/10.1186/s12874-022-01665-y	journal	July 2022
Improving Cancer Data Interoperability: The Promise of the Minimal Common Oncology Data Elements (mCODE) Initiative Osterman, Travis J.; Terry, May; Miller, Robert S. JCO Clinical Cancer Informatics, Issue 4 https://doi.org/10.1200/CCI.20.00059	journal	November 2020
Population Health Informatics Can Advance Interoperability: National Program of Cancer Registries Electronic Pathology Reporting Project Pollack, Lori A.; Jones, Sandra F.; Blumenthal, Wendy JCO Clinical Cancer Informatics, Issue 4 https://doi.org/10.1200/CCI.20.00098	journal	November 2020
Effective Convolutional Attention Network for Multi-label Clinical Document Classification Liu, Yang; Cheng, Hua; Klopfer, Russell Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing https://doi.org/10.18653/v1/2021.emnlp-main.481	conference	January 2021
Phrase-level Self-Attention Networks for Universal Sentence Encoding Wu, Wei; Wang, Houfeng; Liu, Tianyu Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing https://doi.org/10.18653/v1/D18-1408	conference	January 2018
The Fast Health Interoperability Resources (FHIR) Standard: Systematic Literature Review of Implementations, Applications, Challenges and Opportunities Ayaz, Muhammad; Pasha, Muhammad F.; Alzahrani, Mohammed Y. JMIR Medical Informatics, Vol. 9, Issue 7 https://doi.org/10.2196/21929	journal	July 2021
Text Classification Algorithms: A Survey No authors listed Information, Vol. 10, Issue 4 https://doi.org/10.3390/info10040150	journal	April 2019
Validation of natural language processing to extract breast cancer pathology procedures and results Wieneke, Arika E.; Bowles, Erin J. A.; Cronkite, David Journal of Pathology Informatics, Vol. 6, Issue 1 https://doi.org/10.4103/2153-3539.159215	journal	January 2015

Similar Records

Class imbalance in out-of-distribution datasets: Improving the robustness of the TextCNN for the classification of rare cancer types

Journal Article · Sun Nov 21 19:00:00 EST 2021 · Journal of Biomedical Informatics · OSTI ID:1884003

FrESCO: Framework for Exploring Scalable Computational Oncology

Journal Article · Sun Sep 10 20:00:00 EDT 2023 · Journal of Open Source Software · OSTI ID:2000393

Related Subjects

Automated medical encoding
Common data models
Electronic health records
Machine learning
Natural language processing

Deformable phrase level attention: A flexible approach for improving AI based medical coding

Citation Formats

References (31)

Similar Records

Related Subjects