cpgQA: A Benchmark Dataset for Machine Reading Comprehension Tasks on Clinical Practice Guidelines and a Case Study Using Transfer Learning
[1];
[3];
[1]
- Department of Electrical Engineering and Computer Science, University of Tennessee, Knoxville, TN, USA
- Oak Ridge National Laboratory, Cyber Resilience and Intelligence Division, Oak Ridge, TN, USA
- Department of Veterans Affairs, Program Evaluation and Resource Center, Office of Mental Health and Suicide Prevention, Menlo Park, CA, USA
- Oak Ridge National Laboratory, Computational Sciences and Engineering Division, Oak Ridge, TN, USA
Biomedical machine reading comprehension (bio-MRC), a crucial task in natural language processing, is a vital application of a computer-assisted clinical decision support system. It can help clinicians extract critical information effortlessly for clinical decision-making by comprehending and answering questions from biomedical text data. While recent advances in bio-MRC consider text data from resources such as clinical notes and scholarly articles, the clinical practice guidelines (CPGs) are still unexplored in this regard. CPGs are a pivotal component of clinical decision-making at the point of care as they provide recommendations for patient care based on the most up-to-date information available. Although CPGs are inherently terse compared to a multitude of articles, often, clinicians find them lengthy and complicated to use. In this paper, we define a new problem domain – bio-MRC on CPGs – where the ultimate goal is to assist clinicians in efficiently interpreting the clinical practice guidelines using MRC systems. To that end, we develop a manually annotated and subject-matter expert-validated benchmark dataset for the bio-MRC task on CPGs – cpgQA. This dataset aims to evaluate intelligent systems performing MRC tasks on CPGs. Hence, we employ the state-of-the-art MRC models to present a case study illustrating an extensive evaluation of the proposed dataset. We address the problem of lack of training data in this newly defined domain by applying transfer learning. The results show that while the current state-of-the-art models perform well with 78% exact match scores on the dataset, there is still room for improvement, warranting further research on this problem domain.
- Research Organization:
- Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
- Sponsoring Organization:
- USDOE
- Grant/Contract Number:
- AC05-00OR22725; VA118-16-M-1062
- OSTI ID:
- 1909280
- Alternate ID(s):
- OSTI ID: 1909281; OSTI ID: 1972583
- Journal Information:
- IEEE Access, Journal Name: IEEE Access Vol. 11; ISSN 2169-3536
- Publisher:
- Institute of Electrical and Electronics EngineersCopyright Statement
- Country of Publication:
- United States
- Language:
- English
Similar Records
TH-C-204-00: Medical Physics Workshop: An Update On the Journal’s Improvement Activities and Guidance On Writing and Reviewing Papers
TH-C-204-01: Vision for Medical Physics and Status of Current Initiatives