DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: CMed: Crowd Analytics for Medical Imaging Data

Abstract

In this work, we present a visual analytics framework, CMed, for exploring medical image data annotations acquired from crowdsourcing. CMed can be used to visualize, classify, and filter crowdsourced clinical data based on a number of different metrics such as detection rate, logged events, and clustering of the annotations. CMed provides several interactive linked visualization components to analyze the crowd annotation results for a particular video and the associated workers. Additionally, all results of an individual worker can be inspected using multiple linked views in our CMed framework. We allow a crowdsourcing application analyst to observe patterns and gather insights into the crowdsourced medical data, helping him/her design future crowdsourcing applications for optimal output from the workers. We demonstrate the efficacy of our framework with two medical crowdsourcing studies: polyp detection in virtual colonoscopy videos and lung nodule detection in CT thin-slab maximum intensity projection videos. We also provide experts' feedback to show the effectiveness of our framework. Lastly, we share the lessons we learned from our framework with suggestions for integrating our framework into a clinical workflow.

Authors:
 [1];  [2];  [3];  [3];  [3]
  1. Brookhaven National Lab. (BNL), Upton, NY (United States)
  2. Memorial Sloan Kettering Cancer Center, New York, NY (United States). Sloan Kettering Institute
  3. Stony Brook Univ., NY (United States)
Publication Date:
Research Org.:
Brookhaven National Lab. (BNL), Upton, NY (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Advanced Scientific Computing Research; National Science Foundation (NSF)
OSTI Identifier:
1677652
Report Number(s):
BNL-219939-2020-JAAM
Journal ID: ISSN 1077-2626
Grant/Contract Number:  
SC0012704; NRT1633299; CNS1650499; OAC1919752
Resource Type:
Accepted Manuscript
Journal Name:
IEEE Transactions on Visualization and Computer Graphics
Additional Journal Information:
Journal Volume: 27; Journal Issue: 6; Journal ID: ISSN 1077-2626
Publisher:
IEEE
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; Crowdsourcing; biomedical imaging; data visualization; task analysis; visual analytics; lung; computed tomography; virtual colonoscopy; lung nodules

Citation Formats

Park, Ji Hwan, Nadeem, Saad, Boorboor, Saeed, Marino, Joseph, and Kaufman, Arie E. CMed: Crowd Analytics for Medical Imaging Data. United States: N. p., 2019. Web. doi:10.1109/tvcg.2019.2953026.
Park, Ji Hwan, Nadeem, Saad, Boorboor, Saeed, Marino, Joseph, & Kaufman, Arie E. CMed: Crowd Analytics for Medical Imaging Data. United States. https://doi.org/10.1109/tvcg.2019.2953026
Park, Ji Hwan, Nadeem, Saad, Boorboor, Saeed, Marino, Joseph, and Kaufman, Arie E. Wed . "CMed: Crowd Analytics for Medical Imaging Data". United States. https://doi.org/10.1109/tvcg.2019.2953026. https://www.osti.gov/servlets/purl/1677652.
@article{osti_1677652,
title = {CMed: Crowd Analytics for Medical Imaging Data},
author = {Park, Ji Hwan and Nadeem, Saad and Boorboor, Saeed and Marino, Joseph and Kaufman, Arie E.},
abstractNote = {In this work, we present a visual analytics framework, CMed, for exploring medical image data annotations acquired from crowdsourcing. CMed can be used to visualize, classify, and filter crowdsourced clinical data based on a number of different metrics such as detection rate, logged events, and clustering of the annotations. CMed provides several interactive linked visualization components to analyze the crowd annotation results for a particular video and the associated workers. Additionally, all results of an individual worker can be inspected using multiple linked views in our CMed framework. We allow a crowdsourcing application analyst to observe patterns and gather insights into the crowdsourced medical data, helping him/her design future crowdsourcing applications for optimal output from the workers. We demonstrate the efficacy of our framework with two medical crowdsourcing studies: polyp detection in virtual colonoscopy videos and lung nodule detection in CT thin-slab maximum intensity projection videos. We also provide experts' feedback to show the effectiveness of our framework. Lastly, we share the lessons we learned from our framework with suggestions for integrating our framework into a clinical workflow.},
doi = {10.1109/tvcg.2019.2953026},
journal = {IEEE Transactions on Visualization and Computer Graphics},
number = 6,
volume = 27,
place = {United States},
year = {2019},
month = {11}
}