CMed: Crowd Analytics for Medical Imaging Data

Park, Ji Hwan; Nadeem, Saad; Boorboor, Saeed; Marino, Joseph; Kaufman, Arie E.

doi:10.1109/tvcg.2019.2953026

Title: CMed: Crowd Analytics for Medical Imaging Data

Full Record
Other Related Research

Abstract

In this work, we present a visual analytics framework, CMed, for exploring medical image data annotations acquired from crowdsourcing. CMed can be used to visualize, classify, and filter crowdsourced clinical data based on a number of different metrics such as detection rate, logged events, and clustering of the annotations. CMed provides several interactive linked visualization components to analyze the crowd annotation results for a particular video and the associated workers. Additionally, all results of an individual worker can be inspected using multiple linked views in our CMed framework. We allow a crowdsourcing application analyst to observe patterns and gather insights into the crowdsourced medical data, helping him/her design future crowdsourcing applications for optimal output from the workers. We demonstrate the efficacy of our framework with two medical crowdsourcing studies: polyp detection in virtual colonoscopy videos and lung nodule detection in CT thin-slab maximum intensity projection videos. We also provide experts' feedback to show the effectiveness of our framework. Lastly, we share the lessons we learned from our framework with suggestions for integrating our framework into a clinical workflow.

Authors:

Park, Ji Hwan ^[1]; Nadeem, Saad ^[2]; Boorboor, Saeed ^[3]; Marino, Joseph ^[3]; Kaufman, Arie E. ^[3]

Brookhaven National Lab. (BNL), Upton, NY (United States)
Memorial Sloan Kettering Cancer Center, New York, NY (United States). Sloan Kettering Institute
Stony Brook Univ., NY (United States)

Publication Date:: 2019-11-20

Research Org.:: Brookhaven National Lab. (BNL), Upton, NY (United States)

Sponsoring Org.:: USDOE Office of Science (SC), Advanced Scientific Computing Research; National Science Foundation (NSF)

OSTI Identifier:: 1677652

Report Number(s):: BNL-219939-2020-JAAM
Journal ID: ISSN 1077-2626

Grant/Contract Number:: SC0012704; NRT1633299; CNS1650499; OAC1919752

Resource Type:: Accepted Manuscript

Journal Name:: IEEE Transactions on Visualization and Computer Graphics

Additional Journal Information:: Journal Volume: 27; Journal Issue: 6; Journal ID: ISSN 1077-2626

Publisher:: IEEE

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING; Crowdsourcing; biomedical imaging; data visualization; task analysis; visual analytics; lung; computed tomography; virtual colonoscopy; lung nodules

Citation Formats


                    Park, Ji Hwan, Nadeem, Saad, Boorboor, Saeed, Marino, Joseph, and Kaufman, Arie E. CMed: Crowd Analytics for Medical Imaging Data.  United States: N. p., 2019. 
Web.  doi:10.1109/tvcg.2019.2953026.

Copy to clipboard


                    Park, Ji Hwan, Nadeem, Saad, Boorboor, Saeed, Marino, Joseph, & Kaufman, Arie E. CMed: Crowd Analytics for Medical Imaging Data.  United States.  https://doi.org/10.1109/tvcg.2019.2953026

Copy to clipboard


                    Park, Ji Hwan, Nadeem, Saad, Boorboor, Saeed, Marino, Joseph, and Kaufman, Arie E. Wed .  
"CMed: Crowd Analytics for Medical Imaging Data".  United States.  https://doi.org/10.1109/tvcg.2019.2953026.  https://www.osti.gov/servlets/purl/1677652.

Copy to clipboard


                    
@article{osti_1677652,

  title        = {CMed: Crowd Analytics for Medical Imaging Data},

  author       = {Park, Ji Hwan and Nadeem, Saad and Boorboor, Saeed and Marino, Joseph and Kaufman, Arie E.},

  abstractNote = {In this work, we present a visual analytics framework, CMed, for exploring medical image data annotations acquired from crowdsourcing. CMed can be used to visualize, classify, and filter crowdsourced clinical data based on a number of different metrics such as detection rate, logged events, and clustering of the annotations. CMed provides several interactive linked visualization components to analyze the crowd annotation results for a particular video and the associated workers. Additionally, all results of an individual worker can be inspected using multiple linked views in our CMed framework. We allow a crowdsourcing application analyst to observe patterns and gather insights into the crowdsourced medical data, helping him/her design future crowdsourcing applications for optimal output from the workers. We demonstrate the efficacy of our framework with two medical crowdsourcing studies: polyp detection in virtual colonoscopy videos and lung nodule detection in CT thin-slab maximum intensity projection videos. We also provide experts' feedback to show the effectiveness of our framework. Lastly, we share the lessons we learned from our framework with suggestions for integrating our framework into a clinical workflow.},

  doi          = {10.1109/tvcg.2019.2953026},

  journal      = {IEEE Transactions on Visualization and Computer Graphics},

  number       = 6,

  volume       = 27,

  place        = {United States},

  year         = {2019},

  month        = {11}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1109/tvcg.2019.2953026

Other availability

Search WorldCat to find libraries that may hold this journal

Save / Share:

Export Metadata

Save to My Library

Similar Records in DOE PAGES and OSTI.GOV collections:

Improve Learning from Crowds via Generative Augmentation

Conference Chu, Zhendong ; Wang, Hongning - Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining

Crowdsourcing provides an efficient label collection schema for supervised machine learning. However, to control annotation cost, each instance in the crowdsourced data is typically annotated by a small number of annotators. This creates a sparsity issue and limits the quality of machine learning models trained on such data. In this paper, we study how to handle sparsity in crowdsourced data using data augmentation. Specifically, we propose to directly learn a classifier by augmenting the raw sparse annotations. We implement two principles of high-quality augmentation using Generative Adversarial Networks: 1) the generated annotations should follow the distribution of authentic ones, whichmore »« less
https://doi.org/10.1145/3447548.3467409

Full Text Available
WE-G-213-01: Roentgen and the Birth of Modern Medical Physics

Journal Article Sprawls, P. - Medical Physics

Roentgen and the Birth of Modern Medical Physics – Perry Sprawls Wilhelm Roentgen is well known for his discovery of x-radiation. What is less known and appreciated is his intensive research following the discovery to determine the characteristics of the “new kind of radiation” and demonstrate its great value for medical purposes. In this presentation we will imagine ourselves in Roentgen’s mind and follow his thinking, including questions and doubts, as he designs and conducts a series of innovative experiments that provided the foundation for the rapid growth of medical physics. Learning Objectives: Become familiar with the personal characteristics andmore »« less
https://doi.org/10.1118/1.4926041
Clinical evaluation of a medical high dynamic range display

Journal Article Marchessoux, Cedric ; Paepe, Lode de ; Vanovermeire, Olivier ; ... - Medical Physics

Purpose: Recent new medical displays do have higher contrast and higher luminance but do not have a High Dynamic Range (HDR). HDR implies a minimum luminance value close to zero. A medical HDR display prototype based on two Liquid Crystal layers has been developed. The goal of this study is to evaluate the potential clinical benefit of such display in comparison with a low dynamic range (LDR) display. Methods: The study evaluated the clinical performance of the displays in a search and detection task. Eight radiologists read chest x-ray images some of which contained simulated lung nodules. The study usedmore »« less
https://doi.org/10.1118/1.4953187
Learning from crowds with variational Gaussian processes

Journal Article Ruiz, Pablo ; Morales-Alvarez, Pablo ; Molina, Rafael ; ... - Pattern Recognition

Solving a supervised learning problem requires to label a training set. This task is traditionally performed by an expert, who provides a label for each sample. The proliferation of social web services (e.g., Amazon Mechanical Turk) has introduced an alternative crowdsourcing approach. Anybody with a computer can register in one of these services and label, either partially or completely, a dataset. The effort of labeling is then shared between a great number of annotators. However, this approach introduces scientifically challenging problems such as combining the unknown expertise of the annotators, handling disagreements on the annotated samples, or detecting the existencemore »« less
Cited by 21
https://doi.org/10.1016/j.patcog.2018.11.021

Full Text Available
Detecting and Analyzing Multiple Moving Objects in Crowded Environments with Coherent Motion Regions

Software Cheriyadat, Anil M.

Understanding the world around us from large-scale video data requires vision systems that can perform automatic interpretation. While human eyes can unconsciously perceive independent objects in crowded scenes and other challenging operating environments, automated systems have difficulty detecting, counting, and understanding their behavior in similar scenes. Computer scientists at ORNL have a developed a technology termed as "Coherent Motion Region Detection" that invloves identifying multiple indepedent moving objects in crowded scenes by aggregating low-level motion cues extracted from moving objects. Humans and other species exploit such low-level motion cues seamlessely to perform perceptual grouping for visual understanding. The algorithm detectsmore »« less

Similar Records