skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Saliency-driven system models for cell analysis with deep learning

Abstract

Saliency refers to the visual perception quality that makes objects in a scene to stand out from others and attract attention. While computational saliency models can simulate the expert's visual attention, there is little evidence about how these models perform when used to predict the cytopathologist's eye fixations. Saliency models may be the key to instrumenting fast object detection on large Pap smear slides under real noisy conditions, artifacts, and cell occlusions. This paper describes how our computational schemes retrieve regions of interest (ROI) of clinical relevance using visual attention models. We also compare the performance of different computed saliency models as part of cell screening tasks, aiming to design a computer-aided diagnosis systems that supports cytopathologists. We record eye fixation maps from cytopathologists at work, and compare with 13 different saliency prediction algorithms, including deep learning. We develop cell-specific convolutional neural networks (CNN) to investigate the impact of bottom-up and top-down factors on saliency prediction from real routine exams. By combining the eye tracking data from pathologists with computed saliency models, we assess algorithms reliability in identifying clinically relevant cells. The proposed cell-specific CNN model outperforms all other saliency prediction methods, particularly regarding the number of false positives. Ourmore » algorithm also detects the most clinically relevant cells, which are among the three top salient regions, with accuracy above 98% for all diseases, except carcinoma (87%). Bottom-up methods performed satisfactorily, with saliency maps that enabled ROI detection above 75% for carcinoma and 86% for other pathologies. ROIs extraction using our saliency prediction methods enabled ranking the most relevant clinical areas within the image, a viable data reduction strategy to guide automatic analyses of Pap smear slides. Top-down factors for saliency prediction on cell images increases the accuracy of the estimated maps while bottom-up algorithms proved to be useful for predicting the cytopathologist's eye fixations depending on parameters, such as the number of false positive and negative. Finally, our contributions are: comparison among 13 state-of-the-art saliency models to cytopathologists' visual attention and deliver a method that the associate the most conspicuous regions to clinically relevant cells.« less

Authors:
ORCiD logo; ORCiD logo; ; ; ; ; ; ; ORCiD logo
Publication Date:
Research Org.:
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Sponsoring Org.:
USDOE Office of Science (SC); CAPES/CNPq-PVE; Coordenacao de Aperfeicoamento de Pessoal de Nıvel Superior - Brasil (CAPES); PPSUS/FAPEMIG; Propp/UFOP; Moore-Sloan Foundation
OSTI Identifier:
1813638
Alternate Identifier(s):
OSTI ID: 1561344; OSTI ID: 1604715
Grant/Contract Number:  
AC02-05CH11231
Resource Type:
Journal Article: Published Article
Journal Name:
Computer Methods and Programs in BioMedicine
Additional Journal Information:
Journal Name: Computer Methods and Programs in BioMedicine Journal Volume: 182 Journal Issue: C; Journal ID: ISSN 0169-2607
Publisher:
Elsevier
Country of Publication:
Netherlands
Language:
English
Subject:
60 APPLIED LIFE SCIENCES; 97 MATHEMATICS AND COMPUTING; saliency prediction; convolutional neural network; cell analysis; eye tracking experiment; cervical

Citation Formats

Ferreira, Daniel S., Ramalho, Geraldo L. B., Torres, Débora, Tobias, Alessandra H. G., Rezende, Mariana T., Medeiros, Fátima N. S., Bianchi, Andrea G. C., Carneiro, Cláudia M., and Ushizima, Daniela M. Saliency-driven system models for cell analysis with deep learning. Netherlands: N. p., 2019. Web. doi:10.1016/j.cmpb.2019.105053.
Ferreira, Daniel S., Ramalho, Geraldo L. B., Torres, Débora, Tobias, Alessandra H. G., Rezende, Mariana T., Medeiros, Fátima N. S., Bianchi, Andrea G. C., Carneiro, Cláudia M., & Ushizima, Daniela M. Saliency-driven system models for cell analysis with deep learning. Netherlands. https://doi.org/10.1016/j.cmpb.2019.105053
Ferreira, Daniel S., Ramalho, Geraldo L. B., Torres, Débora, Tobias, Alessandra H. G., Rezende, Mariana T., Medeiros, Fátima N. S., Bianchi, Andrea G. C., Carneiro, Cláudia M., and Ushizima, Daniela M. 2019. "Saliency-driven system models for cell analysis with deep learning". Netherlands. https://doi.org/10.1016/j.cmpb.2019.105053.
@article{osti_1813638,
title = {Saliency-driven system models for cell analysis with deep learning},
author = {Ferreira, Daniel S. and Ramalho, Geraldo L. B. and Torres, Débora and Tobias, Alessandra H. G. and Rezende, Mariana T. and Medeiros, Fátima N. S. and Bianchi, Andrea G. C. and Carneiro, Cláudia M. and Ushizima, Daniela M.},
abstractNote = {Saliency refers to the visual perception quality that makes objects in a scene to stand out from others and attract attention. While computational saliency models can simulate the expert's visual attention, there is little evidence about how these models perform when used to predict the cytopathologist's eye fixations. Saliency models may be the key to instrumenting fast object detection on large Pap smear slides under real noisy conditions, artifacts, and cell occlusions. This paper describes how our computational schemes retrieve regions of interest (ROI) of clinical relevance using visual attention models. We also compare the performance of different computed saliency models as part of cell screening tasks, aiming to design a computer-aided diagnosis systems that supports cytopathologists. We record eye fixation maps from cytopathologists at work, and compare with 13 different saliency prediction algorithms, including deep learning. We develop cell-specific convolutional neural networks (CNN) to investigate the impact of bottom-up and top-down factors on saliency prediction from real routine exams. By combining the eye tracking data from pathologists with computed saliency models, we assess algorithms reliability in identifying clinically relevant cells. The proposed cell-specific CNN model outperforms all other saliency prediction methods, particularly regarding the number of false positives. Our algorithm also detects the most clinically relevant cells, which are among the three top salient regions, with accuracy above 98% for all diseases, except carcinoma (87%). Bottom-up methods performed satisfactorily, with saliency maps that enabled ROI detection above 75% for carcinoma and 86% for other pathologies. ROIs extraction using our saliency prediction methods enabled ranking the most relevant clinical areas within the image, a viable data reduction strategy to guide automatic analyses of Pap smear slides. Top-down factors for saliency prediction on cell images increases the accuracy of the estimated maps while bottom-up algorithms proved to be useful for predicting the cytopathologist's eye fixations depending on parameters, such as the number of false positive and negative. Finally, our contributions are: comparison among 13 state-of-the-art saliency models to cytopathologists' visual attention and deliver a method that the associate the most conspicuous regions to clinically relevant cells.},
doi = {10.1016/j.cmpb.2019.105053},
url = {https://www.osti.gov/biblio/1813638}, journal = {Computer Methods and Programs in BioMedicine},
issn = {0169-2607},
number = C,
volume = 182,
place = {Netherlands},
year = {2019},
month = {12}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record at https://doi.org/10.1016/j.cmpb.2019.105053

Save / Share:

Works referenced in this record:

Keyframe extraction from laparoscopic videos based on visual saliency detection
journal, October 2018


Use of Ranks in One-Criterion Variance Analysis
journal, December 1952


A survey on deep learning for big data
journal, July 2018


Learning Discriminative Subspaces on Random Contrasts for Image Saliency Analysis
journal, May 2017


Toward a unified chromatic induction model
journal, October 2010


Attentive Systems: A Survey
journal, September 2017


DeepFix: A Fully Convolutional Neural Network for Predicting Human Eye Fixations
journal, September 2017


What Do Different Evaluation Metrics Tell Us About Saliency Models?
journal, March 2019


Top-down saliency detection driven by visual classification
journal, July 2018


State of the Art: Eye-Tracking Studies in Medical Imaging
journal, January 2018


Modeling eye movement patterns to characterize perceptual skill in image-based diagnostic reasoning processes
journal, October 2016


Predicting Human Eye Fixations via an LSTM-Based Saliency Attentive Model
journal, October 2018


Visual attention: The past 25 years
journal, July 2011


Salient Object Detection: A Benchmark
journal, December 2015


Morphometry in the Cytological Diagnosis of Cervical Smears
journal, January 2014


Methods for comparing scanpaths and saliency maps: strengths and weaknesses
journal, July 2012


A model of saliency-based visual attention for rapid scene analysis
journal, January 1998


Learning to Predict Eye Fixations via Multiresolution Convolutional Neural Networks
journal, February 2018


Exploring visual attention and saliency modeling for task-based visual analysis
journal, May 2018


A review of image analysis and machine learning techniques for automated cervical cancer screening from pap-smear images
journal, October 2018


A Deep Spatial Contextual Long-Term Recurrent Convolutional Network for Saliency Detection
journal, July 2018


Where Do Neurologists Look When Viewing Brain CT Images? An Eye-Tracking Study Involving Stroke Cases
journal, December 2011


Contextual guidance of eye movements and attention in real-world scenes: The role of global features in object search.
journal, October 2006


State-of-the-Art in Visual Attention Modeling
journal, January 2013


The Pap Test and Bethesda 2014
journal, January 2015