Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

Distilling Knowledge from Ensembles of Cluster-Constrained-Attention Multiple-Instance Learners for Whole Slide Image Classification

Conference ·
 [1];  [1];  [1];  [1];  [1];  [1]
  1. ORNL
+ Show Author Affiliations
The peculiar nature of whole slide imaging (WSI), digitizing conventional glass slides to obtain multiple high resolution images which capture microscopic details of a patient’s histopathological features, has garnered increased interest from the computer vision research community over the last two decades. Given the unique computational space and time complexity inherent to gigapixel-size whole slide image data, researchers have proposed novel machine learning algorithms to aid in the performance of diagnostic tasks in clinical pathology. One effective algorithm represents a Whole slide image as a bag of smaller image patches, which can be represented as low-dimension image patch embeddings. Weakly supervised deep-learning methods, such as cluster-constrained-attention multiple instance learning (CLAM), have shown promising results when combined with image patch embeddings. While traditional ensemble classifiers yield improved task performance, such methods come with a steep cost in model complexity. Through knowledge distillation, it is possible to retain some performance improvements from an ensemble, while minimizing costs to model complexity. In this work, we implement a weakly supervised ensemble using clustering-constrained-attention multiple-instance learners (CLAM), which uses attention and instance-level clustering to identify task salient regions and feature extraction in whole slides. By applying logit-based and attention-based knowledge distillation, we show it is possible to retain some performance improvements resulting from the ensemble at zero cost to model complexity.
Research Organization:
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
Sponsoring Organization:
USDOE; USDOE Office of Science (SC)
DOE Contract Number:
AC05-00OR22725
OSTI ID:
1965255
Country of Publication:
United States
Language:
English

References (18)

Ensemble learning: A survey journal February 2018
Data-efficient and weakly supervised computational pathology on whole-slide images journal March 2021
On the Efficacy of Knowledge Distillation conference October 2019
Artificial intelligence in digital pathology — new tools for diagnosis and precision oncology journal August 2019
Clinical-grade computational pathology using weakly supervised deep learning on whole slide images journal July 2019
An ensemble learning method based on deep neural network and group decision making journal March 2022
MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks conference June 2018
Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning journal September 2018
Deep Residual Learning for Image Recognition conference June 2016
A survey on ensemble learning journal August 2019
Quantized Convolutional Neural Networks for Mobile Devices conference June 2016
RMDL: Recalibrated multi-instance deep learning for whole slide gastric image classification journal December 2019
Neural Image Compression for Gigapixel Histopathology Image Analysis journal February 2021
Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer journal December 2017
Automatic detection of invasive ductal carcinoma in whole slide images with convolutional neural networks conference March 2014
Digital pathology and artificial intelligence journal May 2019
A Scalable Pipeline for Gigapixel Whole Slide Imaging Analysis on Leadership Class HPC Systems conference May 2022
Model compression
  • Buciluǎ, Cristian; Caruana, Rich; Niculescu-Mizil, Alexandru
  • Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD '06 https://doi.org/10.1145/1150402.1150464
conference January 2006

Similar Records

A comparison of histopathology imaging comprehension algorithms based on multiple instance learning
Conference · Sat Apr 01 00:00:00 EDT 2023 · OSTI ID:1969817
A Scalable Pipeline for Gigapixel Whole Slide Imaging Analysis on Leadership Class HPC Systems
Conference · Sun May 01 00:00:00 EDT 2022 · OSTI ID:1885343
Root identification in minirhizotron imagery with multiple instance learning
Journal Article · Tue Jun 23 20:00:00 EDT 2020 · Machine Vision and Applications · OSTI ID:1773853

Related Subjects