Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

Classifying T cell activity in autofluorescence intensity images with convolutional neural networks

Journal Article · · Journal of Biophotonics
 [1];  [2];  [3];  [4]
  1. Univ. of Wisconsin‐Madison, Madison, WI (United States). Dept. of Computer Sciences; Morgridge Institute for Research, Madison, WI (United States); DOE/OSTI
  2. Morgridge Institute for Research, Madison, WI (United States)
  3. Morgridge Institute for Research, Madison, WI (United States); Univ. of Wisconsin‐Madison, Madison, WI (United States). Dept. of Biomedical Engineering
  4. Univ. of Wisconsin‐Madison, Madison, WI (United States). Dept. of Computer Sciences; Morgridge Institute for Research Madison, WI (United States); Univ. of Wisconsin‐Madison, Madison, WI (United States). Dept. of Biostatistics and Medical Informatics
+ Show Author Affiliations
The importance of T cells in immunotherapy has motivated developing technologies to improve therapeutic efficacy. One objective is assessing antigen-induced T cell activation because only functionally active T cells are capable of killing the desired targets. Autofluorescence imaging can distinguish T cell activity states in a non-destructive manner by detecting endogenous changes in metabolic co-enzymes such as NAD(P)H. However, recognizing robust activity patterns is computationally challenging in the absence of exogenous labels. We demonstrate machine learning methods that can accurately classify T cell activity across human donors from NAD(P)H intensity images. Using 8260 cropped single-cell images from six donors, we evaluate classifiers ranging from traditional models that use previously-extracted image features to convolutional neural networks (CNNs) pre-trained on general non-biological images. Adapting pre-trained CNNs for the T cell activity classification task provides substantially better performance than traditional models or a simple CNN trained with the autofluorescence images alone. Visualizing the images with dimension reduction provides intuition into why the CNNs achieve higher accuracy than other approaches. Our image processing and classifier training software is available at https://github.com/gitter-lab/t-cell-classification.
Research Organization:
Argonne National Lab (ANL)
Sponsoring Organization:
Morgridge Institute for Research; National Institutes of Health (NIH), National Cancer Institute (NCI); USDOE Office of Science (SC); University of Wisconsin‐Madison
Grant/Contract Number:
AC02-06CH11357
OSTI ID:
1623563
Journal Information:
Journal of Biophotonics, Journal Name: Journal of Biophotonics Journal Issue: 3 Vol. 13; ISSN 1864-063X
Publisher:
WileyCopyright Statement
Country of Publication:
United States
Language:
English

References (64)

Deep Learning in Image Cytometry: A Review journal December 2018
Label‐Free Identification of White Blood Cells Using Machine Learning journal May 2019
Detection of the T cell activation state using nonlinear optical microscopy journal November 2018
Assessment of automatic rib fracture detection on chest CT using a deep learning algorithm journal October 2022
Intelligent Image-Activated Cell Sorting journal September 2018
Repurposing High-Throughput Image Assays Enables Biological Activity Prediction for Drug Discovery journal May 2018
Metabolism in T cell activation and differentiation journal June 2010
Accurate Prediction of Biological Assays with High-Throughput Microscopy Images and Convolutional Networks journal October 2018
Classification evaluation journal July 2016
Prospective identification of hematopoietic lineage choice by deep learning journal February 2017
The blockade of immune checkpoints in cancer immunotherapy journal March 2012
Adoptive immunotherapy for cancer: harnessing the T cell response journal March 2012
Reconstructing cell cycle and disease progression using deep learning journal September 2017
Genetic screening identifies a SUMO protease dynamically maintaining centromeric chromatin journal January 2020
Leveraging machine vision in cell-based diagnostics to do more with less journal April 2019
U-Net: deep learning for cell counting, detection, and morphometry journal December 2018
Deep learning for cellular image analysis journal May 2019
Identification of non-activated lymphocytes using three-dimensional refractive index tomography and machine learning journal July 2017
Real-time Image Processing for Microscopy-based Label-free Imaging Flow Cytometry in a Microfluidic Chip journal September 2017
Deep Learning in Label-free Cell Classification journal March 2016
In Vivo Visualization of Stromal Macrophages via label-free FLIM-based metabolite imaging journal May 2016
Noninvasive detection of macrophage activation with single-cell resolution through machine learning journal March 2018
Single-cell imaging of CAR T cell activity in vivo reveals extensive functional and anatomical heterogeneity journal April 2019
Classifying and segmenting microscopy images with deep multiple instance learning journal June 2016
A multi-scale convolutional neural network for phenotyping high-content cellular images journal February 2017
Gradient-based learning applied to document recognition journal January 1998
End-to-end optimization of nonlinear transform codes for perceptual quality conference January 2016
Imaging Mitochondria in Living Corneal Endothelial Cells Using Autofluorescence Microscopy journal November 2007
Improved Automated Detection of Diabetic Retinopathy on a Publicly Available Dataset Through Integration of Deep Learning journal October 2016
High-Content Analysis of Breast Cancer Using Single-Cell Deep Transfer Learning journal January 2016
Label-free characterization of white blood cells using fluorescence lifetime imaging and flow-cytometry: molecular heterogeneity and erythrophagocytosis [Invited] journal January 2019
Accurate Classification of Protein Subcellular Localization from High-Throughput Microscopy Images Using Deep Learning journal April 2017
UMAP: Uniform Manifold Approximation and Projection journal September 2018
Administration of CD4+CD25highCD127- Regulatory T Cells Preserves  -Cell Function in Type 1 Diabetes in Children journal June 2012
Striving for Simplicity: The All Convolutional Net preprint January 2014
Classification of Human White Blood Cells Using Machine Learning for Stain‐Free Imaging Flow Cytometry journal November 2019
U-Net: Convolutional Networks for Biomedical Image Segmentation
  • Ronneberger, Olaf; Fischer, Philipp; Brox, Thomas
  • Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III https://doi.org/10.1007/978-3-319-24574-4_28
book November 2015
Stem cell motion-tracking by using deep neural networks with multi-output journal November 2017
Multiobjective evolutionary algorithms to identify highly autocorrelated areas: the case of spatial distribution in financially compromised farms journal January 2011
A Structured Tumor-Immune Microenvironment in Triple Negative Breast Cancer Revealed by Multiplexed Ion Beam Imaging journal September 2018
NWChem: A comprehensive and scalable open-source solution for large scale molecular simulations journal September 2010
Reagent-Free and Rapid Assessment of T Cell Activation State Using Diffraction Phase Microscopy and Deep Learning journal February 2019
IL-33 expression in response to SARS-CoV-2 correlates with seropositivity in COVID-19 convalescent individuals journal April 2021
Artificial intelligence for decision support in acute stroke — current roles and potential journal August 2020
Content-aware image restoration: pushing the limits of fluorescence microscopy journal November 2018
DeepCell Kiosk: scaling deep learning–enabled cellular image analysis with Kubernetes journal January 2021
Opportunities and obstacles for deep learning in biology and medicine journal April 2018
Opportunities and obstacles for deep learning in biology and medicine posted_content January 2018
ChestX-Ray8: Hospital-Scale Chest X-Ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases conference July 2017
Transfer learning of a convolutional neural network for HEp-2 cell image classification
  • Phan, Ha Tran Hong; Kumar, Ashnil; Kim, Jinman
  • 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI 2016), 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI) https://doi.org/10.1109/ISBI.2016.7493483
conference April 2016
ImageNet: A large-scale hierarchical image database
  • Deng, Jia; Dong, Wei; Socher, Richard
  • 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPR Workshops), 2009 IEEE Conference on Computer Vision and Pattern Recognition https://doi.org/10.1109/cvpr.2009.5206848
conference June 2009
Automatic white blood cell classification using pre-trained deep learning models: ResNet and Inception conference April 2018
Single-Cell Phenotype Classification Using Deep Convolutional Neural Networks journal July 2016
Transfer Learning with Deep Convolutional Neural Networks for Classifying Cellular Morphological Changes journal January 2019
CellProfiler: image analysis software for identifying and quantifying cell phenotypes journal October 2006
Label-free characterization of white blood cells using fluorescence lifetime imaging and flow-cytometry: molecular heterogeneity and erythrophagocytosis [Invited] journal January 2019
Binder 2.0 - Reproducible, interactive, sharable environments for science at scale conference January 2018
Transfer Learning with Deep Convolutional Neural Networks for Classifying Cellular Morphological Changes collection January 2019
Jupyter Notebooks – a publishing format for reproducible computational workflows book January 2021
ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases text January 2017
Noninvasive Detection Of Macrophage Activation With Single-Cell Resolution Through Machine Learning dataset January 2018
Noninvasive Detection Of Macrophage Activation With Single-Cell Resolution Through Machine Learning dataset January 2018
Transfer learning with deep convolutional neural networks for classifying cellular morphological changes image January 2019
U-Net: deep learning for cell counting, detection, and morphometry text January 2019

Similar Records

Classifying Radio Galaxies with the Convolutional Neural Network
Journal Article · Thu Jun 01 00:00:00 EDT 2017 · Astrophysical Journal, Supplement Series · OSTI ID:22661117
AutoSourceID-Classifier: Star-galaxy classification using a convolutional neural network with spatial information
Journal Article · Thu Nov 30 19:00:00 EST 2023 · Astronomy and Astrophysics · OSTI ID:2582047
Convolutional neural network identification of galaxy post-mergers in UNIONS using IllustrisTNG
Journal Article · Thu Mar 18 20:00:00 EDT 2021 · Monthly Notices of the Royal Astronomical Society · OSTI ID:1806345

Related Subjects

Biochemistry & Molecular Biology
Biophysics
Optics