YOLO2U-Net: Detection-guided 3D instance segmentation for microscopy

Ziabari, Amirkoushyar; Rose, Derek C.; Shirinifard, Abbas; Solecki, David

doi:10.1016/j.patrec.2024.03.015

YOLO2U-Net: Detection-guided 3D instance segmentation for microscopy

Journal Article · Sun Mar 24 00:00:00 EDT 2024 · Pattern Recognition Letters

DOI:https://doi.org/10.1016/j.patrec.2024.03.015· OSTI ID:2333861

^[1]; ^[1]; Shirinifard, Abbas ^[2]; Solecki, David ^[2]

Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
St. Jude Children’s Hospital, Memphis, TN (United States)

Microscopy imaging techniques are instrumental for characterization and analysis of biological structures. As these techniques typically render 3D visualization of cells by stacking 2D projections, issues such as out-of-plane excitation and low resolution in the z-axis may pose challenges (even for human experts) to detect individual cells in 3D volumes as these non-overlapping cells may appear as overlapping. In this paper a comprehensive method for accurate 3D instance segmentation of cells in the brain tissue is introduced. The proposed method combines the 2D YOLO detection method with a multi-view fusion algorithm to construct a 3D localization of the cells. Next, the 3D bounding boxes along with the data volume are input to a 3D U-Net network that is designed to segment the primary cell in each 3D bounding box, and in turn, to carry out instance segmentation of cells in the entire volume. The promising performance of the proposed method is shown in comparison with current deep learning-based 3D instance segmentation methods.

View Accepted Manuscript (DOE)

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Organization:: American Lebanese Syrian Associated Charities (ALSAC); National Institute of Neurological Disorders (NINDS).; USDOE

Grant/Contract Number:: AC05-00OR22725

OSTI ID:: 2333861

Journal Information:: Pattern Recognition Letters, Journal Name: Pattern Recognition Letters Journal Issue: 1 Vol. 181; ISSN 0167-8655

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

References (16)

The Pascal Visual Object Classes Challenge: A Retrospective Everingham, Mark; Eslami, S. M. Ali; Van Gool, Luc International Journal of Computer Vision, Vol. 111, Issue 1 https://doi.org/10.1007/s11263-014-0733-5	journal	June 2014
Multi-Scale Modeling of Tissues Using CompuCell3D Swat, Maciej H.; Thomas, Gilberto L.; Belmonte, Julio M. Methods in Cell Biology https://doi.org/10.1016/B978-0-12-388403-9.00013-8	book	April 2012
Learn to segment single cells with deep distance estimator and deep cell detector Wang, Weikang; Taft, David A.; Chen, Yi-Jiun Computers in Biology and Medicine, Vol. 108 https://doi.org/10.1016/j.compbiomed.2019.04.006	journal	May 2019
Benchmarking HEp-2 specimen cells classification using linear discriminant analysis on higher order spectra features of cell shape Al-Dulaimi, Khamael; Chandran, Vinod; Nguyen, Kien Pattern Recognition Letters, Vol. 125 https://doi.org/10.1016/j.patrec.2019.06.020	journal	July 2019
BshapeNet: Object detection and instance segmentation with bounding shape masks Kang, Ba Rom; Lee, Hyunku; Park, Keunju Pattern Recognition Letters, Vol. 131 https://doi.org/10.1016/j.patrec.2020.01.024	journal	March 2020
Contour deformation network for instance segmentation Lv, Kefeng; Zhang, Yongsheng; Yu, Ying Pattern Recognition Letters, Vol. 159 https://doi.org/10.1016/j.patrec.2022.05.025	journal	July 2022
A boundary-preserving conditional convolution network for instance segmentation Zhang, Qing-Long; Yang, Yu-Bin Pattern Recognition Letters, Vol. 163 https://doi.org/10.1016/j.patrec.2022.09.003	journal	November 2022
DeepSynth: Three-dimensional nuclear segmentation of biological images using neural networks trained with synthetic data Dunn, Kenneth W.; Fu, Chichen; Ho, David Joon Scientific Reports, Vol. 9, Issue 1 https://doi.org/10.1038/s41598-019-54244-5	journal	December 2019
Fully convolutional networks for semantic segmentation Long, Jonathan; Shelhamer, Evan; Darrell, Trevor 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2015.7298965	conference	June 2015
Deep Watershed Transform for Instance Segmentation Bai, Min; Urtasun, Raquel 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2017.305	conference	July 2017
YOLO9000: Better, Faster, Stronger Redmon, Joseph; Farhadi, Ali 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2017.690	conference	July 2017
Mask-RCNN and U-Net Ensembled for Nuclei Segmentation Vuola, Aarno Oskar; Akram, Saad Ullah; Kannala, Juho 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019) https://doi.org/10.1109/ISBI.2019.8759574	conference	April 2019
Cell Nuclei and Cytoplasm Joint Segmentation Using the Sliding Band Filter Quelhas, Pedro; Marcuzzo, Monica; Mendonca, Ana Maria IEEE Transactions on Medical Imaging, Vol. 29, Issue 8 https://doi.org/10.1109/TMI.2010.2048253	journal	August 2010
Star-convex Polyhedra for 3D Object Detection and Segmentation in Microscopy Weigert, Martin; Schmidt, Uwe; Haase, Robert 2020 IEEE Winter Conference on Applications of Computer Vision (WACV) https://doi.org/10.1109/WACV45572.2020.9093435	conference	March 2020
Swin Unet3D: a three-dimensional medical image segmentation network combining vision transformer and convolution Cai, Yimin; Long, Yuqing; Han, Zhenggong BMC Medical Informatics and Decision Making, Vol. 23, Issue 1 https://doi.org/10.1186/s12911-023-02129-z	journal	February 2023
StarDist Image Segmentation Improves Circulating Tumor Cell Detection Stevens, Michiel; Nanou, Afroditi; Terstappen, Leon W. M. M. Cancers, Vol. 14, Issue 12 https://doi.org/10.3390/cancers14122916	journal	June 2022

Similar Records

A 2.5d Yolo-Based Fusion Algorithm for 3d Localization Of Cells

Conference · Fri Nov 01 00:00:00 EDT 2019 · OSTI ID:1615203

From Voxels to Knowledge: A Practical Guide to the Segmentation of Complex Electron Microscopy 3D-Data

Journal Article · Tue Aug 12 20:00:00 EDT 2014 · Journal of Visualized Experiments · OSTI ID:1628638

Related Subjects

3D instance segmentation
47 OTHER INSTRUMENTATION
cell microscopy
deep learning

YOLO2U-Net: Detection-guided 3D instance segmentation for microscopy

Citation Formats

References (16)

Similar Records

Related Subjects