Recurrent and convolutional neural networks for sequential multispectral optoacoustic tomography ( MSOT ) imaging

Juhong, Aniwat; Li, Bo; Liu, Yifan; Yao, Cheng‐You; Yang, Chia‐Wei; Agnew, Dalen W.; Lei, Yu Leo; Luker, Gary D.; Bumpers, Harvey; Huang, Xuefei; Piyawattanametha, Wibool; Qiu, Zhen

doi:10.1002/jbio.202300142

Title: Recurrent and convolutional neural networks for sequential multispectral optoacoustic tomography ( MSOT ) imaging

Journal Article · 19 July 2023 · Journal of Biophotonics

DOI: https://doi.org/10.1002/jbio.202300142 · OSTI ID:1991655

^[1];

^[2];

^[3];

^[4];

^[5];

^[6];

^[7];

^[8];

^[9];

^[10]

Department of Electrical and Computer Engineering Michigan State University East Lansing Michigan USA, Institute for Quantitative Health Science and Engineering, Michigan State University East Lansing Michigan USA
Institute for Quantitative Health Science and Engineering, Michigan State University East Lansing Michigan USA, Department of Biomedical Engineering Michigan State University East Lansing Michigan USA
Institute for Quantitative Health Science and Engineering, Michigan State University East Lansing Michigan USA, Department of Chemistry Michigan State University East Lansing Michigan USA
Department of Pathobiology and Diagnostic Investigation College of Veterinary Medicine, Michigan State University East Lansing Michigan USA
Department of Periodontics and Oral Medicine University of Michigan Ann Arbor Michigan USA
Department of Radiology, Microbiology and Immunology, and Biomedical Engineering University of Michigan Ann Arbor Michigan USA
Department of Surgery Michigan State University East Lansing Michigan USA
Institute for Quantitative Health Science and Engineering, Michigan State University East Lansing Michigan USA, Department of Biomedical Engineering Michigan State University East Lansing Michigan USA, Department of Chemistry Michigan State University East Lansing Michigan USA
Institute for Quantitative Health Science and Engineering, Michigan State University East Lansing Michigan USA, Department of Biomedical Engineering School of Engineering, King Mongkut's Institute of Technology Ladkrabang (KMITL) Bangkok Thailand
Department of Electrical and Computer Engineering Michigan State University East Lansing Michigan USA, Institute for Quantitative Health Science and Engineering, Michigan State University East Lansing Michigan USA, Department of Biomedical Engineering Michigan State University East Lansing Michigan USA

Abstract Multispectral optoacoustic tomography (MSOT) is a beneficial technique for diagnosing and analyzing biological samples since it provides meticulous details in anatomy and physiology. However, acquiring high through‐plane resolution volumetric MSOT is time‐consuming. Here, we propose a deep learning model based on hybrid recurrent and convolutional neural networks to generate sequential cross‐sectional images for an MSOT system. This system provides three modalities (MSOT, ultrasound, and optoacoustic imaging of a specific exogenous contrast agent) in a single scan. This study used ICG‐conjugated nanoworms particles (NWs‐ICG) as the contrast agent. Instead of acquiring seven images with a step size of 0.1 mm, we can receive two images with a step size of 0.6 mm as input for the proposed deep learning model. The deep learning model can generate five other images with a step size of 0.1 mm between these two input images meaning we can reduce acquisition time by approximately 71%.

View Journal Article

Cite

Export

Save

Sponsoring Organization:: USDOE

OSTI ID:: 1991655

Journal Information:: Journal of Biophotonics, Journal Name: Journal of Biophotonics Journal Issue: 11 Vol. 16; ISSN 1864-063X

Publisher:: Wiley Blackwell (John Wiley & Sons)Copyright Statement

Country of Publication:: Germany

Language:: English

References (40)

LV‐GAN : A deep learning approach for limited‐view optoacoustic imaging based on hybrid datasets Lu, Tong; Chen, Tingting; Gao, Feng Journal of Biophotonics, Vol. 14, Issue 2 https://doi.org/10.1002/jbio.202000325	journal	November 2020
Enhanced Permeability and Retention (EPR) Effect for Anticancer Nanomedicine Drug Targeting Greish, Khaled Methods in Molecular Biology https://doi.org/10.1007/978-1-60761-609-2_3	book	January 2010
Deep Learning for Image Processing and Reconstruction to Enhance LED-Based Photoacoustic Imaging Sivasubramanian, Kathyayini; Xing, Lei LED-Based Photoacoustic Imaging https://doi.org/10.1007/978-981-15-3984-8_9	book	January 2020
Modality specific U-Net variants for biomedical image segmentation: a survey Punn, Narinder Singh; Agarwal, Sonali Artificial Intelligence Review, Vol. 55, Issue 7 https://doi.org/10.1007/s10462-022-10152-1	journal	March 2022
Identification of pancreatic tumors in vivo with ligand-targeted, pH responsive mesoporous silica nanoparticles by multispectral optoacoustic tomography Gurka, Marie K.; Pender, Dillon; Chuong, Phillip Journal of Controlled Release, Vol. 231 https://doi.org/10.1016/j.jconrel.2015.12.055	journal	June 2016
Quantitative, real-time in vivo tracking of magnetic nanoparticles using multispectral optoacoustic tomography (MSOT) imaging Anani, Tareq; Brannen, Andrew; Panizzi, Peter Journal of Pharmaceutical and Biomedical Analysis, Vol. 178 https://doi.org/10.1016/j.jpba.2019.112951	journal	January 2020
MultiResUNet : Rethinking the U-Net architecture for multimodal biomedical image segmentation Ibtehaz, Nabil; Rahman, M. Sohel Neural Networks, Vol. 121 https://doi.org/10.1016/j.neunet.2019.08.025	journal	January 2020
In-vivo handheld optoacoustic tomography of the human thyroid Dima, Alexander; Ntziachristos, Vasilis Photoacoustics, Vol. 4, Issue 2 https://doi.org/10.1016/j.pacs.2016.05.003	journal	June 2016
Indocyanine Green-Conjugated Superparamagnetic Iron Oxide Nanoworm for Multimodality Breast Cancer Imaging Yang, Chia-Wei; Liu, Kunli; Yao, Cheng-You ACS Applied Nano Materials, Vol. 5, Issue 12 https://doi.org/10.1021/acsanm.2c04687	journal	December 2022
Molecular Imaging by Means of Multispectral Optoacoustic Tomography (MSOT) Ntziachristos, Vasilis; Razansky, Daniel Chemical Reviews, Vol. 110, Issue 5 https://doi.org/10.1021/cr9002566	journal	May 2010
Multispectral opto-acoustic tomography of deep-seated fluorescent proteins in vivo Razansky, Daniel; Distel, Martin; Vinegoni, Claudio Nature Photonics, Vol. 3, Issue 7 https://doi.org/10.1038/nphoton.2009.98	journal	June 2009
Optoacoustics delineates murine breast cancer models displaying angiogenesis and vascular mimicry Quiros-Gonzalez, Isabel; Tomaszewski, Michal R.; Aitken, Sarah J. British Journal of Cancer, Vol. 118, Issue 8 https://doi.org/10.1038/s41416-018-0033-x	journal	March 2018
Detection of collagens by multispectral optoacoustic tomography as an imaging biomarker for Duchenne muscular dystrophy Regensburger, Adrian P.; Fonteyne, Lina M.; Jüngert, Jörg Nature Medicine, Vol. 25, Issue 12 https://doi.org/10.1038/s41591-019-0669-y	journal	December 2019
Automated screening of sickle cells using a smartphone-based microscope and deep learning de Haan, Kevin; Ceylan Koydemir, Hatice; Rivenson, Yair npj Digital Medicine, Vol. 3, Issue 1 https://doi.org/10.1038/s41746-020-0282-y	journal	May 2020
Deep learning optoacoustic tomography with sparse data Davoudi, Neda; Deán-Ben, Xosé Luís; Razansky, Daniel Nature Machine Intelligence, Vol. 1, Issue 10 https://doi.org/10.1038/s42256-019-0095-3	journal	September 2019
Comprehensive studies of pharmacokinetics and biodistribution of indocyanine green and liposomal indocyanine green by multispectral optoacoustic tomography Song, Wantong; Tang, Zhaohui; Zhang, Dawei RSC Advances, Vol. 5, Issue 5 https://doi.org/10.1039/C4RA09735A	journal	January 2015
A Deep Learning Model for Automated Segmentation of Fluorescence Cell images Aydın, Musa; Kiraz, Berna; Eren, Furkan Journal of Physics: Conference Series, Vol. 2191, Issue 1 https://doi.org/10.1088/1742-6596/2191/1/012003	journal	February 2022
Universal back-projection algorithm for photoacoustic computed tomography Xu, Minghua; Wang, Lihong V. Physical Review E, Vol. 71, Issue 1 https://doi.org/10.1103/PhysRevE.71.016706	journal	January 2005
Applications of photoacoustic sensing techniques Tam, Andrew C. Reviews of Modern Physics, Vol. 58, Issue 2, p. 381-431 https://doi.org/10.1103/RevModPhys.58.381	journal	April 1986
Spectral unmixing Keshava, N.; Mustard, J. F. IEEE Signal Processing Magazine, Vol. 19, Issue 1 https://doi.org/10.1109/79.974727	journal	January 2002
Photoacoustic Image Classification and Segmentation of Breast Cancer: A Feasibility Study Zhang, Jiayao; Chen, Bin; Zhou, Meng IEEE Access, Vol. 7 https://doi.org/10.1109/ACCESS.2018.2888910	journal	January 2019
Quantitative Photoacoustic Blood Oxygenation Imaging Using Deep Residual And Recurrent Neural Network Yang, Changchun; Lan, Hengrong; Zhong, Hongtao 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019) https://doi.org/10.1109/ISBI.2019.8759438	conference	April 2019
Generative Adversarial Networks: An Overview Creswell, Antonia; White, Tom; Dumoulin, Vincent IEEE Signal Processing Magazine, Vol. 35, Issue 1 https://doi.org/10.1109/MSP.2017.2765202	journal	January 2018
Unmixing Molecular Agents From Absorbing Tissue in Multispectral Optoacoustic Tomography Tzoumas, Stratis; Deliolanis, Nikolaos; Morscher, Stefan IEEE Transactions on Medical Imaging, Vol. 33, Issue 1 https://doi.org/10.1109/TMI.2013.2279994	journal	January 2014
Photoacoustic Source Detection and Reflection Artifact Removal Enabled by Deep Learning Allman, Derek; Reiter, Austin; Bell, Muyinatu A. Lediju IEEE Transactions on Medical Imaging, Vol. 37, Issue 6 https://doi.org/10.1109/TMI.2018.2829662	journal	June 2018
Deep Learning for Automatic Segmentation of Hybrid Optoacoustic Ultrasound (OPUS) Images Lafci, Berkan; Mercep, Elena; Morscher, Stefan IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, Vol. 68, Issue 3 https://doi.org/10.1109/TUFFC.2020.3022324	journal	March 2021
Toward accurate quantitative photoacoustic imaging: learning vascular blood oxygen saturation in three dimensions Bench, Ciaran; Hauptmann, Andreas; Cox, Ben Journal of Biomedical Optics, Vol. 25, Issue 08 https://doi.org/10.1117/1.JBO.25.8.085003	journal	August 2020
A tumor microenvironment–induced absorption red-shifted polymer nanoparticle for simultaneously activated photoacoustic imaging and photothermal therapy Wang, Shulong; Zhang, Liangliang; Zhao, Jingjin Science Advances, Vol. 7, Issue 12 https://doi.org/10.1126/sciadv.abe3588	journal	March 2021
Photoacoustic Tomography: In Vivo Imaging from Organelles to Organs Wang, L. V.; Hu, S. Science, Vol. 335, Issue 6075 https://doi.org/10.1126/science.1216210	journal	March 2012
Mesoscopic and Macroscopic Optoacoustic Imaging of Cancer Taruttis, Adrian; van Dam, Gooitzen M.; Ntziachristos, Vasilis Cancer Research, Vol. 75, Issue 8 https://doi.org/10.1158/0008-5472.CAN-14-2522	journal	April 2015
Oxygen-Enhanced and Dynamic Contrast-Enhanced Optoacoustic Tomography Provide Surrogate Biomarkers of Tumor Vascular Function, Hypoxia, and Necrosis Tomaszewski, Michal R.; Gehrung, Marcel; Joseph, James Cancer Research, Vol. 78, Issue 20 https://doi.org/10.1158/0008-5472.CAN-18-1033	journal	October 2018
Volumetric Optoacoustic Imaging Unveils High-Resolution Patterns of Acute and Cyclic Hypoxia in a Murine Model of Breast Cancer Ron, Avihai; Deán-Ben, Xosé Luís; Gottschalk, Sven Cancer Research, Vol. 79, Issue 18 https://doi.org/10.1158/0008-5472.CAN-18-3769	journal	September 2019
Multispectral Optoacoustic Tomography (MSOT) of Human Breast Cancer Diot, Gael; Metz, Stephan; Noske, Aurelia Clinical Cancer Research, Vol. 23, Issue 22 https://doi.org/10.1158/1078-0432.CCR-16-3200	journal	November 2017
Reduced Breast Tumor Growth after Immunization with a Tumor-Restricted MUC1 Glycopeptide Conjugated to Tetanus Toxoid Stergiou, Natascha; Gaidzik, Nikola; Heimes, Anne-Sophie Cancer Immunology Research, Vol. 7, Issue 1 https://doi.org/10.1158/2326-6066.CIR-18-0256	journal	January 2019
Long Short-Term Memory Hochreiter, Sepp; Schmidhuber, Jürgen Neural Computation, Vol. 9, Issue 8 https://doi.org/10.1162/neco.1997.9.8.1735	journal	November 1997
Deep learning improves contrast in low-fluence photoacoustic imaging Hariri, Ali; Alipour, Kamran; Mantri, Yash Biomedical Optics Express, Vol. 11, Issue 6 https://doi.org/10.1364/BOE.395683	journal	May 2020
Hybrid deep learning network for vascular segmentation in photoacoustic imaging Yuan, Alan Yilun; Gao, Yang; Peng, Liangliang Biomedical Optics Express, Vol. 11, Issue 11 https://doi.org/10.1364/BOE.409246	journal	October 2020
Real-time handheld multispectral optoacoustic imaging Buehler, Andreas; Kacprowicz, Marcin; Taruttis, Adrian Optics Letters, Vol. 38, Issue 9 https://doi.org/10.1364/OL.38.001404	journal	April 2013
End-to-end deep neural network for optical inversion in quantitative photoacoustic imaging Cai, Chuangjian; Deng, Kexin; Ma, Cheng Optics Letters, Vol. 43, Issue 12 https://doi.org/10.1364/OL.43.002752	journal	June 2018
Investigating the Effect of Chemical Structure of Semiconducting Polymer Nanoparticle on Photothermal Therapy and Photoacoustic Imaging Li, Dongdong; Zhang, Guobing; Xu, Weiguo Theranostics, Vol. 7, Issue 16 https://doi.org/10.7150/thno.19538	journal	January 2017

Similar Records

Resolution‐dependent behavior of subgrid‐scale vertical transport in the Z hang‐ M c F arlane convection parameterization

Journal Article · 2015 · Journal of Advances in Modeling Earth Systems · OSTI ID:1295962

Xiao, Heng; Gustafson, Jr., William I.; Hagos, Samson M.; +2 more

Multihead Attention U‐Net for Magnetic Particle Imaging–Computed Tomography Image Segmentation

Journal Article · 2024 · Advanced Intelligent Systems · OSTI ID:2440444

Juhong, Aniwat; Li, Bo; Liu, Yifan; +9 more

Snapshot multispectral imaging using a diffractive optical network

Journal Article · 2023 · Light, Science & Applications · OSTI ID:1968626

Mengu, Deniz; Tabassum, Anika; Jarrahi, Mona; +1 more

Title: Recurrent and convolutional neural networks for sequential multispectral optoacoustic tomography ( MSOT ) imaging

Citation Formats

References (40)

Similar Records

Related Subjects