Physics-assisted generative adversarial network for X-ray tomography

Guo, Zhen; Song, Jung Ki; Barbastathis, George; Glinsky, Michael E.; Vaughan, Courtenay T.; Larson, Kurt W.; Alpert, Bradley K.; Levine, Zachary H.

doi:10.1364/OE.460208

Title: Physics-assisted generative adversarial network for X-ray tomography

Journal Article · Fri Jun 10 00:00:00 EDT 2022 · Optics Express

DOI:https://doi.org/10.1364/OE.460208· OSTI ID:1871842

; Song, Jung Ki; Barbastathis, George; Glinsky, Michael E.; Vaughan, Courtenay T.; Larson, Kurt W.; Alpert, Bradley K.;

X-ray tomography is capable of imaging the interior of objects in three dimensions non-invasively, with applications in biomedical imaging, materials science, electronic inspection, and other fields. The reconstruction process can be an ill-conditioned inverse problem, requiring regularization to obtain satisfactory results. Recently, deep learning has been adopted for tomographic reconstruction. Unlike iterative algorithms which require a distribution that is known a priori , deep reconstruction networks can learn a prior distribution through sampling the training distributions. In this work, we develop a Physics-assisted Generative Adversarial Network (PGAN), a two-step algorithm for tomographic reconstruction. In contrast to previous efforts, our PGAN utilizes maximum-likelihood estimates derived from the measurements to regularize the reconstruction with both known physics and the learned prior. Compared with methods with less physics assisting in training, PGAN can reduce the photon requirement with limited projection angles to achieve a given error rate. The advantages of using a physics-assisted learned prior in X-ray tomography may further enable low-photon nanoscale imaging.

View Journal Article

Cite

Export

Save

Research Organization:: Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA); USDOE Advanced Research Projects Agency - Energy (ARPA-E); Singapore’s National Research Foundation; Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA)

Grant/Contract Number:: NA0003525; FA8050-17-C-9113; NRF2019-THE002-0006; D2019-1908080004; D2019-1906200003

OSTI ID:: 1871842

Alternate ID(s):: OSTI ID: 1872055

Report Number(s):: SAND2022-7407J; OPEXFF

Journal Information:: Optics Express, Journal Name: Optics Express Vol. 30 Journal Issue: 13; ISSN 1094-4087

Publisher:: Optical Society of AmericaCopyright Statement

Country of Publication:: United States

Language:: English

References (49)

Framing U-Net via Deep Convolutional Framelets: Application to Sparse-View CT Han, Yoseob; Ye, Jong Chul IEEE Transactions on Medical Imaging, Vol. 37, Issue 6 https://doi.org/10.1109/TMI.2018.2823768	journal	June 2018
Adaptive Batch Normalization for practical domain adaptation Li, Yanghao; Wang, Naiyan; Shi, Jianping Pattern Recognition, Vol. 80 https://doi.org/10.1016/j.patcog.2018.03.005	journal	August 2018
Multi-energy CT reconstruction using tensor nonlocal similarity and spatial sparsity regularization Zhang, Wenkun; Liang, Ningning; Wang, Zhe Quantitative Imaging in Medicine and Surgery, Vol. 10, Issue 10 https://doi.org/10.21037/qims-20-594	journal	October 2020
High-resolution limited-angle phase tomography of dense layered objects using deep neural networks Goy, Alexandre; Rughoobur, Girish; Li, Shuai Proceedings of the National Academy of Sciences, Vol. 116, Issue 40 https://doi.org/10.1073/pnas.1821378116	journal	September 2019
Tomographic image reconstruction from limited projections using iterative revisions in image and transform spaces Sato, Takuso; Norton, Stephen J.; Linzer, Melvin Applied Optics, Vol. 20, Issue 3 https://doi.org/10.1364/AO.20.000395	journal	January 1981
Deep Learning Computed Tomography: Learning Projection-Domain Weights From Image Domain in Limited Angle Problems Wurfl, Tobias; Hoffmann, Mathis; Christlein, Vincent IEEE Transactions on Medical Imaging, Vol. 37, Issue 6 https://doi.org/10.1109/TMI.2018.2833499	journal	June 2018
Deep-learning tomography Araya-Polo, Mauricio; Jennings, Joseph; Adler, Amir The Leading Edge, Vol. 37, Issue 1 https://doi.org/10.1190/tle37010058.1	journal	January 2018
Data fusion in X-ray computed tomography using a superiorization approach Schrapp, Michael J.; Herman, Gabor T. Review of Scientific Instruments, Vol. 85, Issue 5 https://doi.org/10.1063/1.4872378	journal	May 2014
TomoGAN: low-dose synchrotron x-ray tomography with generative adversarial networks: discussion Liu, Zhengchun; Bicer, Tekin; Kettimuthu, Rajkumar Journal of the Optical Society of America A, Vol. 37, Issue 3 https://doi.org/10.1364/JOSAA.375595	journal	January 2020
Scatter Corrections in X-Ray Computed Tomography: A Physics-Based Analysis Levine, Zachary H.; Blattner, Timothy J.; Peskin, Adele P. Journal of Research of the National Institute of Standards and Technology, Vol. 124 https://doi.org/10.6028/jres.124.013	journal	January 2019
An X-ray tomography facility for I.C. industry at STMicroelectronics Grenoble Bord, S.; Clement, A.; Lecomte, J. C. Microelectronic Engineering, Vol. 61-62 https://doi.org/10.1016/S0167-9317(02)00523-3	journal	July 2002
Prior image constrained compressed sensing (PICCS): A method to accurately reconstruct dynamic CT images from highly undersampled projection data sets: Prior image constrained compressed sensing (PICCS) Chen, Guang-Hong; Tang, Jie; Leng, Shuai Medical Physics, Vol. 35, Issue 2 https://doi.org/10.1118/1.2836423	journal	January 2008
Real-time automated counterfeit integrated circuit detection using x-ray microscopy Mahmood, Kaleel; Carmona, Pedro Latorre; Shahbazmohamadi, Sina Applied Optics, Vol. 54, Issue 13 https://doi.org/10.1364/AO.54.000D25	journal	January 2015
Image reconstruction by domain-transform manifold learning Zhu, Bo; Liu, Jeremiah Z.; Cauley, Stephen F. Nature, Vol. 555, Issue 7697 https://doi.org/10.1038/nature25988	journal	March 2018
Group Invariant Scattering Mallat, Stéphane Communications on Pure and Applied Mathematics, Vol. 65, Issue 10 https://doi.org/10.1002/cpa.21413	journal	July 2012
GAN-Based Priors for Quantifying Uncertainty in Supervised Learning Patel, Dhruv V.; Oberai, Assad A. SIAM/ASA Journal on Uncertainty Quantification, Vol. 9, Issue 3 https://doi.org/10.1137/20M1354210	journal	January 2021
DeepCGH: 3D computer-generated holography using deep learning Hossein Eybposh, M.; Caira, Nicholas W.; Atisa, Mathew Optics Express, Vol. 28, Issue 18 https://doi.org/10.1364/OE.399624	journal	January 2020
Prior-based artifact correction (PBAC) in computed tomography: Prior-based artifact correction (PBAC) in computed tomography Heußer, Thorsten; Brehm, Marcus; Ritschl, Ludwig Medical Physics, Vol. 41, Issue 2 https://doi.org/10.1118/1.4851536	journal	January 2014
X-ray computed tomography Withers, Philip J.; Bouman, Charles; Carmignato, Simone Nature Reviews Methods Primers, Vol. 1, Issue 1 https://doi.org/10.1038/s43586-021-00015-4	journal	February 2021
Deep Scattering Spectrum Anden, Joakim; Mallat, Stephane IEEE Transactions on Signal Processing, Vol. 62, Issue 16 https://doi.org/10.1109/TSP.2014.2326991	journal	August 2014
Limited angle tomography for transmission X-ray microscopy using deep learning Huang, Yixing; Wang, Shengxiang; Guan, Yong Journal of Synchrotron Radiation, Vol. 27, Issue 2 https://doi.org/10.1107/S160057752000017X	journal	February 2020
Phase–contrast X–ray computed tomography for observing biological soft tissues Momose, Atsushi; Takeda, Tohoru; Itai, Yuji Nature Medicine, Vol. 2, Issue 4 https://doi.org/10.1038/nm0496-473	journal	April 1996
Deep Convolutional Neural Network for Inverse Problems in Imaging Jin, Kyong Hwan; McCann, Michael T.; Froustey, Emmanuel IEEE Transactions on Image Processing, Vol. 26, Issue 9 https://doi.org/10.1109/TIP.2017.2713099	journal	September 2017
Recent Advances of Generative Adversarial Networks in Computer Vision Cao, Yang-Jie; Jia, Li-Li; Chen, Yong-Xia IEEE Access, Vol. 7 https://doi.org/10.1109/ACCESS.2018.2886814	journal	January 2019
Design of a 3000-Pixel Transition-Edge Sensor X-Ray Spectrometer for Microcircuit Tomography Szypryt, Paul; Bennett, Douglas A.; Boone, William J. IEEE Transactions on Applied Superconductivity, Vol. 31, Issue 5 https://doi.org/10.1109/TASC.2021.3052723	journal	August 2021
A generalized Gaussian image model for edge-preserving MAP estimation Bouman, C.; Sauer, K. IEEE Transactions on Image Processing, Vol. 2, Issue 3 https://doi.org/10.1109/83.236536	journal	July 1993
The Ill-Conditioned Nature of the Limited Angle Tomography Problem Davison, Mark E. SIAM Journal on Applied Mathematics, Vol. 43, Issue 2 https://doi.org/10.1137/0143028	journal	April 1983
Radon Inversion via Deep Learning He, Ji; Wang, Yongbo; Ma, Jianhua IEEE Transactions on Medical Imaging, Vol. 39, Issue 6 https://doi.org/10.1109/TMI.2020.2964266	journal	June 2020
Employing temporal self-similarity across the entire time domain in computed tomography reconstruction Kazantsev, D.; Van Eyndhoven, G.; Lionheart, W. R. B. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 373, Issue 2043 https://doi.org/10.1098/rsta.2014.0389	journal	June 2015
Physics-informed machine learning Karniadakis, George Em; Kevrekidis, Ioannis G.; Lu, Lu Nature Reviews Physics, Vol. 3, Issue 6 https://doi.org/10.1038/s42254-021-00314-5	journal	May 2021
Adaptive dynamic range shift (ADRIFT) quantitative phase imaging Toda, Keiichiro; Tamamitsu, Miu; Ideguchi, Takuro Light: Science & Applications, Vol. 10, Issue 1 https://doi.org/10.1038/s41377-020-00435-z	journal	January 2021
Deep null space learning for inverse problems: convergence analysis and rates Schwab, Johannes; Antholzer, Stephan; Haltmeier, Markus Inverse Problems, Vol. 35, Issue 2 https://doi.org/10.1088/1361-6420/aaf14a	journal	January 2019
Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations Yang, Liu; Zhang, Dongkun; Karniadakis, George Em SIAM Journal on Scientific Computing, Vol. 42, Issue 1 https://doi.org/10.1137/18M1225409	journal	January 2020
Low-Dose CT Image Denoising Using a Generative Adversarial Network With Wasserstein Distance and Perceptual Loss Yang, Qingsong; Yan, Pingkun; Zhang, Yanbo IEEE Transactions on Medical Imaging, Vol. 37, Issue 6 https://doi.org/10.1109/TMI.2018.2827462	journal	June 2018
Generative adversarial network in medical imaging: A review Yi, Xin; Walia, Ekta; Babyn, Paul Medical Image Analysis, Vol. 58 https://doi.org/10.1016/j.media.2019.101552	journal	December 2019
X-ray computed tomography using partially coherent Fresnel diffraction with application to an optical fiber Levine, Zachary H.; Garboczi, Edward J.; Peskin, Adele P. Optics Express, Vol. 29, Issue 2 https://doi.org/10.1364/OE.414398	journal	January 2021
Randomized probe imaging through deep k-learning Guo, Zhen; Levitan, Abraham; Barbastathis, George Optics Express, Vol. 30, Issue 2 https://doi.org/10.1364/OE.445498	journal	January 2022
Impact of X-Ray Tomography on the Reliability of Integrated Circuits Alam, Mahbub; Shen, Haoting; Asadizanjani, Navid IEEE Transactions on Device and Materials Reliability, Vol. 17, Issue 1 https://doi.org/10.1109/TDMR.2017.2656839	journal	March 2017
Low Photon Count Phase Retrieval Using Deep Learning Goy, Alexandre; Arthur, Kwabena; Li, Shuai Physical Review Letters, Vol. 121, Issue 24 https://doi.org/10.1103/PhysRevLett.121.243902	journal	December 2018
Deep learning based image reconstruction algorithm for limited-angle translational computed tomography Wang, Jiaxi; Liang, Jun; Cheng, Jingye PLOS ONE, Vol. 15, Issue 1 https://doi.org/10.1371/journal.pone.0226963	journal	January 2020
Learning the invisible: a hybrid deep learning-shearlet framework for limited angle computed tomography Bubba, Tatiana A.; Kutyniok, Gitta; Lassas, Matti Inverse Problems, Vol. 35, Issue 6 https://doi.org/10.1088/1361-6420/ab10ca	journal	June 2019
Tikhonov Regularization and Total Least Squares Golub, Gene H.; Hansen, Per Christian; O'Leary, Dianne P. SIAM Journal on Matrix Analysis and Applications, Vol. 21, Issue 1 https://doi.org/10.1137/S0895479897326432	journal	January 1999
Low-Dose CT With a Residual Encoder-Decoder Convolutional Neural Network Chen, Hu; Zhang, Yi; Kalra, Mannudeep K. IEEE Transactions on Medical Imaging, Vol. 36, Issue 12 https://doi.org/10.1109/TMI.2017.2715284	journal	December 2017
X-Rays Tomographic Reconstruction Images using Proximal Methods based on L 1 Norm and TV Regularization Allag, Aicha; Drai, Redouane; Benammar, Abdessalem Procedia Computer Science, Vol. 127 https://doi.org/10.1016/j.procs.2018.01.119	journal	January 2018
On instabilities of deep learning in image reconstruction and the potential costs of AI Antun, Vegard; Renna, Francesco; Poon, Clarice Proceedings of the National Academy of Sciences, Vol. 117, Issue 48 https://doi.org/10.1073/pnas.1907377117	journal	May 2020
On the interplay between physical and content priors in deep learning for computational imaging Deng, Mo; Li, Shuai; Zhang, Zhengyun Optics Express, Vol. 28, Issue 16 https://doi.org/10.1364/OE.395204	journal	January 2020
Limited-data computed tomography algorithms for the physical sciences Verhoeven, Dean Applied Optics, Vol. 32, Issue 20 https://doi.org/10.1364/AO.32.003736	journal	January 1993
3D imaging in material science: Application of X-ray tomography Salvo, Luc; Suéry, Michel; Marmottant, Ariane Comptes Rendus Physique, Vol. 11, Issue 9-10 https://doi.org/10.1016/j.crhy.2010.12.003	journal	November 2010
A local update strategy for iterative reconstruction from projections Sauer, K.; Bouman, C. IEEE Transactions on Signal Processing, Vol. 41, Issue 2 https://doi.org/10.1109/78.193196	journal	January 1993

Similar Records

Attentional Ptycho-Tomography (APT) for three-dimensional nanoscale X-ray imaging with minimal data acquisition and computation time

Journal Article · Tue May 30 00:00:00 EDT 2023 · Light, Science & Applications · OSTI ID:1871842

Kang, Iksung; Wu, Ziling; Jiang, Yi; +5 more

Improving field boundary delineation in ResUNets via adversarial deep learning

Journal Article · Fri Jul 01 00:00:00 EDT 2022 · International Journal of Applied Earth Observation and Geoinformation · OSTI ID:1871842

Jong, Maxwell; Guan, Kaiyu; Wang, Sibo; +2 more

Novel usage of deep learning and high-performance computing in long-baseline neutrino oscillation experiments

Conference · Fri Jan 01 00:00:00 EST 2021 · EPJ Web Conf. · OSTI ID:1871842

Alonso Monsalve, Saúl; Rodriguez, Manuel J.

Related Subjects

47 OTHER INSTRUMENTATION

Title: Physics-assisted generative adversarial network for X-ray tomography

Citation Formats

References (49)

Similar Records

Related Subjects