CoIL: Coordinate-based Internal Learning for Imaging Inverse Problems

Sun, Yu; Liu, Jiaming; Xie, Mingyang; Wohlberg, Brendt Egon; Kamilov, Ulugbek S.

doi:10.1109/TCI.2021.3125564

CoIL: Coordinate-based Internal Learning for Imaging Inverse Problems

Journal Article · 08 November 2021 · IEEE Transactions on Computational Imaging

DOI:https://doi.org/10.1109/TCI.2021.3125564· OSTI ID:1883143

Sun, Yu ^[1]; Liu, Jiaming ^[1]; Xie, Mingyang ^[1]; ^[2]; Kamilov, Ulugbek S. ^[1]

Washington University in St. Louis
Los Alamos National Laboratory

Research Organization:: Los Alamos National Lab. (LANL), Los Alamos, NM (United States)

Sponsoring Organization:: USDOE Laboratory Directed Research and Development (LDRD) Program

DOE Contract Number:: 89233218CNA000001

OSTI ID:: 1883143

Report Number(s):: LA-UR-21-21526; 2334-0118 (Electronic)

Journal Information:: IEEE Transactions on Computational Imaging, Vol. 7

Country of Publication:: United States

Language:: English

References (73)

Self‐supervised learning of physics‐guided reconstruction neural networks without fully sampled reference data Yaman, Burhaneddin; Hosseini, Seyed Amir Hossein; Moeller, Steen Magnetic Resonance in Medicine, Vol. 84, Issue 6 https://doi.org/10.1002/mrm.28378	journal	July 2020
Momentum-Net: Fast and convergent iterative neural network for inverse problems Chun, Il Yong; Huang, Zhengyu; Lim, Hongki IEEE Transactions on Pattern Analysis and Machine Intelligence https://doi.org/10.1109/TPAMI.2020.3012955	journal	January 2020
Dense Recurrent Neural Networks for Accelerated MRI: History-Cognizant Unrolling of Optimization Algorithms Hosseini, Seyed Amir Hossein; Yaman, Burhaneddin; Moeller, Steen IEEE Journal of Selected Topics in Signal Processing, Vol. 14, Issue 6 https://doi.org/10.1109/JSTSP.2020.3003170	journal	October 2020
MoDL: Model-Based Deep Learning Architecture for Inverse Problems Aggarwal, Hemant K.; Mani, Merry P.; Jacob, Mathews IEEE Transactions on Medical Imaging, Vol. 38, Issue 2 https://doi.org/10.1109/TMI.2018.2865356	journal	February 2019
NeRF Mildenhall, Ben; Srinivasan, Pratul P.; Tancik, Matthew Communications of the ACM, Vol. 65, Issue 1 https://doi.org/10.1145/3503250	journal	January 2022
SGD-Net: Efficient Model-Based Deep Learning With Theoretical Guarantees Liu, Jiaming; Sun, Yu; Gan, Weijie IEEE Transactions on Computational Imaging, Vol. 7 https://doi.org/10.1109/TCI.2021.3085534	journal	January 2021
Memory-Efficient Learning for Large-Scale Computational Imaging Kellman, Michael; Zhang, Kevin; Markley, Eric IEEE Transactions on Computational Imaging, Vol. 6 https://doi.org/10.1109/TCI.2020.3025735	journal	January 2020
J-MoDL: Joint Model-Based Deep Learning for Optimized Sampling and Reconstruction Aggarwal, Hemant Kumar; Jacob, Mathews IEEE Journal of Selected Topics in Signal Processing, Vol. 14, Issue 6 https://doi.org/10.1109/JSTSP.2020.3004094	journal	October 2020
Model-Based Learning for Accelerated, Limited-View 3-D Photoacoustic Tomography Hauptmann, Andreas; Lucka, Felix; Betcke, Marta IEEE Transactions on Medical Imaging, Vol. 37, Issue 6 https://doi.org/10.1109/TMI.2018.2820382	journal	June 2018
Learned Primal-Dual Reconstruction Adler, Jonas; Oktem, Ozan IEEE Transactions on Medical Imaging, Vol. 37, Issue 6 https://doi.org/10.1109/TMI.2018.2799231	journal	June 2018
Plug-and-Play ADMM for Image Restoration: Fixed-Point Convergence and Applications Chan, Stanley H.; Wang, Xiran; Elgendy, Omar A. IEEE Transactions on Computational Imaging, Vol. 3, Issue 1 https://doi.org/10.1109/TCI.2016.2629286	journal	March 2017
Plug-and-Play Unplugged: Optimization-Free Reconstruction Using Consensus Equilibrium Buzzard, Gregery T.; Chan, Stanley H.; Sreehari, Suhas SIAM Journal on Imaging Sciences, Vol. 11, Issue 3 https://doi.org/10.1137/17M1122451	journal	January 2018
Provable Convergence of Plug-and-Play Priors With MMSE Denoisers Xu, Xiaojian; Sun, Yu; Liu, Jiaming IEEE Signal Processing Letters, Vol. 27 https://doi.org/10.1109/LSP.2020.3006390	journal	January 2020
An Online Plug-and-Play Algorithm for Regularized Image Reconstruction Sun, Yu; Wohlberg, Brendt; Kamilov, Ulugbek S. IEEE Transactions on Computational Imaging, Vol. 5, Issue 3 https://doi.org/10.1109/TCI.2019.2893568	journal	September 2019
Solving ill-posed inverse problems using iterative deep neural networks Adler, Jonas; Öktem, Ozan Inverse Problems, Vol. 33, Issue 12 https://doi.org/10.1088/1361-6420/aa9581	journal	November 2017
ISTA-Net: Interpretable Optimization-Inspired Deep Network for Image Compressive Sensing Zhang, Jian; Ghanem, Bernard 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2018.00196	conference	June 2018
Block Coordinate Regularization by Denoising Sun, Yu; Liu, Jiaming; Kamilov, Ulugbek S. IEEE Transactions on Computational Imaging, Vol. 6 https://doi.org/10.1109/TCI.2020.2996385	journal	January 2020
FFDNet: Toward a Fast and Flexible Solution for CNN-Based Image Denoising Zhang, Kai; Zuo, Wangmeng; Zhang, Lei IEEE Transactions on Image Processing, Vol. 27, Issue 9 https://doi.org/10.1109/TIP.2018.2839891	journal	September 2018
A New Recurrent Plug-and-Play Prior Based on the Multiple Self-Similarity Network Song, Guangxiao; Sun, Yu; Liu, Jiaming IEEE Signal Processing Letters, Vol. 27 https://doi.org/10.1109/LSP.2020.2977214	journal	January 2020
A Convergent Image Fusion Algorithm Using Scene-Adapted Gaussian-Mixture-Based Denoising Teodoro, Afonso M.; Bioucas-Dias, Jose M.; Figueiredo, Mario A. T. IEEE Transactions on Image Processing, Vol. 28, Issue 1 https://doi.org/10.1109/TIP.2018.2869727	journal	January 2019
Image Restoration by Iterative Denoising and Backward Projections Tirer, Tom; Giryes, Raja IEEE Transactions on Image Processing, Vol. 28, Issue 3 https://doi.org/10.1109/TIP.2018.2875569	journal	March 2019
Learning Proximal Operators: Using Denoising Networks for Regularizing Inverse Imaging Problems Meinhardt, Tim; Moeller, Michael; Hazirbas, Caner 2017 IEEE International Conference on Computer Vision (ICCV) https://doi.org/10.1109/ICCV.2017.198	conference	October 2017
Plug-and-Play Methods for Magnetic Resonance Imaging: Using Denoisers for Image Recovery Ahmad, Rizwan; Bouman, Charles A.; Buzzard, Gregery T. IEEE Signal Processing Magazine, Vol. 37, Issue 1 https://doi.org/10.1109/MSP.2019.2949470	journal	January 2020
Deep Plug-And-Play Super-Resolution for Arbitrary Blur Kernels Zhang, Kai; Zuo, Wangmeng; Zhang, Lei 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2019.00177	conference	June 2019
Regularized Fourier Ptychography Using an Online Plug-and-play Algorithm Sun, Yu; Xu, Shiqi; Li, Yunzhe ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) https://doi.org/10.1109/ICASSP.2019.8683057	conference	May 2019
Learning Deep CNN Denoiser Prior for Image Restoration Zhang, Kai; Zuo, Wangmeng; Gu, Shuhang 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2017.300	conference	July 2017
Plug-and-Play Priors for Bright Field Electron Tomography and Sparse Interpolation Sreehari, Suhas; Venkatakrishnan, Singanallur V.; Wohlberg, Brendt IEEE Transactions on Computational Imaging https://doi.org/10.1109/TCI.2016.2599778	journal	January 2016
BM3D Frames and Variational Image Deblurring Danielyan, A.; Katkovnik, V.; Egiazarian, K. IEEE Transactions on Image Processing, Vol. 21, Issue 4 https://doi.org/10.1109/TIP.2011.2176954	journal	April 2012
Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries Elad, Michael; Aharon, Michal IEEE Transactions on Image Processing, Vol. 15, Issue 12 https://doi.org/10.1109/TIP.2006.881969	journal	January 2006
Using Deep Neural Networks for Inverse Problems in Imaging: Beyond Analytical Methods Lucas, Alice; Iliadis, Michael; Molina, Rafael IEEE Signal Processing Magazine, Vol. 35, Issue 1 https://doi.org/10.1109/MSP.2017.2760358	journal	January 2018
Convolutional Neural Networks for Inverse Problems in Imaging: A Review McCann, Michael T.; Jin, Kyong Hwan; Unser, Michael IEEE Signal Processing Magazine, Vol. 34, Issue 6 https://doi.org/10.1109/MSP.2017.2739299	journal	November 2017
Deep learning for tomographic image reconstruction Wang, Ge; Ye, Jong Chul; De Man, Bruno Nature Machine Intelligence, Vol. 2, Issue 12 https://doi.org/10.1038/s42256-020-00273-z	journal	December 2020
Deep Learning Techniques for Inverse Problems in Imaging Ongie, Gregory; Jalal, Ajil; Metzler, Christopher A. IEEE Journal on Selected Areas in Information Theory, Vol. 1, Issue 1 https://doi.org/10.1109/JSAIT.2020.2991563	journal	May 2020
Deep Convolutional Neural Network for Inverse Problems in Imaging Jin, Kyong Hwan; McCann, Michael T.; Froustey, Emmanuel IEEE Transactions on Image Processing, Vol. 26, Issue 9 https://doi.org/10.1109/TIP.2017.2713099	journal	September 2017
Proximité et dualité dans un espace hilbertien Moreau, J. J. Bulletin de la Société mathématique de France, Vol. 79 https://doi.org/10.24033/bsmf.1625	journal	January 1951
Image reconstruction by domain-transform manifold learning Zhu, Bo; Liu, Jeremiah Z.; Cauley, Stephen F. Nature, Vol. 555, Issue 7697 https://doi.org/10.1038/nature25988	journal	March 2018
Deep Learning Computed Tomography: Learning Projection-Domain Weights From Image Domain in Limited Angle Problems Wurfl, Tobias; Hoffmann, Mathis; Christlein, Vincent IEEE Transactions on Medical Imaging, Vol. 37, Issue 6 https://doi.org/10.1109/TMI.2018.2833499	journal	June 2018
A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems Beck, Amir; Teboulle, Marc SIAM Journal on Imaging Sciences, Vol. 2, Issue 1 https://doi.org/10.1137/080716542	journal	January 2009
Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers Boyd, Stephen; Parikh, Neal; Chu, Eric https://doi.org/10.1561/2200000016	book	January 2011
A dual algorithm for the solution of nonlinear variational problems via finite element approximation Gabay, Daniel; Mercier, Bertrand Computers & Mathematics with Applications, Vol. 2, Issue 1 https://doi.org/10.1016/0898-1221(76)90003-1	journal	January 1976
Deep-Neural-Network-Based Sinogram Synthesis for Sparse-View CT Image Reconstruction Lee, Hoyeon; Lee, Jongha; Kim, Hyeongseok IEEE Transactions on Radiation and Plasma Medical Sciences, Vol. 3, Issue 2 https://doi.org/10.1109/TRPMS.2018.2867611	journal	March 2019
Fast Enhanced CT Metal Artifact Reduction Using Data Domain Deep Learning Ghani, Muhammad Usman; Karl, W. Clem IEEE Transactions on Computational Imaging, Vol. 6 https://doi.org/10.1109/TCI.2019.2937221	journal	January 2020
Data and Image Prior Integration for Image Reconstruction Using Consensus Equilibrium Ghani, Muhammad Usman; Karl, W. Clem IEEE Transactions on Computational Imaging, Vol. 7 https://doi.org/10.1109/TCI.2021.3062986	journal	January 2021
${k}$ -Space Deep Learning for Accelerated MRI Han, Yoseo; Sunwoo, Leonard; Ye, Jong Chul IEEE Transactions on Medical Imaging, Vol. 39, Issue 2 https://doi.org/10.1109/TMI.2019.2927101	journal	February 2020
Regularization by Denoising: Clarifications and New Interpretations Reehorst, Edward T.; Schniter, Philip IEEE Transactions on Computational Imaging, Vol. 5, Issue 1 https://doi.org/10.1109/TCI.2018.2880326	journal	March 2019
Regularization by Denoising via Fixed-Point Projection (RED-PRO) Cohen, Regev; Elad, Michael; Milanfar, Peyman SIAM Journal on Imaging Sciences, Vol. 14, Issue 3 https://doi.org/10.1137/20M1337168	journal	January 2021
Dynamic MRI using model‐based deep learning and SToRM priors: MoDL‐SToRM Biswas, Sampurna; Aggarwal, Hemant K.; Jacob, Mathews Magnetic Resonance in Medicine, Vol. 82, Issue 1 https://doi.org/10.1002/mrm.27706	journal	March 2019
Solving Linear Inverse Problems Using Gan Priors: An Algorithm with Provable Guarantees Shah, Viraj; Hegde, Chinmay ICASSP 2018 - 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) https://doi.org/10.1109/ICASSP.2018.8462233	conference	April 2018
Wavelet-based image estimation: an empirical Bayes approach using Jeffrey's noninformative prior Figueiredo, M. A. T.; Nowak, R. D. IEEE Transactions on Image Processing, Vol. 10, Issue 9 https://doi.org/10.1109/83.941856	journal	January 2001
Nonlinear total variation based noise removal algorithms Rudin, Leonid I.; Osher, Stanley; Fatemi, Emad Physica D: Nonlinear Phenomena, Vol. 60, Issue 1-4 https://doi.org/10.1016/0167-2789(92)90242-F	journal	November 1992
Multilayer feedforward networks are universal approximators Hornik, Kurt; Stinchcombe, Maxwell; White, Halbert Neural Networks, Vol. 2, Issue 5 https://doi.org/10.1016/0893-6080(89)90020-8	journal	January 1989
Versatile reconstruction framework for diffraction tomography with intensity measurements and multiple scattering Pham, Thanh-An; Soubies, Emmanuel; Goy, Alexandre Optics Express, Vol. 26, Issue 3 https://doi.org/10.1364/OE.26.002749	journal	January 2018
Learning approach to optical tomography Kamilov, Ulugbek S.; Papadopoulos, Ioannis N.; Shoreh, Morteza H. Optica, Vol. 2, Issue 6 https://doi.org/10.1364/OPTICA.2.000517	journal	January 2015
Optical diffraction tomography for high resolution live cell imaging Sung, Yongjin; Choi, Wonshik; Fang-Yen, Christopher Optics Express, Vol. 17, Issue 1 https://doi.org/10.1364/OE.17.000266	journal	January 2009
NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis Srinivasan, Pratul P.; Deng, Boyang; Zhang, Xiuming 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR46437.2021.00741	conference	June 2021
Scalable Plug-and-Play ADMM With Convergence Guarantees Sun, Yu; Wu, Zihui; Xu, Xiaojian IEEE Transactions on Computational Imaging, Vol. 7 https://doi.org/10.1109/TCI.2021.3094062	journal	January 2021
TU-FG-207A-04: Overview of the Low Dose CT Grand Challenge McCollough, C. Medical Physics, Vol. 43, Issue 6Part35 https://doi.org/10.1118/1.4957556	journal	June 2016
A deep convolutional neural network using directional wavelets for low-dose X-ray CT reconstruction Kang, Eunhee; Min, Junhong; Ye, Jong Chul Medical Physics, Vol. 44, Issue 10 https://doi.org/10.1002/mp.12344	journal	October 2017
Efficient and accurate inversion of multiple scattering with deep learning Sun, Yu; Xia, Zhihao; Kamilov, Ulugbek S. Optics Express, Vol. 26, Issue 11 https://doi.org/10.1364/OE.26.014678	journal	January 2018
Framing U-Net via Deep Convolutional Framelets: Application to Sparse-View CT Han, Yoseob; Ye, Jong Chul IEEE Transactions on Medical Imaging, Vol. 37, Issue 6 https://doi.org/10.1109/TMI.2018.2823768	journal	June 2018
Plug-and-Play priors for model based reconstruction Venkatakrishnan, Singanallur V.; Bouman, Charles A.; Wohlberg, Brendt 2013 IEEE Global Conference on Signal and Information Processing (GlobalSIP) https://doi.org/10.1109/GlobalSIP.2013.6737048	conference	December 2013
The Little Engine That Could: Regularization by Denoising (RED) Romano, Yaniv; Elad, Michael; Milanfar, Peyman SIAM Journal on Imaging Sciences, Vol. 10, Issue 4 https://doi.org/10.1137/16M1102884	journal	January 2017
Primal-Dual Plug-and-Play Image Restoration Ono, Shunsuke IEEE Signal Processing Letters, Vol. 24, Issue 8 https://doi.org/10.1109/LSP.2017.2710233	journal	August 2017
A Plug-and-Play Priors Approach for Solving Nonlinear Imaging Inverse Problems Kamilov, Ulugbek S.; Mansour, Hassan; Wohlberg, Brendt IEEE Signal Processing Letters, Vol. 24, Issue 12 https://doi.org/10.1109/LSP.2017.2763583	journal	December 2017
Zero-Shot Super-Resolution Using Deep Internal Learning Shocher, Assaf; Cohen, Nadav; Irani, Michal 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2018.00329	conference	June 2018
SIMBA: Scalable Inversion in Optical Tomography Using Deep Denoising Priors Wu, Zihui; Sun, Yu; Matlock, Alex IEEE Journal of Selected Topics in Signal Processing, Vol. 14, Issue 6 https://doi.org/10.1109/JSTSP.2020.2999820	journal	October 2020
Image Restoration Using Total Variation Regularized Deep Image Prior Liu, Jiaming; Sun, Yu; Xu, Xiaojian ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) https://doi.org/10.1109/ICASSP.2019.8682856	conference	May 2019
RARE: Image Reconstruction Using Deep Priors Learned Without Groundtruth Liu, Jiaming; Sun, Yu; Eldeniz, Cihat IEEE Journal of Selected Topics in Signal Processing, Vol. 14, Issue 6 https://doi.org/10.1109/JSTSP.2020.2998402	journal	October 2020
Deep Image Prior Ulyanov, Dmitry; Vedaldi, Andrea; Lempitsky, Victor International Journal of Computer Vision, Vol. 128, Issue 7 https://doi.org/10.1007/s11263-020-01303-4	journal	March 2020
Scan‐specific robust artificial‐neural‐networks for k‐space interpolation (RAKI) reconstruction: Database‐free deep learning for fast imaging Akçakaya, Mehmet; Moeller, Steen; Weingärtner, Sebastian Magnetic Resonance in Medicine, Vol. 81, Issue 1 https://doi.org/10.1002/mrm.27420	journal	September 2018
“Double-DIP”: Unsupervised Image Decomposition via Coupled Deep-Image-Priors Gandelsman, Yosef; Shocher, Assaf; Irani, Michal 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2019.01128	conference	June 2019
Learning Implicit Fields for Generative Shape Modeling Chen, Zhiqin; Zhang, Hao 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2019.00609	conference	June 2019
DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation Park, Jeong Joon; Florence, Peter; Straub, Julian 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2019.00025	conference	June 2019

Similar Records

Deep Equilibrium Learning of Explicit Regularization Functionals for Imaging Inverse Problems

Journal Article · 2023 · IEEE Open Journal of Signal Processing (Online) · OSTI ID:1994422

Zou, Zihao; Liu, Jiaming; Wohlberg, Brendt Egon; +1 more

Coordinate-Based Seismic Interpolation in Irregular Land Survey: A Deep Internal Learning Approach

Journal Article · 2023 · IEEE Transactions on Geoscience and Remote Sensing · OSTI ID:2217488

Goyes-Peñafiel, Paul; Vargas, Edwin; Correa, Claudia V.; +4 more

Related Subjects

Computer Science
Mathematics

CoIL: Coordinate-based Internal Learning for Imaging Inverse Problems

Citation Formats

References (73)

Similar Records

Related Subjects