Non-Blind Deblurring for Fluorescence: A Deformable Latent Space Approach with Kernel Parameterization

Guan, Ziqiao; Tsai, Esther R.; Huang, Xiaojing; Yager, Kevin G.; Qin, Hong

doi:10.1109/wacv51458.2022.00018

Non-Blind Deblurring for Fluorescence: A Deformable Latent Space Approach with Kernel Parameterization

Journal Article · Mon Feb 14 23:00:00 EST 2022 · Proceedings - IEEE Workshop on Applications of Computer Vision

DOI:https://doi.org/10.1109/wacv51458.2022.00018· OSTI ID:1887817

Guan, Ziqiao ^[1]; ^[2]; Huang, Xiaojing ^[2]; Yager, Kevin G. ^[2]; Qin, Hong ^[1]

Stony Brook University, NY (United States)
Brookhaven National Lab. (BNL), Upton, NY (United States)

We report N\non-blind deblurring (NBD) is a modeling method of the image deblurring problem in computer vision, where the blurring kernel is known or can be externally estimated. In this paper, we attempt to solve a parametric NBD problem, inspired by the simultaneous acquisition of ptychography and fluorescent imaging (FI). Ptychography is an imaging method that favors larger probes, i.e. convolutional kernels, while FI relies on a small probe for high resolution. Also, the kernel can be solved during ptychographic reconstruction. With Ptycho-FI using the same larger kernel, we can perform NBD on the blurred fluorescent images to achieve high-resolution FI, and thus speed up the experiments. To this end, we design a deep latent space deformation network that is directly parameterized by the kernel. The network consists of three components: encoder, deformer, and decoder, where the deformer is specifically meant to rectify the latent space representations of blurred images to a standard latent space, regardless of the kernel. The deformation network is trained with a two-stage training scheme. We conduct extensive experiments to confirm that our parametric model can adapt to drastically different blurring kernels and perform robust deblurring.

View Accepted Manuscript (DOE)

Research Organization:: Brookhaven National Laboratory (BNL), Upton, NY (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Basic Energy Sciences (BES); National Science Foundation

Grant/Contract Number:: SC0012704

OSTI ID:: 1887817

Report Number(s):: BNL-223348-2022-JAAM

Journal Information:: Proceedings - IEEE Workshop on Applications of Computer Vision, Journal Name: Proceedings - IEEE Workshop on Applications of Computer Vision Vol. 2022; ISSN 1550-5790

Publisher:: IEEECopyright Statement

Country of Publication:: United States

Language:: English

References (25)

ImageNet Large Scale Visual Recognition Challenge Russakovsky, Olga; Deng, Jia; Su, Hao International Journal of Computer Vision, Vol. 115, Issue 3 https://doi.org/10.1007/s11263-015-0816-y	journal	April 2015
Nonlinear total variation based noise removal algorithms Rudin, Leonid I.; Osher, Stanley; Fatemi, Emad Physica D: Nonlinear Phenomena, Vol. 60, Issue 1-4 https://doi.org/10.1016/0167-2789(92)90242-F	journal	November 1992
Deconvolution processing of HAADF STEM images Watanabe, K.; Kotaka, Y.; Nakanishi, N. Ultramicroscopy, Vol. 92, Issue 3-4 https://doi.org/10.1016/S0304-3991(02)00132-8	journal	August 2002
A three-dimensional view of structural changes caused by deactivation of fluid catalytic cracking catalysts Ihli, J.; Jacob, R. R.; Holler, M. Nature Communications, Vol. 8, Issue 1 https://doi.org/10.1038/s41467-017-00789-w	journal	October 2017
Movable Aperture Lensless Transmission Microscopy: A Novel Phase Retrieval Algorithm Faulkner, H. M. L.; Rodenburg, J. M. Physical Review Letters, Vol. 93, Issue 2 https://doi.org/10.1103/PhysRevLett.93.023903	journal	July 2004
Hard-X-Ray Lensless Imaging of Extended Objects Rodenburg, J. M.; Hurst, A. C.; Cullis, A. G. Physical Review Letters, Vol. 98, Issue 3 https://doi.org/10.1103/PhysRevLett.98.034801	journal	January 2007
Scale-Iterative Upscaling Network for Image Deblurring Ye, Minyuan; Lyu, Dong; Chen, Gengsheng IEEE Access, Vol. 8 https://doi.org/10.1109/ACCESS.2020.2967823	journal	January 2020
A Machine Learning Approach for Non-blind Image Deconvolution Schuler, Christian J.; Burger, Harold Christopher; Harmeling, Stefan 2013 IEEE Conference on Computer Vision and Pattern Recognition https://doi.org/10.1109/CVPR.2013.142	conference	June 2013
Discriminative Non-blind Deblurring Schmidt, Uwe; Rother, Carsten; Nowozin, Sebastian 2013 IEEE Conference on Computer Vision and Pattern Recognition https://doi.org/10.1109/CVPR.2013.84	conference	June 2013
Shrinkage Fields for Effective Image Restoration Schmidt, Uwe; Roth, Stefan 2014 IEEE Conference on Computer Vision and Pattern Recognition https://doi.org/10.1109/CVPR.2014.349	conference	June 2014
Context Encoders: Feature Learning by Inpainting Pathak, Deepak; Krahenbuhl, Philipp; Donahue, Jeff 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2016.278	conference	June 2016
Learning Deep CNN Denoiser Prior for Image Restoration Zhang, Kai; Zuo, Wangmeng; Gu, Shuhang 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2017.300	conference	July 2017
Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring Nah, Seungjun; Kim, Tae Hyun; Lee, Kyoung Mu 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2017.35	conference	July 2017
Learning Fully Convolutional Networks for Iterative Non-blind Deconvolution Zhang, Jiawei; Pan, Jinshan; Lai, Wei-Sheng 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2017.737	conference	July 2017
Image Deblurring via Extreme Channels Prior Yan, Yanyang; Ren, Wenqi; Guo, Yuanfang 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2017.738	conference	July 2017
A Poisson-Gaussian Denoising Dataset With Real Fluorescence Microscopy Images Zhang, Yide; Zhu, Yinhao; Nichols, Evan 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2019.01198	conference	June 2019
From learning models of natural image patches to whole image restoration Zoran, Daniel; Weiss, Yair 2011 IEEE International Conference on Computer Vision (ICCV), 2011 International Conference on Computer Vision https://doi.org/10.1109/ICCV.2011.6126278	conference	November 2011
Handling outliers in non-blind image deconvolution Cho, Sunghyun; Lee, Seungyong 2011 International Conference on Computer Vision https://doi.org/10.1109/ICCV.2011.6126280	conference	November 2011
Learning to Push the Limits of Efficient FFT-Based Image Deconvolution Kruse, Jakob; Rother, Carsten; Schmidt, Uwe 2017 IEEE International Conference on Computer Vision (ICCV) https://doi.org/10.1109/ICCV.2017.491	conference	October 2017
Deconvolution methods for 3-D fluorescence microscopy images Sarder, P.; Nehorai, A. IEEE Signal Processing Magazine, Vol. 23, Issue 3 https://doi.org/10.1109/MSP.2006.1628876	journal	May 2006
Image Super-Resolution Using Deep Convolutional Networks Dong, Chao; Loy, Chen Change; He, Kaiming IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 38, Issue 2 https://doi.org/10.1109/TPAMI.2015.2439281	journal	February 2016
Extracting and composing robust features with denoising autoencoders Vincent, Pascal; Larochelle, Hugo; Bengio, Yoshua Proceedings of the 25th international conference on Machine learning - ICML '08 https://doi.org/10.1145/1390156.1390294	conference	January 2008
X-ray deconvolution microscopy Ehn, Sebastian; Epple, Franz Michael; Fehringer, Andreas Biomedical Optics Express, Vol. 7, Issue 4 https://doi.org/10.1364/BOE.7.001227	journal	March 2016
Bayesian-Based Iterative Method of Image Restoration* Richardson, William Hadley Journal of the Optical Society of America, Vol. 62, Issue 1 https://doi.org/10.1364/JOSA.62.000055	journal	January 1972
Relations Between Two Sets of Variates Hotelling, Harold Biometrika, Vol. 28, Issue 3/4 https://doi.org/10.2307/2333955	journal	December 1936

Similar Records

MO-G-17A-05: PET Image Deblurring Using Adaptive Dictionary Learning

Journal Article · Sun Jun 15 00:00:00 EDT 2014 · Medical Physics · OSTI ID:22409624

Deblurring galaxy images with Tikhonov regularization on magnitude domain

Journal Article · Fri Sep 16 00:00:00 EDT 2022 · Publications of the Astronomical Society of Japan · OSTI ID:1887923

Blind Ptychographic Phase Retrieval via Convergent Alternating Direction Method of Multipliers

Journal Article · Mon Jan 21 23:00:00 EST 2019 · SIAM Journal on Imaging Sciences · OSTI ID:1563992

Related Subjects

77 NANOSCIENCE AND NANOTECHNOLOGY
GANS image processing
autoencoders
deep learning
neural generative models

Non-Blind Deblurring for Fluorescence: A Deformable Latent Space Approach with Kernel Parameterization

Citation Formats

References (25)

Similar Records

Related Subjects