skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Imaging through glass diffusers using densely connected convolutional networks

Abstract

Computational imaging through scatter generally is accomplished by first characterizing the scattering medium so that its forward operator is obtained and then imposing additional priors in the form of regularizers on the reconstruction functional to improve the condition of the originally ill-posed inverse problem. In the functional, the forward operator and regularizer must be entered explicitly or parametrically (e.g., scattering matrices and dictionaries, respectively). However, the process of determining these representations is often incomplete, prone to errors, or infeasible. Recently, deep learning architectures have been proposed to instead learn both the forward operator and regularizer through examples. Here, we propose for the first time, to our knowledge, a convolutional neural network architecture called “IDiffNet” for the problem of imaging through diffuse media and demonstrate that IDiffNet has superior generalization capability through extensive tests with well-calibrated diffusers. We also introduce the negative Pearson correlation coefficient (NPCC) loss function for neural net training and show that the NPCC is more appropriate for spatially sparse objects and strong scattering conditions. Our results show that the convolutional architecture is robust to the choice of prior, as demonstrated by the use of multiple training and testing object databases, and capable of achieving higher space–bandwidth productmore » reconstructions than previously reported.« less

Authors:
; ; ; ;
Publication Date:
Research Org.:
Krell Inst., Inc., Ames, IA (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
1459049
Alternate Identifier(s):
OSTI ID: 1502468
Grant/Contract Number:  
FG02-97ER25308
Resource Type:
Published Article
Journal Name:
Optica
Additional Journal Information:
Journal Name: Optica Journal Volume: 5 Journal Issue: 7; Journal ID: ISSN 2334-2536
Publisher:
Optical Society of America
Country of Publication:
United States
Language:
English
Subject:
47 OTHER INSTRUMENTATION

Citation Formats

Li, Shuai, Deng, Mo, Lee, Justin, Sinha, Ayan, and Barbastathis, George. Imaging through glass diffusers using densely connected convolutional networks. United States: N. p., 2018. Web. doi:10.1364/OPTICA.5.000803.
Li, Shuai, Deng, Mo, Lee, Justin, Sinha, Ayan, & Barbastathis, George. Imaging through glass diffusers using densely connected convolutional networks. United States. doi:10.1364/OPTICA.5.000803.
Li, Shuai, Deng, Mo, Lee, Justin, Sinha, Ayan, and Barbastathis, George. Fri . "Imaging through glass diffusers using densely connected convolutional networks". United States. doi:10.1364/OPTICA.5.000803.
@article{osti_1459049,
title = {Imaging through glass diffusers using densely connected convolutional networks},
author = {Li, Shuai and Deng, Mo and Lee, Justin and Sinha, Ayan and Barbastathis, George},
abstractNote = {Computational imaging through scatter generally is accomplished by first characterizing the scattering medium so that its forward operator is obtained and then imposing additional priors in the form of regularizers on the reconstruction functional to improve the condition of the originally ill-posed inverse problem. In the functional, the forward operator and regularizer must be entered explicitly or parametrically (e.g., scattering matrices and dictionaries, respectively). However, the process of determining these representations is often incomplete, prone to errors, or infeasible. Recently, deep learning architectures have been proposed to instead learn both the forward operator and regularizer through examples. Here, we propose for the first time, to our knowledge, a convolutional neural network architecture called “IDiffNet” for the problem of imaging through diffuse media and demonstrate that IDiffNet has superior generalization capability through extensive tests with well-calibrated diffusers. We also introduce the negative Pearson correlation coefficient (NPCC) loss function for neural net training and show that the NPCC is more appropriate for spatially sparse objects and strong scattering conditions. Our results show that the convolutional architecture is robust to the choice of prior, as demonstrated by the use of multiple training and testing object databases, and capable of achieving higher space–bandwidth product reconstructions than previously reported.},
doi = {10.1364/OPTICA.5.000803},
journal = {Optica},
number = 7,
volume = 5,
place = {United States},
year = {2018},
month = {7}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
DOI: 10.1364/OPTICA.5.000803

Citation Metrics:
Cited by: 30 works
Citation information provided by
Web of Science

Figures / Tables:

Fig. 1. Fig. 1. : Optical configuration. (a) Experimental arrangement. SF, spatial filter; CL, collimating lens; M, mirror; POL, linear polarizer; BS, beam splitter; SLM, spatial light modulator. (b) Detail of the telescopic imaging system.

Save / Share:

Works referenced in this record:

Widefield lensless imaging through a fiber bundle via speckle correlations
journal, January 2016

  • Porat, Amir; Andresen, Esben Ravn; Rigneault, Hervé
  • Optics Express, Vol. 24, Issue 15
  • DOI: 10.1364/OE.24.016835

Hierarchical classification of images by sparse approximation
journal, December 2013


Object classification through scattering media with deep learning on time resolved measurement
journal, January 2017

  • Satat, Guy; Tancik, Matthew; Gupta, Otkrist
  • Optics Express, Vol. 25, Issue 15
  • DOI: 10.1364/OE.25.017466

Memory Effects in Propagation of Optical Waves through Disordered Media
journal, November 1988


Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information
journal, February 2006

  • Candes, E.J.; Romberg, J.; Tao, T.
  • IEEE Transactions on Information Theory, Vol. 52, Issue 2, p. 489-509
  • DOI: 10.1109/TIT.2005.862083

Generalized optical memory effect
journal, January 2017

  • Osnabrugge, Gerwin; Horstmeyer, Roarke; Papadopoulos, Ioannis N.
  • Optica, Vol. 4, Issue 8
  • DOI: 10.1364/OPTICA.4.000886

Lensless computational imaging through deep learning
journal, January 2017


Image transmission through an opaque material
journal, September 2010

  • Popoff, Sébastien; Lerosey, Geoffroy; Fink, Mathias
  • Nature Communications, Vol. 1, Issue 1
  • DOI: 10.1038/ncomms1078

Single-shot diffuser-encoded light field imaging
conference, May 2016

  • Antipa, Nicholas; Necula, Sylvia; Ng, Ren
  • 2016 IEEE International Conference on Computational Photography (ICCP)
  • DOI: 10.1109/ICCPHOT.2016.7492880

Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy
journal, January 1994


DiffuserCam: lensless single-exposure 3D imaging
journal, December 2017


Measuring the Transmission Matrix in Optics: An Approach to the Study and Control of Light Propagation in Disordered Media
journal, March 2010


Correlations and Fluctuations of Coherent Wave Transmission through Disordered Media
journal, August 1988


ImageNet Large Scale Visual Recognition Challenge
journal, April 2015

  • Russakovsky, Olga; Deng, Jia; Su, Hao
  • International Journal of Computer Vision, Vol. 115, Issue 3
  • DOI: 10.1007/s11263-015-0816-y

Membrane imaging by second-harmonic generation microscopy
journal, January 2000

  • Moreaux, L.; Sandre, O.; Mertz, J.
  • Journal of the Optical Society of America B, Vol. 17, Issue 10
  • DOI: 10.1364/JOSAB.17.001685

SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
journal, December 2017

  • Badrinarayanan, Vijay; Kendall, Alex; Cipolla, Roberto
  • IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 39, Issue 12
  • DOI: 10.1109/TPAMI.2016.2644615

Reference-less measurement of the transmission matrix of a highly scattering material using a DMD and phase retrieval techniques
journal, January 2015

  • Drémeau, Angélique; Liutkus, Antoine; Martina, David
  • Optics Express, Vol. 23, Issue 9
  • DOI: 10.1364/OE.23.011898

Non-invasive imaging through opaque scattering layers
journal, November 2012

  • Bertolotti, Jacopo; van Putten, Elbert G.; Blum, Christian
  • Nature, Vol. 491, Issue 7423
  • DOI: 10.1038/nature11578

Speckle-field digital holographic microscopy
journal, January 2009

  • Park, YongKeun; Choi, Wonshik; Yaqoob, Zahid
  • Optics Express, Vol. 17, Issue 15
  • DOI: 10.1364/OE.17.012285

Non-invasive single-shot imaging through scattering layers and around corners via speckle correlations
journal, August 2014


Reconstruction of an object from the modulus of its Fourier transform
journal, January 1978


Optical sectioning in fluorescence microscopy: OPTICAL SECTIONING IN FLUORESCENCE MICROSCOPY
journal, November 2010


3D imaging in volumetric scattering media using phase-space measurements
journal, January 2015

  • Liu, Hsiou-Yuan; Jonas, Eric; Tian, Lei
  • Optics Express, Vol. 23, Issue 11
  • DOI: 10.1364/OE.23.014461

Deep Convolutional Neural Network for Inverse Problems in Imaging
journal, September 2017

  • Jin, Kyong Hwan; McCann, Michael T.; Froustey, Emmanuel
  • IEEE Transactions on Image Processing, Vol. 26, Issue 9
  • DOI: 10.1109/TIP.2017.2713099

Calibration-free imaging through a multicore fiber using speckle scanning microscopy
journal, January 2016

  • Stasio, Nicolino; Moser, Christophe; Psaltis, Demetri
  • Optics Letters, Vol. 41, Issue 13
  • DOI: 10.1364/OL.41.003078

Compressive Holography
journal, January 2009

  • Brady, David J.; Choi, Kerkil; Marks, Daniel L.
  • Optics Express, Vol. 17, Issue 15
  • DOI: 10.1364/OE.17.013040

Learning-based imaging through scattering media
journal, January 2016

  • Horisaki, Ryoichi; Takagi, Ryosuke; Tanida, Jun
  • Optics Express, Vol. 24, Issue 13
  • DOI: 10.1364/OE.24.013738

Wide-field fluorescence sectioning with hybrid speckle and uniform-illumination microscopy
journal, January 2008

  • Lim, Daryl; Chu, Kengyeh K.; Mertz, Jerome
  • Optics Letters, Vol. 33, Issue 16
  • DOI: 10.1364/OL.33.001819

Two-photon laser scanning fluorescence microscopy
journal, April 1990


    Works referencing / citing this record:

    ImageNet Large Scale Visual Recognition Challenge
    journal, April 2015

    • Russakovsky, Olga; Deng, Jia; Su, Hao
    • International Journal of Computer Vision, Vol. 115, Issue 3
    • DOI: 10.1007/s11263-015-0816-y

    Hierarchical classification of images by sparse approximation
    journal, December 2013


    Non-invasive imaging through opaque scattering layers
    journal, November 2012

    • Bertolotti, Jacopo; van Putten, Elbert G.; Blum, Christian
    • Nature, Vol. 491, Issue 7423
    • DOI: 10.1038/nature11578

    Image transmission through an opaque material
    journal, September 2010

    • Popoff, Sébastien; Lerosey, Geoffroy; Fink, Mathias
    • Nature Communications, Vol. 1, Issue 1
    • DOI: 10.1038/ncomms1078

    Non-invasive single-shot imaging through scattering layers and around corners via speckle correlations
    journal, August 2014


    Optical sectioning in fluorescence microscopy: OPTICAL SECTIONING IN FLUORESCENCE MICROSCOPY
    journal, November 2010


    Two-photon laser scanning fluorescence microscopy
    journal, April 1990


    Learning to decompose the modes in few-mode fibers with deep convolutional neural network
    journal, January 2019


    One-step robust deep learning phase unwrapping
    journal, January 2019


    Reliable deep-learning-based phase imaging with uncertainty quantification
    journal, January 2019


    Single-shot multispectral imaging through a thin scatterer
    journal, January 2019


    Low Photon Count Phase Retrieval Using Deep Learning
    journal, December 2018


    Low Photon Count Phase Retrieval Using Deep Learning
    journal, December 2018


    Learning to decompose the modes in few-mode fibers with deep convolutional neural network
    journal, January 2019


    One-step robust deep learning phase unwrapping
    journal, January 2019


    Reliable deep-learning-based phase imaging with uncertainty quantification
    journal, January 2019


    Single-shot multispectral imaging through a thin scatterer
    journal, January 2019