skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Quantum-inspired computational imaging

Abstract

Computational imaging combines measurement and computational methods with the aim of forming images even when the measurement conditions are weak, few in number, or highly indirect. The recent surge in quantum-inspired imaging sensors, together with a new wave of algorithms allowing on-chip, scalable and robust data processing, has induced an increase of activity with notable results in the domain of low-light flux imaging and sensing. We provide an overview of the major challenges encountered in low-illumination (e.g., ultrafast) imaging and how these problems have recently been addressed for imaging applications in extreme conditions. Finally, these methods provide examples of the future imaging solutions to be developed, for which the best results are expected to arise from an efficient codesign of the sensors and data analysis tools.

Authors:
ORCiD logo [1]; ORCiD logo [1];  [2]; ORCiD logo [3]; ORCiD logo [4]; ORCiD logo [2]
  1. Heriot-Watt Univ., Edinburgh (United Kingdom). School of Engineering and Physical Sciences
  2. Univ. of Glasgow, Scotland (United Kingdom). School of Physics and Astronomy
  3. Boston Univ., MA (United States). Dept. of Electrical and Computer Engineering
  4. Univ. of Michigan, Ann Arbor, MI (United States). Dept. of Electrical Engineering and Computer Science
Publication Date:
Research Org.:
Univ. of Michigan, Ann Arbor, MI (United States)
Sponsoring Org.:
USDOE National Nuclear Security Administration (NNSA), Office of Nonproliferation and Verification Research and Development (NA-22); US Army Research Office (ARO); National Science Foundation (NSF); UK Engineering and Physical Sciences Research Council
OSTI Identifier:
1524450
Grant/Contract Number:  
NA0002534; EP/J015180/1; HR0011-16-C-0030; 1161413; 1422034; EP/M006514/1; EP/M01326X/1
Resource Type:
Accepted Manuscript
Journal Name:
Science
Additional Journal Information:
Journal Volume: 361; Journal Issue: 6403; Journal ID: ISSN 0036-8075
Publisher:
AAAS
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING

Citation Formats

Altmann, Yoann, McLaughlin, Stephen, Padgett, Miles J., Goyal, Vivek K., Hero, Alfred O., and Faccio, Daniele. Quantum-inspired computational imaging. United States: N. p., 2018. Web. doi:10.1126/science.aat2298.
Altmann, Yoann, McLaughlin, Stephen, Padgett, Miles J., Goyal, Vivek K., Hero, Alfred O., & Faccio, Daniele. Quantum-inspired computational imaging. United States. doi:10.1126/science.aat2298.
Altmann, Yoann, McLaughlin, Stephen, Padgett, Miles J., Goyal, Vivek K., Hero, Alfred O., and Faccio, Daniele. Thu . "Quantum-inspired computational imaging". United States. doi:10.1126/science.aat2298. https://www.osti.gov/servlets/purl/1524450.
@article{osti_1524450,
title = {Quantum-inspired computational imaging},
author = {Altmann, Yoann and McLaughlin, Stephen and Padgett, Miles J. and Goyal, Vivek K. and Hero, Alfred O. and Faccio, Daniele},
abstractNote = {Computational imaging combines measurement and computational methods with the aim of forming images even when the measurement conditions are weak, few in number, or highly indirect. The recent surge in quantum-inspired imaging sensors, together with a new wave of algorithms allowing on-chip, scalable and robust data processing, has induced an increase of activity with notable results in the domain of low-light flux imaging and sensing. We provide an overview of the major challenges encountered in low-illumination (e.g., ultrafast) imaging and how these problems have recently been addressed for imaging applications in extreme conditions. Finally, these methods provide examples of the future imaging solutions to be developed, for which the best results are expected to arise from an efficient codesign of the sensors and data analysis tools.},
doi = {10.1126/science.aat2298},
journal = {Science},
number = 6403,
volume = 361,
place = {United States},
year = {2018},
month = {8}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Citation Metrics:
Cited by: 18 works
Citation information provided by
Web of Science

Save / Share:

Works referenced in this record:

Design and Evaluation of Multispectral LiDAR for the Recovery of Arboreal Parameters
journal, August 2014

  • Wallace, Andrew M.; McCarthy, Aongus; Nichol, Caroline J.
  • IEEE Transactions on Geoscience and Remote Sensing, Vol. 52, Issue 8
  • DOI: 10.1109/TGRS.2013.2285942

Observation of laser pulse propagation in optical fibers with a SPAD camera
journal, March 2017

  • Warburton, Ryan; Aniculaesei, Constantin; Clerici, Matteo
  • Scientific Reports, Vol. 7, Issue 1
  • DOI: 10.1038/srep43302

Towards transient imaging at interactive rates with single-photon detectors
conference, May 2018

  • Lindell, David B.; O'Toole, Matthew; Wetzstein, Gordon
  • 2018 IEEE International Conference on Computational Photography (ICCP)
  • DOI: 10.1109/ICCPHOT.2018.8368466

Tracking objects outside the line of sight using 2D intensity images
journal, August 2016

  • Klein, Jonathan; Peters, Christoph; Martín, Jaime
  • Scientific Reports, Vol. 6, Issue 1
  • DOI: 10.1038/srep32491

Computational ghost imaging
journal, December 2008


Nonuniformity Analysis of a 65-kpixel CMOS SPAD Imager
journal, January 2016

  • Antolovic, Ivan Michel; Burri, Samuel; Bruschini, Claudio
  • IEEE Transactions on Electron Devices, Vol. 63, Issue 1
  • DOI: 10.1109/TED.2015.2458295

Performance Analysis of Low-Flux Least-Squares Single-Pixel Imaging
journal, December 2016

  • Shin, Dongeek; Shapiro, Jeffrey H.; Goyal, Vivek K.
  • IEEE Signal Processing Letters, Vol. 23, Issue 12
  • DOI: 10.1109/LSP.2016.2617329

Demonstration of measuring sea fog with an SNSPD-based Lidar system
journal, November 2017


Object recognition through a multi-mode fiber
journal, February 2017


Ultrahigh Speed Photography of Picosecond Light Pulses and Echoes
journal, January 1971


Lensless computational imaging through deep learning
journal, January 2017


Advances in Spectral-Spatial Classification of Hyperspectral Images
journal, March 2013


Adaptive foveated single-pixel imaging with dynamic supersampling
journal, April 2017

  • Phillips, David B.; Sun, Ming-Jie; Taylor, Jonathan M.
  • Science Advances, Vol. 3, Issue 4
  • DOI: 10.1126/sciadv.1601782

Photon-Efficient Computational 3-D and Reflectivity Imaging With Single-Photon Detectors
journal, June 2015

  • Shin, Dongeek; Kirmani, Ahmed; Goyal, Vivek K.
  • IEEE Transactions on Computational Imaging, Vol. 1, Issue 2
  • DOI: 10.1109/TCI.2015.2453093

Optical imaging by means of two-photon quantum entanglement
journal, November 1995


Effects of propagation through atmospheric turbulence on photon statistics
journal, July 2004

  • Milonni, Peter W.; Carter, John H.; Peterson, Charles G.
  • Journal of Optics B: Quantum and Semiclassical Optics, Vol. 6, Issue 8
  • DOI: 10.1088/1464-4266/6/8/018

Single-pixel imaging via compressive sampling
journal, March 2008

  • Duarte, Marco F.; Davenport, Mark A.; Takhar, Dharmpal
  • IEEE Signal Processing Magazine, Vol. 25, Issue 2, p. 83-91
  • DOI: 10.1109/MSP.2007.914730

Screened poisson surface reconstruction
journal, June 2013


Laser-based distance measurement using picosecond resolution time-correlated single-photon counting
journal, May 2000

  • Pellegrini, Sara; Buller, Gerald S.; Smith, Jason M.
  • Measurement Science and Technology, Vol. 11, Issue 6
  • DOI: 10.1088/0957-0233/11/6/314

Three decades of hyperspectral remote sensing of the Earth: A personal view
journal, September 2009


Diffuse Imaging: Creating Optical Images With Unfocused Time-Resolved Illumination and Sensing
journal, January 2012

  • Kirmani, Ahmed; Jeelani, Haris; Montazerhodjat, Vahid
  • IEEE Signal Processing Letters, Vol. 19, Issue 1
  • DOI: 10.1109/LSP.2011.2174222

Comparative study of sampling strategies for sparse photon multispectral lidar imaging: towards mosaic filter arrays
journal, August 2017


Spectral classification of sparse photon depth images
journal, January 2018

  • Altmann, Yoann; Maccarone, Aurora; McCarthy, Aongus
  • Optics Express, Vol. 26, Issue 5
  • DOI: 10.1364/OE.26.005514

Two‐Photon Excitation of Fluorescence by Picosecond Light Pulses
journal, October 1967

  • Giordmaine, J. A.; Rentzepis, P. M.; Shapiro, S. L.
  • Applied Physics Letters, Vol. 11, Issue 7
  • DOI: 10.1063/1.1755105

Hyperspectral imaging – an emerging process analytical tool for food quality and safety control
journal, December 2007


Simultaneous real-time visible and infrared video with single-pixel detectors
journal, May 2015

  • Edgar, Matthew. P.; Gibson, Graham M.; Bowman, Richard W.
  • Scientific Reports, Vol. 5, Issue 1
  • DOI: 10.1038/srep10669

Recent advances in techniques for hyperspectral image processing
journal, September 2009

  • Plaza, Antonio; Benediktsson, Jon Atli; Boardman, Joseph W.
  • Remote Sensing of Environment, Vol. 113
  • DOI: 10.1016/j.rse.2007.07.028

Robust Spectral Unmixing of Sparse Multispectral Lidar Waveforms Using Gamma Markov Random Fields
journal, December 2017

  • Altmann, Yoann; Maccarone, Aurora; McCarthy, Aongus
  • IEEE Transactions on Computational Imaging, Vol. 3, Issue 4
  • DOI: 10.1109/TCI.2017.2703144

Segmentation in dermatological hyperspectral images: dedicated methods
journal, August 2016


Correlated imaging, quantum and classical
journal, July 2004


A 2.5 pJ/b Binary Image Sensor as a Pathfinder for Quanta Image Sensors
journal, January 2016

  • Masoodian, Saleh; Rao, Arun; Ma, Jiaju
  • IEEE Transactions on Electron Devices, Vol. 63, Issue 1
  • DOI: 10.1109/TED.2015.2457418

Detection and tracking of moving objects hidden from view
journal, December 2015

  • Gariepy, Genevieve; Tonolini, Francesco; Henderson, Robert
  • Nature Photonics, Vol. 10, Issue 1
  • DOI: 10.1038/nphoton.2015.234

Non-line-of-sight imaging using a time-gated single photon avalanche diode
journal, January 2015

  • Buttafava, Mauro; Zeman, Jessica; Tosi, Alberto
  • Optics Express, Vol. 23, Issue 16
  • DOI: 10.1364/OE.23.020997

Photon counting compressive depth mapping
journal, January 2013

  • Howland, Gregory A.; Lum, Daniel J.; Ware, Matthew R.
  • Optics Express, Vol. 21, Issue 20
  • DOI: 10.1364/OE.21.023822

Single-pixel three-dimensional imaging with time-based depth resolution
journal, July 2016

  • Sun, Ming-Jie; Edgar, Matthew P.; Gibson, Graham M.
  • Nature Communications, Vol. 7, Issue 1
  • DOI: 10.1038/ncomms12010

Lensless Imaging With Compressive Ultrafast Sensing
journal, September 2017

  • Satat, Guy; Tancik, Matthew; Raskar, Ramesh
  • IEEE Transactions on Computational Imaging, Vol. 3, Issue 3
  • DOI: 10.1109/TCI.2017.2684624

Object Depth Profile and Reflectivity Restoration From Sparse Single-Photon Data Acquired in Underwater Environments
journal, September 2017

  • Halimi, Abderrahim; Maccarone, Aurora; McCarthy, Aongus
  • IEEE Transactions on Computational Imaging, Vol. 3, Issue 3
  • DOI: 10.1109/TCI.2017.2669867

Object classification through scattering media with deep learning on time resolved measurement
journal, January 2017

  • Satat, Guy; Tancik, Matthew; Gupta, Otkrist
  • Optics Express, Vol. 25, Issue 15
  • DOI: 10.1364/OE.25.017466

The physics of ghost imaging
journal, January 2012


Image Reconstruction for Quanta Image Sensors Using Deep Neural Networks
conference, April 2018

  • Choi, Joon Hee; Elgendy, Omar A.; Chan, Stanley H.
  • ICASSP 2018 - 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
  • DOI: 10.1109/ICASSP.2018.8461685

A Bayesian Approach to Denoising of Single-Photon Binary Images
journal, September 2017

  • Altmann, Yoann; Aspden, Reuben; Padgett, Miles
  • IEEE Transactions on Computational Imaging, Vol. 3, Issue 3
  • DOI: 10.1109/TCI.2017.2703900

Scintillation index measurement using time-correlated single-photon counting laser radar
journal, March 2014


High-speed photon-counting laser ranging for broad range of distances
journal, March 2018


Hyperspectral image data analysis
journal, January 2002

  • Landgrebe, D.
  • IEEE Signal Processing Magazine, Vol. 19, Issue 1
  • DOI: 10.1109/79.974718

Arrayable Voltage-Controlled Ring-Oscillator for Direct Time-of-Flight Image Sensors
journal, November 2017

  • Vornicu, Ion; Carmona-Galan, Ricardo; Rodriguez-Vazquez, Angel
  • IEEE Transactions on Circuits and Systems I: Regular Papers, Vol. 64, Issue 11
  • DOI: 10.1109/TCSI.2017.2706324

Lidar Waveform-Based Analysis of Depth Images Constructed Using Sparse Single-Photon Data
journal, May 2016

  • Altmann, Yoann; Ren, Ximing; McCarthy, Aongus
  • IEEE Transactions on Image Processing, Vol. 25, Issue 5
  • DOI: 10.1109/TIP.2016.2526784

A Few Photons Among Many: Unmixing Signal and Noise for Photon-Efficient Active Imaging
journal, September 2017

  • Rapp, Joshua; Goyal, Vivek K.
  • IEEE Transactions on Computational Imaging, Vol. 3, Issue 3
  • DOI: 10.1109/TCI.2017.2706028

Kilometer-range, high resolution depth imaging via 1560 nm wavelength single-photon detection
journal, January 2013

  • McCarthy, Aongus; Krichel, Nils J.; Gemmell, Nathan R.
  • Optics Express, Vol. 21, Issue 7
  • DOI: 10.1364/OE.21.008904

Underwater depth imaging using time-correlated single-photon counting
journal, January 2015

  • Maccarone, Aurora; McCarthy, Aongus; Ren, Ximing
  • Optics Express, Vol. 23, Issue 26
  • DOI: 10.1364/OE.23.033911

Computational multi-depth single-photon imaging
journal, January 2016

  • Shin, Dongeek; Xu, Feihu; Wong, Franco N. C.
  • Optics Express, Vol. 24, Issue 3
  • DOI: 10.1364/OE.24.001873

Joint spectral clustering and range estimation for 3D scene reconstruction using multispectral lidar waveforms
conference, August 2016

  • Altmann, Yoann; Maccarone, Aurora; McCarthy, Aongus
  • 2016 24th European Signal Processing Conference (EUSIPCO)
  • DOI: 10.1109/EUSIPCO.2016.7760301

Hyperspectral Unmixing Overview: Geometrical, Statistical, and Sparse Regression-Based Approaches
journal, April 2012

  • Bioucas-Dias, José M.; Plaza, Antonio; Dobigeon, Nicolas
  • IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, Vol. 5, Issue 2
  • DOI: 10.1109/JSTARS.2012.2194696

First-Photon Imaging
journal, November 2013


Confocal non-line-of-sight imaging based on the light-cone transform
journal, March 2018

  • O’Toole, Matthew; Lindell, David B.; Wetzstein, Gordon
  • Nature, Vol. 555, Issue 7696
  • DOI: 10.1038/nature25489

Kilometer-range depth imaging at 1550 nm wavelength using an InGaAs/InP single-photon avalanche diode detector
journal, January 2013

  • McCarthy, Aongus; Ren, Ximing; Della Frera, Adriano
  • Optics Express, Vol. 21, Issue 19
  • DOI: 10.1364/OE.21.022098

Hyperspectral Pansharpening: A Review
journal, September 2015

  • Loncan, Laetitia; de Almeida, Luis B.; Bioucas-Dias, Jose M.
  • IEEE Geoscience and Remote Sensing Magazine, Vol. 3, Issue 3
  • DOI: 10.1109/MGRS.2015.2440094

Impact of Turbulence in Long Range Quantum and Classical Communications
journal, November 2012


Single-photon 3D imaging with deep sensor fusion
journal, July 2018

  • Lindell, David B.; O'Toole, Matthew; Wetzstein, Gordon
  • ACM Transactions on Graphics, Vol. 37, Issue 4
  • DOI: 10.1145/3197517.3201316

Photon-efficient imaging with a single-photon camera
journal, June 2016

  • Shin, Dongeek; Xu, Feihu; Venkatraman, Dheera
  • Nature Communications, Vol. 7, Issue 1
  • DOI: 10.1038/ncomms12046

Nonline-of-sight laser gated viewing of scattered photons
journal, February 2014


Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging
journal, January 2012

  • Velten, Andreas; Willwacher, Thomas; Gupta, Otkrist
  • Nature Communications, Vol. 3, Issue 1
  • DOI: 10.1038/ncomms1747

Nonlinear Unmixing of Hyperspectral Images: Models and Algorithms
journal, January 2014

  • Dobigeon, Nicolas; Tourneret, Jean-Yves; Richard, Cedric
  • IEEE Signal Processing Magazine, Vol. 31, Issue 1
  • DOI: 10.1109/MSP.2013.2279274

Single view reflectance capture using multiplexed scattering and time-of-flight imaging
journal, December 2011

  • Naik, Nikhil; Zhao, Shuang; Velten, Andreas
  • ACM Transactions on Graphics, Vol. 30, Issue 6
  • DOI: 10.1145/2070781.2024205

Imaging with a small number of photons
journal, January 2015

  • Morris, Peter A.; Aspden, Reuben S.; Bell, Jessica E. C.
  • Nature Communications, Vol. 6, Issue 1
  • DOI: 10.1038/ncomms6913

CMOS Imager With 1024 SPADs and TDCs for Single-Photon Timing and 3-D Time-of-Flight
journal, November 2014

  • Villa, Federica; Lussana, Rudi; Bronzi, Danilo
  • IEEE Journal of Selected Topics in Quantum Electronics, Vol. 20, Issue 6
  • DOI: 10.1109/JSTQE.2014.2342197

Erratum: Single-photon sensitive light-in-flight imaging
journal, February 2015

  • Gariepy, Genevieve; Krstajić, Nikola; Henderson, Robert
  • Nature Communications, Vol. 6, Issue 1
  • DOI: 10.1038/ncomms7408

Non-line-of-sight tracking of people at long range
journal, January 2017

  • Chan, Susan; Warburton, Ryan E.; Gariepy, Genevieve
  • Optics Express, Vol. 25, Issue 9
  • DOI: 10.1364/OE.25.010109

All Photons Imaging Through Volumetric Scattering
journal, September 2016

  • Satat, Guy; Heshmat, Barmak; Raviv, Dan
  • Scientific Reports, Vol. 6, Issue 1
  • DOI: 10.1038/srep33946

Single-photon three-dimensional imaging at up to 10 kilometers range
journal, January 2017

  • Pawlikowska, Agata M.; Halimi, Abderrahim; Lamb, Robert A.
  • Optics Express, Vol. 25, Issue 10
  • DOI: 10.1364/OE.25.011919

An Ultrafast Light gate
journal, September 1969

  • Duguay, M. A.; Hansen, J. W.
  • Applied Physics Letters, Vol. 15, Issue 6
  • DOI: 10.1063/1.1652962

Optimal Threshold Design for Quanta Image Sensor
journal, March 2018

  • Elgendy, Omar A.; Chan, Stanley H.
  • IEEE Transactions on Computational Imaging, Vol. 4, Issue 1
  • DOI: 10.1109/TCI.2017.2781185

Deep learning for real-time single-pixel video
journal, February 2018

  • Higham, Catherine F.; Murray-Smith, Roderick; Padgett, Miles J.
  • Scientific Reports, Vol. 8, Issue 1
  • DOI: 10.1038/s41598-018-20521-y

Estimating Motion and size of moving non-line-of-sight objects in cluttered environments
conference, June 2011

  • Pandharkar, Rohit; Velten, Andreas; Bardagjy, Andrew
  • 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), CVPR 2011
  • DOI: 10.1109/CVPR.2011.5995465

A 160×128 single-photon image sensor with on-pixel 55ps 10b time-to-digital converter
conference, February 2011

  • Veerappan, Chockalingam; Richardson, Justin; Walker, Richard
  • 2011 IEEE International Solid- State Circuits Conference - (ISSCC), 2011 IEEE International Solid-State Circuits Conference
  • DOI: 10.1109/ISSCC.2011.5746333

A 32×32-pixel time-resolved single-photon image sensor with 44.64μm pitch and 19.48% fill-factor with on-chip row/frame skipping features reaching 800kHz observation rate for quantum physics applications
conference, February 2018

  • Gasparini, Leonardo; Zarghami, Majid; Xu, Hesong
  • 2018 IEEE International Solid-State Circuits Conference (ISSCC), 2018 IEEE International Solid - State Circuits Conference - (ISSCC)
  • DOI: 10.1109/ISSCC.2018.8310202

Towards photography through realistic fog
conference, May 2018

  • Satat, Guy; Tancik, Matthew; Raskar, Ramesh
  • 2018 IEEE International Conference on Computational Photography (ICCP)
  • DOI: 10.1109/ICCPHOT.2018.8368463

Discriminating Underwater LiDAR Target Signatures Using Sparse Multi-Spectral Depth Codes
conference, September 2016

  • Chhabra, Puneet; Maccarone, Aurora; McCarthy, Aongus
  • 2016 Sensor Signal Processing for Defence (SSPD)
  • DOI: 10.1109/SSPD.2016.7590595

Joint range estimation and spectral classification for 3D scene reconstruction using multispectral Lidar waveforms
conference, June 2016

  • Altmann, Yoann; Maccarone, Aurora; McCarthy, Aongus
  • 2016 IEEE Statistical Signal Processing Workshop (SSP)
  • DOI: 10.1109/SSP.2016.7551837

Bayesian restoration of reflectivity and range profiles from subsampled single-photon multispectral Lidar data
conference, August 2017


Optimal Stopping Times for Estimating Bernoulli Parameters with Applications to Active Imaging
conference, April 2018

  • Medin, Safa C.; Murray-Bruce, John; Goyal, Vivek K.
  • ICASSP 2018 - 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
  • DOI: 10.1109/ICASSP.2018.8462676