Near-zero photon bioimaging by fusing deep learning and ultralow-light microscopy

Sheneman, Lucas; Balogun, Sulaimon; Johnson, Jill L.; Harrison, Maria J.; Vasdekis, Andreas E.

doi:10.1073/pnas.2412261122

Title: Near-zero photon bioimaging by fusing deep learning and ultralow-light microscopy

Journal Article · Mon May 19 00:00:00 UTC 2025 · Proceedings of the National Academy of Sciences of the United States of America

DOI: https://doi.org/10.1073/pnas.2412261122 · OSTI ID:2567106

^[1]; Balogun, Sulaimon ^[1]; Johnson, Jill L. ^[1]; Harrison, Maria J. ^[2];

^[3]

Univ. of Idaho, Moscow, ID (United States)
Boyce Thompson Institute for Plant Research, Ithaca, NY (United States)
Univ. of Idaho, Moscow, ID (United States); Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)

Enhancing the reliability and reproducibility of optical microscopy by reducing specimen irradiance continues to be an important biotechnology target. As irradiance levels are reduced, however, the particle nature of light is heightened, giving rise to Poisson noise, or photon sparsity that restricts only a few (0.5%) image pixels to comprise a photon. Photon sparsity can be addressed by collecting approximately 200 photons per pixel; this, however, requires long acquisitions and, as such, suboptimal imaging rates. Here, we introduce near-zero photon bioimaging, a method that operates at kHz rates and 10,000-fold lower irradiance than standard microscopy. To achieve this level of performance, we uniquely combined a judiciously designed epifluorescence microscope enabling ultralow background levels and AI that learns to reconstruct biological images from as low as 0.01 photons per pixel. We demonstrate that near-zero photon bioimaging captures the structure of multicellular and subcellular features with high fidelity, including features represented by nearly zero photons. Beyond optical microscopy, the near-zero photon bioimaging paradigm can be applied in remote sensing, covert applications, and biomedical imaging that utilize damaging or quantum light.

View Journal Article

Cite

Export

Save

Research Organization:: Pacific Northwest National Laboratory (PNNL), Richland, WA (United States); University of Idaho, Moscow, ID (United States)

Sponsoring Organization:: USDOE; USDOE Office of Science (SC), Biological and Environmental Research (BER)

Grant/Contract Number:: SC0022282

OSTI ID:: 2567106

Report Number(s):: PNNL-SA--211522

Journal Information:: Proceedings of the National Academy of Sciences of the United States of America, Journal Name: Proceedings of the National Academy of Sciences of the United States of America Journal Issue: 21 Vol. 122; ISSN 1091-6490; ISSN 0027-8424

Publisher:: National Academy of SciencesCopyright Statement

Country of Publication:: United States

Language:: English

References (59)

Phototoxicity in live fluorescence microscopy, and how to avoid it Icha, Jaroslav; Weber, Michael; Waters, Jennifer C. BioEssays, Vol. 39, Issue 8 https://doi.org/10.1002/bies.201700003	journal	July 2017
Early Stopping - But When? Prechelt, Lutz Lecture Notes in Computer Science https://doi.org/10.1007/3-540-49430-8_3	book	January 1998
U-Net: Convolutional Networks for Biomedical Image Segmentation Ronneberger, Olaf; Fischer, Philipp; Brox, Thomas Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III https://doi.org/10.1007/978-3-319-24574-4_28	book	November 2015
Max-Pooling Dropout for Regularization of Convolutional Neural Networks Wu, Haibing; Gu, Xiaodong Neural Information Processing https://doi.org/10.1007/978-3-319-26532-2_6	book	November 2015
The Importance of Skip Connections in Biomedical Image Segmentation Drozdzal, Michal; Vorontsov, Eugene; Chartrand, Gabriel Deep Learning and Data Labeling for Medical Applications https://doi.org/10.1007/978-3-319-46976-8_19	book	September 2016
Analysis on the Dropout Effect in Convolutional Neural Networks Park, Sungheon; Kwak, Nojun Computer Vision – ACCV 2016 https://doi.org/10.1007/978-3-319-54184-6_12	book	January 2017
Novel interaction of the Hsp90 chaperone machine with Ssl2, an essential DNA helicase in Saccharomyces cerevisiae Flom, Gary; Weekes, Jared; Johnson, Jill L. Current Genetics, Vol. 47, Issue 6 https://doi.org/10.1007/s00294-005-0580-6	journal	May 2005
Progress in quantitative single-molecule localization microscopy Deschout, H.; Shivanandan, A.; Annibale, P. Histochemistry and Cell Biology, Vol. 142, Issue 1 https://doi.org/10.1007/s00418-014-1217-y	journal	April 2014
Dropout vs. batch normalization: an empirical study of their impact to deep learning Garbin, Christian; Zhu, Xingquan; Marques, Oge Multimedia Tools and Applications, Vol. 79, Issue 19-20 https://doi.org/10.1007/s11042-019-08453-9	journal	January 2020
A comprehensive assessment of the structural similarity index Dosselmann, Richard; Yang, Xue Dong Signal, Image and Video Processing, Vol. 5, Issue 1 https://doi.org/10.1007/s11760-009-0144-1	journal	November 2009
The Photon Counting Histogram in Fluorescence Fluctuation Spectroscopy Chen, Yan; Müller, Joachim D.; So, Peter T. C. Biophysical Journal, Vol. 77, Issue 1 https://doi.org/10.1016/S0006-3495(99)76912-2	journal	July 1999
Towards dropout training for convolutional neural networks Wu, Haibing; Gu, Xiaodong Neural Networks, Vol. 71 https://doi.org/10.1016/j.neunet.2015.07.007	journal	November 2015
Photon-Sparse, Poisson Light-Sheet Microscopy Sanchez, Tristin; Subedi, Nava R.; Thompson, Cole ACS Photonics, Vol. 8, Issue 10 https://doi.org/10.1021/acsphotonics.1c01142	journal	September 2021
Scanning Laser Microscope Davidovits, Paul; Egger, M. David Nature, Vol. 223, Issue 5208 https://doi.org/10.1038/223831a0	journal	August 1969
Single photons on demand from a single molecule at room temperature Lounis, B.; Moerner, W. E. Nature, Vol. 407, Issue 6803 https://doi.org/10.1038/35035032	journal	September 2000
Challenges in long-term imaging and quantification of single-cell dynamics Skylaki, Stavroula; Hilsenbeck, Oliver; Schroeder, Timm Nature Biotechnology, Vol. 34, Issue 11 https://doi.org/10.1038/nbt.3713	journal	November 2016
Deep learning massively accelerates super-resolution localization microscopy Ouyang, Wei; Aristov, Andrey; Lelek, Mickaël Nature Biotechnology, Vol. 36, Issue 5 https://doi.org/10.1038/nbt.4106	journal	April 2018
The elusive Heisenberg limit in quantum-enhanced metrology Demkowicz-Dobrzański, Rafał; Kołodyński, Jan; Guţă, Mădălin Nature Communications, Vol. 3, Issue 1 https://doi.org/10.1038/ncomms2067	journal	January 2012
Imaging with a small number of photons Morris, Peter A.; Aspden, Reuben S.; Bell, Jessica E. C. Nature Communications, Vol. 6, Issue 1 https://doi.org/10.1038/ncomms6913	journal	January 2015
OpenSPIM: an open-access light-sheet microscopy platform Pitrone, Peter G.; Schindelin, Johannes; Stuyvenberg, Luke Nature Methods, Vol. 10, Issue 7 https://doi.org/10.1038/nmeth.2507	journal	June 2013
Light-sheet microscopy using an Airy beam Vettenburg, Tom; Dalgarno, Heather I. C.; Nylk, Jonathan Nature Methods, Vol. 11, Issue 5 https://doi.org/10.1038/nmeth.2922	journal	April 2014
Light-sheet fluorescence microscopy for quantitative biology Stelzer, Ernst H. K. Nature Methods, Vol. 12, Issue 1 https://doi.org/10.1038/nmeth.3219	journal	December 2014
Quantitative assessment of fluorescent proteins Cranfill, Paula J.; Sell, Brittney R.; Baird, Michelle A. Nature Methods, Vol. 13, Issue 7 https://doi.org/10.1038/nmeth.3891	journal	May 2016
A guide to light-sheet fluorescence microscopy for multiscale imaging Power, Rory M.; Huisken, Jan Nature Methods, Vol. 14, Issue 4 https://doi.org/10.1038/nmeth.4224	journal	April 2017
Assessing phototoxicity in live fluorescence imaging Laissue, P. Philippe; Alghamdi, Rana A.; Tomancak, Pavel Nature Methods, Vol. 14, Issue 7 https://doi.org/10.1038/nmeth.4344	journal	July 2017
Single-photon detectors for optical quantum information applications Hadfield, Robert H. Nature Photonics, Vol. 3, Issue 12 https://doi.org/10.1038/nphoton.2009.230	journal	December 2009
Phase imaging with computational specificity (PICS) for measuring dry mass changes in sub-cellular compartments Kandel, Mikhail E.; He, Yuchen R.; Lee, Young Jae Nature Communications, Vol. 11, Issue 1 https://doi.org/10.1038/s41467-020-20062-x	journal	December 2020
CASPI: collaborative photon processing for active single-photon imaging Lee, Jongho; Ingle, Atul; Chacko, Jenu V. Nature Communications, Vol. 14, Issue 1 https://doi.org/10.1038/s41467-023-38893-9	journal	May 2023
Quantum-enhanced nonlinear microscopy Casacio, Catxere A.; Madsen, Lars S.; Terrasson, Alex Nature, Vol. 594, Issue 7862 https://doi.org/10.1038/s41586-021-03528-w	journal	June 2021
Real-time denoising enables high-sensitivity fluorescence time-lapse imaging beyond the shot-noise limit Li, Xinyang; Li, Yixin; Zhou, Yiliang Nature Biotechnology, Vol. 41, Issue 2 https://doi.org/10.1038/s41587-022-01450-8	journal	September 2022
Parameter-free image resolution estimation based on decorrelation analysis Descloux, A.; Grußmayer, K. S.; Radenovic, A. Nature Methods, Vol. 16, Issue 9 https://doi.org/10.1038/s41592-019-0515-7	journal	August 2019
Avoiding a replication crisis in deep-learning-based bioimage analysis Laine, Romain F.; Arganda-Carreras, Ignacio; Henriques, Ricardo Nature Methods, Vol. 18, Issue 10 https://doi.org/10.1038/s41592-021-01284-3	journal	October 2021
Integrative quantitative-phase and airy light-sheet imaging Subedi, N. R.; Jung, P. S.; Bredeweg, E. L. Scientific Reports, Vol. 10, Issue 1 https://doi.org/10.1038/s41598-020-76730-x	journal	November 2020
Density fluctuations, homeostasis, and reproduction effects in bacteria Nemati, Shahla; Singh, Abhyudai; Dhuey, Scott D. Communications Biology, Vol. 5, Issue 1 https://doi.org/10.1038/s42003-022-03348-2	journal	April 2022
Generative adversarial network for low‐light image enhancement Li, Fei; Zheng, Jiangbin; Zhang, Yuan‐fang IET Image Processing, Vol. 15, Issue 7 https://doi.org/10.1049/ipr2.12124	journal	January 2021
Invited Review Article: Single-photon sources and detectors Eisaman, M. D.; Fan, J.; Migdall, A. Review of Scientific Instruments, Vol. 82, Issue 7 https://doi.org/10.1063/1.3610677	journal	July 2011
DELLA proteins regulate arbuscule formation in arbuscular mycorrhizal symbiosis Floss, D. S.; Levy, J. G.; Levesque-Tremblay, V. Proceedings of the National Academy of Sciences, Vol. 110, Issue 51 https://doi.org/10.1073/pnas.1308973110	journal	December 2013
Video-rate Raman-based metabolic imaging by Airy light-sheet illumination and photon-sparse detection Dunn, Lochlann; Luo, Haokun; Subedi, Nava R. Proceedings of the National Academy of Sciences, Vol. 120, Issue 9 https://doi.org/10.1073/pnas.2210037120	journal	February 2023
Quantum Optics: An Introduction Fox, Mark https://doi.org/10.1093/oso/9780198566724.001.0001	book	April 2006
Near Zero Photon Bioimaging Sheneman, Lucas; Balogun, Sulaimon; Johnson, Jill L. https://doi.org/10.1101/2024.06.12.598699	preprint	June 2024
Low Photon Count Phase Retrieval Using Deep Learning Goy, Alexandre; Arthur, Kwabena; Li, Shuai Physical Review Letters, Vol. 121, Issue 24 https://doi.org/10.1103/PhysRevLett.121.243902	journal	December 2018
Optical detection and spectroscopy of single molecules in a solid Moerner, W. E.; Kador, L. Physical Review Letters, Vol. 62, Issue 21 https://doi.org/10.1103/PhysRevLett.62.2535	journal	May 1989
Max-pooling convolutional neural networks for vision-based hand gesture recognition Nagi, Jawad; Ducatelle, Frederick; Di Caro, Gianni A. 2011 IEEE International Conference on Signal and Image Processing Applications (ICSIPA) https://doi.org/10.1109/ICSIPA.2011.6144164	conference	November 2011
Image Denoising in Mixed Poisson–Gaussian Noise Luisier, F.; Blu, Thierry; Unser, M. IEEE Transactions on Image Processing, Vol. 20, Issue 3 https://doi.org/10.1109/TIP.2010.2073477	journal	March 2011
Optical Sectioning Deep Inside Live Embryos by Selective Plane Illumination Microscopy Huisken, J.; Swoger, J.; Del Bene, F. Science, Vol. 305, Issue 5686, p. 1007-1009 https://doi.org/10.1126/science.1100035	journal	August 2004
Imaging Intracellular Fluorescent Proteins at Nanometer Resolution Betzig, E.; Patterson, G. H.; Sougrat, R. Science, Vol. 313, Issue 5793 https://doi.org/10.1126/science.1127344	journal	September 2006
Video-Rate Molecular Imaging in Vivo with Stimulated Raman Scattering Saar, B. G.; Freudiger, C. W.; Reichman, J. Science, Vol. 330, Issue 6009 https://doi.org/10.1126/science.1197236	journal	December 2010
Lattice light-sheet microscopy: Imaging molecules to embryos at high spatiotemporal resolution Chen, Bi-Chang; Legant, Wesley R.; Wang, Kai Science, Vol. 346, Issue 6208 https://doi.org/10.1126/science.1257998	journal	October 2014
Two-photon laser scanning fluorescence microscopy Denk, W.; Strickler, J.; Webb, W. Science, Vol. 248, Issue 4951 https://doi.org/10.1126/science.2321027	journal	April 1990
A unified initiative to harness Earth's microbiomes Alivisatos, A. P.; Blaser, M. J.; Brodie, E. L. Science, Vol. 350, Issue 6260 https://doi.org/10.1126/science.aac8480	journal	October 2015
The Green Fluorescent Protein Tsien, Roger Y. Annual Review of Biochemistry, Vol. 67, Issue 1, p. 509-544 https://doi.org/10.1146/annurev.biochem.67.1.509	journal	June 1998
U-Net-Based Medical Image Segmentation Yin, Xiao-Xia; Sun, Le; Fu, Yuhan Journal of Healthcare Engineering, Vol. 2022 https://doi.org/10.1155/2022/4189781	journal	April 2022
A survey on Image Data Augmentation for Deep Learning Shorten, Connor; Khoshgoftaar, Taghi M. Journal of Big Data, Vol. 6, Issue 1 https://doi.org/10.1186/s40537-019-0197-0	journal	July 2019
Light-sheet microscopy: a tutorial Olarte, Omar E.; Andilla, Jordi; Gualda, Emilio J. Advances in Optics and Photonics, Vol. 10, Issue 1 https://doi.org/10.1364/AOP.10.000111	journal	January 2018
Coded aperture compressive temporal imaging Llull, Patrick; Liao, Xuejun; Yuan, Xin Optics Express, Vol. 21, Issue 9, p. 10526-10545 https://doi.org/10.1364/OE.21.010526	journal	January 2013
Photon-counting, energy-resolving and super-resolution phase contrast X-ray imaging using an integrating detector O’Connell, Dylan W.; Morgan, Kaye S.; Ruben, Gary Optics Express, Vol. 28, Issue 5 https://doi.org/10.1364/OE.384928	journal	January 2020
Noise-robust latent vector reconstruction in ptychography using deep generative models Seifert, Jacob; Shao, Yifeng; Mosk, Allard P. Optics Express, Vol. 32, Issue 1 https://doi.org/10.1364/OE.513556	journal	December 2023
Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy Hell, Stefan W.; Wichmann, Jan Optics Letters, Vol. 19, Issue 11 https://doi.org/10.1364/OL.19.000780	journal	January 1994
Advanced methods of microscope control using μManager software Edelstein, Arthur D.; Tsuchida, Mark A.; Amodaj, Nenad Journal of Biological Methods, Vol. 1, Issue 2 https://doi.org/10.14440/jbm.2014.36	journal	July 2014

Similar Records

Video-rate Raman-based metabolic imaging by Airy light-sheet illumination and photon-sparse detection

Journal Article · Wed Feb 22 00:00:00 UTC 2023 · Proceedings of the National Academy of Sciences of the United States of America · OSTI ID:1958232

Dunn, Lochlann; Luo, Haokun; Subedi, Nava R.; +4 more

Photon-Sparse, Poisson Light-Sheet Microscopy

Journal Article · Mon Sep 20 00:00:00 UTC 2021 · ACS Photonics · OSTI ID:1825070

Sanchez, Tristin; Subedi, Nava R.; Thompson, Cole; +2 more

Graphene-based Materials for Biosensing and Bioimaging

Journal Article · Sat Dec 01 04:00:00 UTC 2012 · MRS Bulletin, 37(12):1290-1296 · OSTI ID:1056755

Du, Dan; Yang, Yuqi; Lin, Yuehe

Related Subjects

AI
bioimaging
imaging
microscopy
single-photon

Title: Near-zero photon bioimaging by fusing deep learning and ultralow-light microscopy

Citation Formats

References (59)

Similar Records

Related Subjects