Image processing tools for petabyte-scale light sheet microscopy data

Ruan, Xiongtao; Mueller, Matthew; Liu, Gaoxiang; Görlitz, Frederik; Fu, Tian-Ming; Milkie, Daniel E.; Lillvis, Joshua L.; Kuhn, Alexander; Gan Chong, Johnny; Hong, Jason Li; Herr, Chu Yi Aaron; Hercule, Wilmene; Nienhaus, Marc; Killilea, Alison N.; Betzig, Eric; Upadhyayula, Srigokul

doi:10.1038/s41592-024-02475-4

Image processing tools for petabyte-scale light sheet microscopy data

Journal Article · Thu Oct 17 00:00:00 EDT 2024 · Nature Methods

DOI:https://doi.org/10.1038/s41592-024-02475-4· OSTI ID:2477940

^[1]; ^[2]; Liu, Gaoxiang ^[1]; Görlitz, Frederik ^[3]; ^[4]; ^[5]; Lillvis, Joshua L. ^[5]; Kuhn, Alexander ^[6]; ^[1]; ^[1]; Herr, Chu Yi Aaron ^[1]; Hercule, Wilmene ^[1]; Nienhaus, Marc ^[6]; ^[1]; ^[7]; ^[8]

University of California, Berkeley, CA (United States)
University of California, Berkeley, CA (United States); Howard Hughes Medical Institute, Berkeley, CA (United States)
University of California, Berkeley, CA (United States); Univ. of Freiburg (Germany)
Howard Hughes Medical Institute, Ashburn, VA (United States); Princeton Univ., NJ (United States)
Howard Hughes Medical Institute, Ashburn, VA (United States)
NVIDIA, Berlin (Germany)
University of California, Berkeley, CA (United States); Howard Hughes Medical Institute, Berkeley, CA (United States); Howard Hughes Medical Institute, Ashburn, VA (United States)
University of California, Berkeley, CA (United States); Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States); Chan Zuckerberg Biohub, San Francisco, CA (United States)

Light sheet microscopy is a powerful technique for high-speed three-dimensional imaging of subcellular dynamics and large biological specimens. However, it often generates datasets ranging from hundreds of gigabytes to petabytes in size for a single experiment. Conventional computational tools process such images far slower than the time to acquire them and often fail outright due to memory limitations. To address these challenges, we present PetaKit5D, a scalable software solution for efficient petabyte-scale light sheet image processing. This software incorporates a suite of commonly used processing tools that are optimized for memory and performance. Notable advancements include rapid image readers and writers, fast and memory-efficient geometric transformations, high-performance Richardson–Lucy deconvolution and scalable Zarr-based stitching. These features outperform state-of-the-art methods by over one order of magnitude, enabling the processing of petabyte-scale image data at the full teravoxel rates of modern imaging cameras. The software opens new avenues for biological discoveries through large-scale imaging experiments.

View Accepted Manuscript (DOE)

Research Organization:: Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)

Sponsoring Organization:: USDOE Laboratory Directed Research and Development (LDRD) Program; USDOE Office of Science (SC), Basic Energy Sciences (BES). Scientific User Facilities (SUF)

Grant/Contract Number:: AC02-05CH11231

OSTI ID:: 2477940

Journal Information:: Nature Methods, Journal Name: Nature Methods Journal Issue: 12 Vol. 21; ISSN 1548-7091

Publisher:: Nature Publishing GroupCopyright Statement

Country of Publication:: United States

Language:: English

References (52)

OME-Zarr: a cloud-optimized bioimaging file format with international community support Moore, Josh; Basurto-Lozada, Daniela; Besson, Sébastien Histochemistry and Cell Biology, Vol. 160, Issue 3 https://doi.org/10.1007/s00418-023-02209-1	journal	July 2023
Deep learning-based image processing in optical microscopy Melanthota, Sindhoora Kaniyala; Gopal, Dharshini; Chakrabarti, Shweta Biophysical Reviews, Vol. 14, Issue 2 https://doi.org/10.1007/s12551-022-00949-3	journal	April 2022
Imaging in focus: An introduction to denoising bioimages in the era of deep learning Laine, Romain F.; Jacquemet, Guillaume; Krull, Alexander The International Journal of Biochemistry & Cell Biology, Vol. 140 https://doi.org/10.1016/j.biocel.2021.106077	journal	November 2021
Deconvolution-free Subcellular Imaging with Axially Swept Light Sheet Microscopy Dean, Kevin M.; Roudot, Philippe; Welf, Erik S. Biophysical Journal, Vol. 108, Issue 12 https://doi.org/10.1016/j.bpj.2015.05.013	journal	June 2015
Plug-and-Play Genetic Access to Drosophila Cell Types using Exchangeable Exon Cassettes Diao, Fengqiu; Ironfield, Holly; Luan, Haojiang Cell Reports, Vol. 10, Issue 8 https://doi.org/10.1016/j.celrep.2015.01.059	journal	March 2015
A survey on applications of deep learning in microscopy image analysis Liu, Zhichao; Jin, Luhong; Chen, Jincheng Computers in Biology and Medicine, Vol. 134 https://doi.org/10.1016/j.compbiomed.2021.104523	journal	July 2021
Fourier shell correlation threshold criteria van Heel, Marin; Schatz, Michael Journal of Structural Biology, Vol. 151, Issue 3 https://doi.org/10.1016/j.jsb.2005.05.009	journal	September 2005
Fast Three-Dimensional Fluorescence Imaging of Activity in Neural Populations by Objective-Coupled Planar Illumination Microscopy Holekamp, Terrence F.; Turaga, Diwakar; Holy, Timothy E. Neuron, Vol. 57, Issue 5, p. 661-672 https://doi.org/10.1016/j.neuron.2008.01.011	journal	March 2008
3D cell cultures toward quantitative high-throughput drug screening Wang, Yichun; Jeon, Hyunsu Trends in Pharmacological Sciences, Vol. 43, Issue 7 https://doi.org/10.1016/j.tips.2022.03.014	journal	July 2022
Spatially isotropic four-dimensional imaging with dual-view plane illumination microscopy Wu, Yicong; Wawrzusin, Peter; Senseney, Justin Nature Biotechnology, Vol. 31, Issue 11 https://doi.org/10.1038/nbt.2713	journal	November 2013
A BaSiC tool for background and shading correction of optical microscopy images Peng, Tingying; Thorn, Kurt; Schroeder, Timm Nature Communications, Vol. 8, Issue 1 https://doi.org/10.1038/ncomms14836	journal	June 2017
Fiji: an open-source platform for biological-image analysis Schindelin, Johannes; Arganda-Carreras, Ignacio; Frise, Erwin Nature Methods, Vol. 9, Issue 7 https://doi.org/10.1038/nmeth.2019	journal	June 2012
Multiview light-sheet microscope for rapid in toto imaging Krzic, Uros; Gunther, Stefan; Saunders, Timothy E. Nature Methods, Vol. 9, Issue 7 https://doi.org/10.1038/nmeth.2064	journal	June 2012
Rapid adaptive optical recovery of optimal resolution over large volumes Wang, Kai; Milkie, Daniel E.; Saxena, Ankur Nature Methods, Vol. 11, Issue 6 https://doi.org/10.1038/nmeth.2925	journal	April 2014
Fourier ring correlation simplifies image restoration in fluorescence microscopy Koho, Sami; Tortarolo, Giorgio; Castello, Marco Nature Communications, Vol. 10, Issue 1 https://doi.org/10.1038/s41467-019-11024-z	journal	July 2019
Rapid image deconvolution and multiview fusion for optical microscopy Guo, Min; Li, Yue; Su, Yijun Nature Biotechnology, Vol. 38, Issue 11 https://doi.org/10.1038/s41587-020-0560-x	journal	June 2020
BigStitcher: reconstructing high-resolution image datasets of cleared and expanded samples Hörl, David; Rojas Rusak, Fabio; Preusser, Friedrich Nature Methods, Vol. 16, Issue 9 https://doi.org/10.1038/s41592-019-0501-0	journal	August 2019
Light-sheet microscopy of cleared tissues with isotropic, subcellular resolution Chakraborty, Tonmoy; Driscoll, Meghan K.; Jeffery, Elise Nature Methods, Vol. 16, Issue 11 https://doi.org/10.1038/s41592-019-0615-4	journal	October 2019
DaXi—high-resolution, large imaging volume and multi-view single-objective light-sheet microscopy Yang, Bin; Lange, Merlin; Millett-Sikking, Alfred Nature Methods, Vol. 19, Issue 4 https://doi.org/10.1038/s41592-022-01417-2	journal	March 2022
Incorporating the image formation process into deep learning improves network performance Li, Yue; Su, Yijun; Guo, Min Nature Methods, Vol. 19, Issue 11 https://doi.org/10.1038/s41592-022-01652-7	journal	October 2022
Light sheet fluorescence microscopy Stelzer, Ernst H. K.; Strobl, Frederic; Chang, Bo-Jui Nature Reviews Methods Primers, Vol. 1, Issue 1 https://doi.org/10.1038/s43586-021-00069-4	journal	November 2021
Using translational enhancers to increase transgene expression in Drosophila Pfeiffer, Barret D.; Truman, James W.; Rubin, Gerald M. Proceedings of the National Academy of Sciences, Vol. 109, Issue 17 https://doi.org/10.1073/pnas.1204520109	journal	April 2012
Decoding the hydrodynamic properties of microscale helical propellers from Brownian fluctuations Djutanta, Franky; Brown, Peter T.; Nainggolan, Bonfilio Proceedings of the National Academy of Sciences, Vol. 120, Issue 22 https://doi.org/10.1073/pnas.2220033120	journal	May 2023
An iterative technique for the rectification of observed distributions Lucy, L. B. The Astronomical Journal, Vol. 79 https://doi.org/10.1086/111605	journal	June 1974
Deep learning in medical image registration: a review Fu, Yabo; Lei, Yang; Wang, Tonghe Physics in Medicine & Biology, Vol. 65, Issue 20 https://doi.org/10.1088/1361-6560/ab843e	journal	October 2020
Membrane dynamics of dividing cells imaged by lattice light-sheet microscopy Aguet, François; Upadhyayula, Srigokul; Gaudin, Raphaël Molecular Biology of the Cell, Vol. 27, Issue 22 https://doi.org/10.1091/mbc.e16-03-0164	journal	November 2016
Real-time multi-view deconvolution Schmid, Benjamin; Huisken, Jan Bioinformatics, Vol. 31, Issue 20 https://doi.org/10.1093/bioinformatics/btv387	journal	June 2015
Unmatched projector/backprojector pairs in an iterative reconstruction algorithm Zeng, G. L.; Gullberg, G. T. IEEE Transactions on Medical Imaging, Vol. 19, Issue 5 https://doi.org/10.1109/42.870265	journal	May 2000
OpenMP: an industry standard API for shared-memory programming Dagum, L.; Menon, R. IEEE Computational Science and Engineering, Vol. 5, Issue 1 https://doi.org/10.1109/99.660313	journal	January 1998
Parallax-Tolerant Image Stitching Based on Robust Elastic Warping Li, Jing; Wang, Zhengming; Lai, Shiming IEEE Transactions on Multimedia, Vol. 20, Issue 7 https://doi.org/10.1109/TMM.2017.2777461	journal	July 2018
Template matching using fast normalized cross correlation Briechle, Kai; Hanebeck, Uwe D. SPIE Proceedings https://doi.org/10.1117/12.421129	conference	March 2001
Characterization, comparison, and optimization of lattice light sheets Liu, Gaoxiang; Ruan, Xiongtao; Milkie, Daniel E. Science Advances, Vol. 9, Issue 13 https://doi.org/10.1126/sciadv.ade6623	journal	March 2023
Lattice light-sheet microscopy: Imaging molecules to embryos at high spatiotemporal resolution Chen, Bi-Chang; Legant, Wesley R.; Wang, Kai Science, Vol. 346, Issue 6208 https://doi.org/10.1126/science.1257998	journal	October 2014
Expansion microscopy Chen, F.; Tillberg, P. W.; Boyden, E. S. Science, Vol. 347, Issue 6221 https://doi.org/10.1126/science.1260088	journal	January 2015
Visualization and analysis of gene expression in tissue sections by spatial transcriptomics Ståhl, Patrik L.; Salmén, Fredrik; Vickovic, Sanja Science, Vol. 353, Issue 6294 https://doi.org/10.1126/science.aaf2403	journal	June 2016
Observing the cell in its native state: Imaging subcellular dynamics in multicellular organisms Liu, Tsung-Li; Upadhyayula, Srigokul; Milkie, Daniel E. Science, Vol. 360, Issue 6386 https://doi.org/10.1126/science.aaq1392	journal	April 2018
Cortical column and whole-brain imaging with molecular contrast and nanoscale resolution Gao, Ruixuan; Asano, Shoh M.; Upadhyayula, Srigokul Science, Vol. 363, Issue 6424 https://doi.org/10.1126/science.aau8302	journal	January 2019
Apache Spark: a unified engine for big data processing Zaharia, Matei; Franklin, Michael J.; Ghodsi, Ali Communications of the ACM, Vol. 59, Issue 11 https://doi.org/10.1145/2934664	journal	October 2016
Light-Sheet Microscopy and Its Potential for Understanding Developmental Processes Wan, Yinan; McDole, Katie; Keller, Philipp J. Annual Review of Cell and Developmental Biology, Vol. 35, Issue 1 https://doi.org/10.1146/annurev-cellbio-100818-125311	journal	October 2019
A high-level 3D visualization API for Java and ImageJ Schmid, Benjamin; Schindelin, Johannes; Cardona, Albert BMC Bioinformatics, Vol. 11, Issue 1 https://doi.org/10.1186/1471-2105-11-274	journal	May 2010
TeraStitcher - A tool for fast automatic 3D-stitching of teravoxel-sized microscopy images Bria, Alessandro; Iannello, Giulio BMC Bioinformatics, Vol. 13, Issue 1 https://doi.org/10.1186/1471-2105-13-316	journal	November 2012
Acceleration of iterative image restoration algorithms Biggs, David S. C.; Andrews, Mark Applied Optics, Vol. 36, Issue 8 https://doi.org/10.1364/AO.36.001766	journal	March 1997
Open-source software package for on-the-fly deskewing and live viewing of volumetric lightsheet microscopy data Lamb, Jacob R.; Ward, Edward N.; Kaminski, Clemens F. Biomedical Optics Express, Vol. 14, Issue 2 https://doi.org/10.1364/BOE.479977	journal	January 2023
Bayesian-Based Iterative Method of Image Restoration* Richardson, William Hadley Journal of the Optical Society of America, Vol. 62, Issue 1 https://doi.org/10.1364/JOSA.62.000055	journal	January 1972
Optically sectioned imaging by oblique plane microscopy Dunsby, C. Optics Express, Vol. 16, Issue 25, p. 20306-20316 https://doi.org/10.1364/OE.16.020306	journal	January 2008
Image Alignment and Stitching: A Tutorial Szeliski, Richard Foundations and Trends® in Computer Graphics and Vision, Vol. 2, Issue 1 https://doi.org/10.1561/0600000009	journal	January 2007
VisPy: Harnessing The GPU For Fast, High-Level Visualization Campagnola, Luke; Klein, Almar; Larson, Eric Proceedings of the Python in Science Conference https://doi.org/10.25080/Majora-7b98e3ed-00e	conference	January 2015
Using Blosc2 NDim As A Fast Explorer Of The Milky Way (Or Any Other NDim Dataset) Blosc, Project; Alted, Francesc; Iborra, Marta Proceedings of the Python in Science Conference https://doi.org/10.25080/gerudo-f2bc6f59-000	conference	January 2023
Data for: Image processing tools for petabyte-scale light sheet microscopy data (Part 2/2) Ruan, Xiongtao; Mueller, Matthew; Liu, Gaoxiang Dryad https://doi.org/10.5061/dryad.jq2bvq8jd	dataset	January 2024
Data for: Image processing tools for petabyte-scale light sheet microscopy data (Part 1/2) Ruan, Xiongtao; Mueller, Matthew; Liu, Gaoxiang Dryad https://doi.org/10.5061/dryad.kh18932g4	dataset	January 2024
A versatile oblique plane microscope for large-scale and high-resolution imaging of subcellular dynamics Sapoznik, Etai; Chang, Bo-Jui; Huh, Jaewon eLife, Vol. 9 https://doi.org/10.7554/eLife.57681	journal	November 2020
Rapid reconstruction of neural circuits using tissue expansion and light sheet microscopy Lillvis, Joshua L.; Otsuna, Hideo; Ding, Xiaoyu eLife, Vol. 11 https://doi.org/10.7554/eLife.81248	journal	October 2022

Similar Records

A novel partial volume effects correction technique integrating deconvolution associated with denoising within an iterative PET image reconstruction

Journal Article · Sat Feb 14 23:00:00 EST 2015 · Medical Physics · OSTI ID:22413446

Scattered–light–sheet microscopy with sub–cellular resolving power

Journal Article · Tue Jun 06 20:00:00 EDT 2023 · Journal of Biophotonics · OSTI ID:1985268

Related Subjects

97 MATHEMATICS AND COMPUTING
image processing
software

Image processing tools for petabyte-scale light sheet microscopy data

Citation Formats

References (52)

Similar Records

Related Subjects