Real-Time Interactive 4D-STEM Phase-Contrast Imaging From Electron Event Representation Data: Less computation with the right representation

Pelz, Philipp M.; Johnson, Ian; Ophus, Colin; Ercius, Peter; Scott, Mary C.

doi:10.1109/msp.2021.3120981

Title: Real-Time Interactive 4D-STEM Phase-Contrast Imaging From Electron Event Representation Data: Less computation with the right representation

Journal Article · Tue Dec 28 00:00:00 EST 2021 · IEEE Signal Processing Magazine

DOI:https://doi.org/10.1109/msp.2021.3120981· OSTI ID:1845304

Pelz, Philipp M. ^[1]; Johnson, Ian ^[2]; Ophus, Colin ^[3]; Ercius, Peter ^[3]; Scott, Mary C. ^[4]

Univ. of California, Berkeley, CA (United States)
Paul Scherrer Inst. (PSI), Villigen (Switzerland)
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Univ. of California, Berkeley, CA (United States); Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)

The arrival of direct electron detectors (DED) with high frame-rates in the field of scanning transmission electron microscopy has enabled many experimental techniques that require collection of a full diffraction pattern at each scan position, a field which is subsumed under the name four dimensional-scanning transmission electron microscopy (4D-STEM). DED frame rates approaching 100 kHz require data transmission rates and data storage capabilities that exceed commonly available computing infrastructure. Current commercial DEDs allow the user to make compromises in pixel bit depth, detector binning or windowing to reduce the per-frame file size and allow higher frame rates. This change in detector specifications requires decisions to be made before data acquisition that may reduce or lose information that could have been advantageous during data analysis. The 4D Camera, a DED with 87 kHz frame-rate developed at Lawrence Berkeley National Laboratory, reduces the raw data to a linear-index encoded electron event representation (EER). Here we show with experimental data from the 4D Camera that linear-index encoded EER and its direct use in 4D-STEM phase contrast imaging methods enables real-time, interactive phase-contrast from large-area 4D-STEM datasets. Furthermore, we detail the computational complexity advantages of the EER and the necessary computational steps to achieve real-time interactive ptychography and center-of-mass differential phase contrast using commonly available hardware accelerators.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Basic Energy Sciences (BES)

Grant/Contract Number:: AC02-05CH11231

OSTI ID:: 1845304

Alternate ID(s):: OSTI ID: 1827921

Journal Information:: IEEE Signal Processing Magazine, Vol. 39, Issue 1; ISSN 1053-5888

Publisher:: IEEECopyright Statement

Country of Publication:: United States

Language:: English

References (29)

py4DSTEM: A Software Package for Four-Dimensional Scanning Transmission Electron Microscopy Data Analysis Savitzky, Benjamin H.; Zeltmann, Steven E.; Hughes, Lauren A. Microscopy and Microanalysis https://doi.org/10.1017/S1431927621000477	journal	May 2021
The theory of super-resolution electron microscopy via Wigner-distribution deconvolution Rodenburg, J. M.; Bates, R. H. T. Philosophical Transactions of the Royal Society of London. Series A: Physical and Engineering Sciences, Vol. 339, Issue 1655, p. 521-553 https://doi.org/10.1098/rsta.1992.0050	journal	June 1992
Mixed-state electron ptychography enables sub-angstrom resolution imaging with picometer precision at low dose Chen, Zhen; Odstrcil, Michal; Jiang, Yi Nature Communications, Vol. 11, Issue 1 https://doi.org/10.1038/s41467-020-16688-6	journal	June 2020
A phase retrieval algorithm for shifting illumination Rodenburg, J. M.; Faulkner, H. M. L. Applied Physics Letters, Vol. 85, Issue 20 https://doi.org/10.1063/1.1823034	journal	November 2004
Efficient phase contrast imaging in STEM using a pixelated detector. Part II: Optimisation of imaging conditions Yang, Hao; Pennycook, Timothy J.; Nellist, Peter D. Ultramicroscopy, Vol. 151 https://doi.org/10.1016/j.ultramic.2014.10.013	journal	April 2015
Low-dose phase retrieval of biological specimens using cryo-electron ptychography Zhou, Liqi; Song, Jingdong; Kim, Judy S. Nature Communications, Vol. 11, Issue 1 https://doi.org/10.1038/s41467-020-16391-6	journal	June 2020
Interpretable and Efficient Interferometric Contrast in Scanning Transmission Electron Microscopy with a Diffraction-Grating Beam Splitter Harvey, Tyler R.; Yasin, Fehmi S.; Chess, Jordan J. Physical Review Applied, Vol. 10, Issue 6 https://doi.org/10.1103/PhysRevApplied.10.061001	journal	December 2018
Phase-contrast imaging of multiply-scattering extended objects at atomic resolution by reconstruction of the scattering matrix Pelz, Philipp M.; Brown, Hamish G.; Stonemeyer, Scott Physical Review Research, Vol. 3, Issue 2 https://doi.org/10.1103/PhysRevResearch.3.023159	journal	May 2021
Single-particle cryo-EM at atomic resolution Nakane, Takanori; Kotecha, Abhay; Sente, Andrija Nature, Vol. 587, Issue 7832 https://doi.org/10.1038/s41586-020-2829-0	journal	October 2020
Determination of the structural phase and octahedral rotation angle in halide perovskites dos Reis, Roberto; Yang, Hao; Ophus, Colin Applied Physics Letters, Vol. 112, Issue 7 https://doi.org/10.1063/1.5017537	journal	February 2018
Simultaneous atomic-resolution electron ptychography and Z-contrast imaging of light and heavy elements in complex nanostructures Yang, H.; Rutte, R. N.; Jones, L. Nature Communications, Vol. 7, Issue 1 https://doi.org/10.1038/ncomms12532	journal	August 2016
Phase reconstruction using fast binary 4D STEM data O'Leary, C. M.; Allen, C. S.; Huang, C. Applied Physics Letters, Vol. 116, Issue 12 https://doi.org/10.1063/1.5143213	journal	March 2020
Electron-event representation data enable efficient cryoEM file storage with full preservation of spatial and temporal resolution Guo, Hui; Franken, Erik; Deng, Yuchen IUCrJ, Vol. 7, Issue 5 https://doi.org/10.1107/S205225252000929X	journal	August 2020
Electron Ptychography of Single Biological Macromolecules Pelz, Philipp M.; Bücker, Robert; Ramm, Georg Microscopy and Microanalysis, Vol. 25, Issue S2 https://doi.org/10.1017/S1431927619001090	journal	August 2019
Resolution beyond the 'information limit' in transmission electron microscopy Nellist, P. D.; McCallum, B. C.; Rodenburg, J. M. Nature, Vol. 374, Issue 6523 https://doi.org/10.1038/374630a0	journal	April 1995
Stabilization of NbTe ₃ , VTe ₃ , and TiTe ₃ via Nanotube Encapsulation Stonemeyer, Scott; Cain, Jeffrey D.; Oh, Sehoon Journal of the American Chemical Society, Vol. 143, Issue 12 https://doi.org/10.1021/jacs.0c10175	journal	December 2020
Beyond the conventional information limit: the relevant coherence function Nellist, P. D.; Rodenburg, J. M. Ultramicroscopy, Vol. 54, Issue 1 https://doi.org/10.1016/0304-3991(94)90092-2	journal	May 1994
Atomic-resolution protein structure determination by cryo-EM Yip, Ka Man; Fischer, Niels; Paknia, Elham Nature, Vol. 587, Issue 7832 https://doi.org/10.1038/s41586-020-2833-4	journal	October 2020
Subsampled STEM-ptychography Stevens, Andrew; Yang, Hao; Hao, Weituo Applied Physics Letters, Vol. 113, Issue 3 https://doi.org/10.1063/1.5040496	journal	July 2018
Electron ptychography of 2D materials to deep sub-ångström resolution Jiang, Yi; Chen, Zhen; Han, Yimo Nature, Vol. 559, Issue 7714 https://doi.org/10.1038/s41586-018-0298-5	journal	July 2018
Image denoising: Can plain neural networks compete with BM3D? Burger, H. C.; Schuler, C. J.; Harmeling, S. 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2012.6247952	conference	June 2012
Differential phase-contrast microscopy at atomic resolution Shibata, Naoya; Findlay, Scott D.; Kohno, Yuji Nature Physics, Vol. 8, Issue 8 https://doi.org/10.1038/nphys2337	journal	June 2012
Experimental tests on double-resolution coherent imaging via STEM Rodenburg, J. M.; McCallum, B. C.; Nellist, P. D. Ultramicroscopy, Vol. 48, Issue 3 https://doi.org/10.1016/0304-3991(93)90105-7	journal	March 1993
Four-Dimensional Scanning Transmission Electron Microscopy (4D-STEM): From Scanning Nanodiffraction to Ptychography and Beyond Ophus, Colin Microscopy and Microanalysis, Vol. 25, Issue 3 https://doi.org/10.1017/S1431927619000497	journal	May 2019
Towards quantitative, atomic-resolution reconstruction of the electrostatic potential via differential phase contrast using electrons Close, R.; Chen, Z.; Shibata, N. Ultramicroscopy, Vol. 159 https://doi.org/10.1016/j.ultramic.2015.09.002	journal	December 2015
Atomic electric fields revealed by a quantum mechanical approach to electron picodiffraction Müller, Knut; Krause, Florian F.; Béché, Armand Nature Communications, Vol. 5, Issue 1 https://doi.org/10.1038/ncomms6653	journal	December 2014
Phase contrast scanning transmission electron microscopy imaging of light and heavy atoms at the limit of contrast and resolution Yücelen, Emrah; Lazić, Ivan; Bosch, Eric G. T. Scientific Reports, Vol. 8, Issue 1 https://doi.org/10.1038/s41598-018-20377-2	journal	February 2018
Low-Dose Aberration-Free Imaging of Li-Rich Cathode Materials at Various States of Charge Using Electron Ptychography Lozano, Juan G.; Martinez, Gerardo T.; Jin, Liyu Nano Letters, Vol. 18, Issue 11 https://doi.org/10.1021/acs.nanolett.8b02718	journal	September 2018
Cluster imaging with a direct detection CMOS pixel sensor in Transmission Electron Microscopy Battaglia, Marco; Contarato, Devis; Denes, Peter Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, Vol. 608, Issue 2 https://doi.org/10.1016/j.nima.2009.07.017	journal	September 2009

Similar Records

High-Dimensional Compressive Scanning Transmission Electron Microscopy

Technical Report · Thu Apr 16 00:00:00 EDT 2020 · OSTI ID:1845304

Reed, Bryan

Development of a fast framing detector for electron microscopy

Conference · Sat Oct 01 00:00:00 EDT 2016 · OSTI ID:1845304

Johnson, Ian J.; Bustillo, Karen C.; Ciston, Jim; +13 more

TEM Video Compressive Sensing

Journal Article · Sat Aug 01 00:00:00 EDT 2015 · Microscopy and Microanalysis · OSTI ID:1845304

Stevens, Andrew; Kovarik, Libor; Abellan, Patricia; +3 more

Related Subjects

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS
transmission electron microscopy
4D-STEM
phase contrast imaging
sparse data representation
scanning electron microscopy
codes
microscopy
data integrity
detectors
graphic processing units

Title: Real-Time Interactive 4D-STEM Phase-Contrast Imaging From Electron Event Representation Data: Less computation with the right representation

Citation Formats

References (29)

Similar Records

Related Subjects