Applying compressive sensing to TEM video: A substantial frame rate increase on any camera

Stevens, Andrew; Kovarik, Libor; Abellan, Patricia; Yuan, Xin; Carin, Lawrence; Browning, Nigel D.

doi:10.1186/s40679-015-0009-3

Title: Applying compressive sensing to TEM video: A substantial frame rate increase on any camera

Journal Article · Thu Aug 13 00:00:00 EDT 2015 · Advanced Structural and Chemical Imaging

DOI:https://doi.org/10.1186/s40679-015-0009-3· OSTI ID:1214819

^[1]; Kovarik, Libor ^[2]; Abellan, Patricia ^[2]; Yuan, Xin ^[3]; Carin, Lawrence ^[3]; Browning, Nigel D. ^[2]

Pacific Northwest National Lab. (PNNL), Richland, WA (United States); Duke Univ., Durham, NC (United States)
Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
Duke Univ., Durham, NC (United States)

One of the main limitations of imaging at high spatial and temporal resolution during in-situ transmission electron microscopy (TEM) experiments is the frame rate of the camera being used to image the dynamic process. While the recent development of direct detectors has provided the hardware to achieve frame rates approaching 0.1 ms, the cameras are expensive and must replace existing detectors. In this paper, we examine the use of coded aperture compressive sensing (CS) methods to increase the frame rate of any camera with simple, low-cost hardware modifications. The coded aperture approach allows multiple sub-frames to be coded and integrated into a single camera frame during the acquisition process, and then extracted upon readout using statistical CS inversion. Here we describe the background of CS and statistical methods in depth and simulate the frame rates and efficiencies for in-situ TEM experiments. Depending on the resolution and signal/noise of the image, it should be possible to increase the speed of any camera by more than an order of magnitude using this approach.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Biological and Environmental Research (BER)

Grant/Contract Number:: AC05-76RL01830

OSTI ID:: 1214819

Journal Information:: Advanced Structural and Chemical Imaging, Vol. 1, Issue 1; ISSN 2198-0926

Publisher:: SpringerCopyright Statement

Country of Publication:: United States

Language:: English

References (43)

The potential for Bayesian compressive sensing to significantly reduce electron dose in high-resolution STEM images Stevens, Andrew; Yang, Hao; Carin, Lawrence Microscopy, Vol. 63, Issue 1 https://doi.org/10.1093/jmicro/dft042	journal	October 2013
Towards sub-Å electron beams Krivanek, O. L.; Dellby, N.; Lupini, A. R. Ultramicroscopy, Vol. 78, Issue 1-4 https://doi.org/10.1016/S0304-3991(99)00013-3	journal	June 1999
A New TwIST: Two-Step Iterative Shrinkage/Thresholding Algorithms for Image Restoration Bioucas-Dias, J. M.; Figueiredo, M. A. T. IEEE Transactions on Image Processing, Vol. 16, Issue 12 https://doi.org/10.1109/TIP.2007.909319	journal	December 2007
Probabilistic Principal Component Analysis Tipping, Michael E.; Bishop, Christopher M. Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 61, Issue 3, p. 611-622 https://doi.org/10.1111/1467-9868.00196	journal	August 1999
Electron tomography based on a total variation minimization reconstruction technique Goris, B.; Van den Broek, W.; Batenburg, K. J. Ultramicroscopy, Vol. 113 https://doi.org/10.1016/j.ultramic.2011.11.004	journal	February 2012
The restricted isometry property and its implications for compressed sensing Candès, Emmanuel J. Comptes Rendus Mathematique, Vol. 346, Issue 9-10, p. 589-592 https://doi.org/10.1016/j.crma.2008.03.014	journal	May 2008
Image resolution and sensitivity in an environmental transmission electron microscope Jinschek, J. R.; Helveg, S. Micron, Vol. 43, Issue 11 https://doi.org/10.1016/j.micron.2012.01.006	journal	November 2012
Compressive Sensing [Lecture Notes] Baraniuk, Richard G. IEEE Signal Processing Magazine, Vol. 24, Issue 4, p. 118-121 https://doi.org/10.1109/MSP.2007.4286571	journal	August 2007
In situ Transmission Electron Microscopy of catalyst sintering DeLaRiva, Andrew T.; Hansen, Thomas W.; Challa, Sivakumar R. Journal of Catalysis, Vol. 308, p. 291-305 https://doi.org/10.1016/j.jcat.2013.08.018	journal	December 2013
Compressed sensing Donoho, D. L. IEEE Transactions on Information Theory, Vol. 52, Issue 4 https://doi.org/10.1109/TIT.2006.871582	journal	April 2006
In Situ Observation of the Electrochemical Lithiation of a Single SnO₂ Nanowire Electrode Huang, J. Y.; Zhong, L.; Wang, C. M. Science, Vol. 330, Issue 6010, p. 1515-1520 https://doi.org/10.1126/science.1195628	journal	December 2010
Visualizing Gas Molecules Interacting with Supported Nanoparticulate Catalysts at Reaction Conditions Yoshida, H.; Kuwauchi, Y.; Jinschek, J. R. Science, Vol. 335, Issue 6066 https://doi.org/10.1126/science.1213194	journal	January 2012
Mixtures of Probabilistic Principal Component Analyzers Tipping, Michael E.; Bishop, Christopher M. Neural Computation, Vol. 11, Issue 2, p. 443-482 https://doi.org/10.1162/089976699300016728	journal	February 1999
Bayesian Methods Gill, Jeff https://doi.org/10.1201/b17888	book	December 2014
Towards 0.1 nm resolution with the first spherically corrected transmission electron microscope Haider, M.; Rose, H.; Uhlemann, S. Journal of Electron Microscopy, Vol. 47, Issue 5, p. 395-405 https://doi.org/10.1093/oxfordjournals.jmicro.a023610	journal	January 1998
In Situ Transmission Electron Microscopy Ferreira, P. J.; Mitsuishi, K.; Stach, E. A. MRS Bulletin, Vol. 33, Issue 02, p. 83-90 https://doi.org/10.1557/mrs2008.20	journal	February 2008
Reduced-dose and high-speed acquisition strategies for multi-dimensional electron microscopy Saghi, Zineb; Benning, Martin; Leary, Rowan Advanced Structural and Chemical Imaging, Vol. 1, Issue 1 https://doi.org/10.1186/s40679-015-0007-5	journal	May 2015
Compressive Sensing on Manifolds Using a Nonparametric Mixture of Factor Analyzers: Algorithm and Performance Bounds Chen, Minhua; Silva, Jorge; Paisley, John IEEE Transactions on Signal Processing, Vol. 58, Issue 12, p. 6140-6155 https://doi.org/10.1109/TSP.2010.2070796	journal	December 2010
A (S)TEM Gas Cell Holder with Localized Laser Heating for In Situ Experiments Mehraeen, Shareghe; McKeown, Joseph T.; Deshmukh, Pushkarraj V. Microscopy and Microanalysis, Vol. 19, Issue 2 https://doi.org/10.1017/S1431927612014419	journal	March 2013
Coded aperture compressive temporal imaging Llull, Patrick; Liao, Xuejun; Yuan, Xin Optics Express, Vol. 21, Issue 9, p. 10526-10545 https://doi.org/10.1364/OE.21.010526	journal	January 2013
Identification of variation in nutritional practice in neonatal units in England and association with clinical outcomes using agnostic machine learning Greenbury, Sam F.; Ougham, Kayleigh; Wu, Jinyi Scientific Reports, Vol. 11, Issue 1 https://doi.org/10.1038/s41598-021-85878-z	journal	March 2021
A tutorial on Bayesian nonparametric models Gershman, Samuel J.; Blei, David M. Journal of Mathematical Psychology, Vol. 56, Issue 1, p. 1-12 https://doi.org/10.1016/j.jmp.2011.08.004	journal	February 2012
Comparison of optimal performance at 300keV of three direct electron detectors for use in low dose electron microscopy McMullan, G.; Faruqi, A. R.; Clare, D. Ultramicroscopy, Vol. 147 https://doi.org/10.1016/j.ultramic.2014.08.002	journal	December 2014
Face recognition using Laplacianfaces IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 27, Issue 3 https://doi.org/10.1109/TPAMI.2005.55	journal	March 2005
Analysis of Ni nanoparticle gas phase sintering Tsyganov, Sergej; Kästner, Jochen; Rellinghaus, Bernd Physical Review B, Vol. 75, Issue 4, Article No. 045421 https://doi.org/10.1103/PhysRevB.75.045421	journal	January 2007
Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information Candes, E.J.; Romberg, J.; Tao, T. IEEE Transactions on Information Theory, Vol. 52, Issue 2, p. 489-509 https://doi.org/10.1109/TIT.2005.862083	journal	February 2006
Controlled Growth of Nanoparticles from Solution with In Situ Liquid Transmission Electron Microscopy Evans, James E.; Jungjohann, Katherine L.; Browning, Nigel D. Nano Letters, Vol. 11, Issue 7 https://doi.org/10.1021/nl201166k	journal	July 2011
$rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation Aharon, M.; Elad, M.; Bruckstein, A. IEEE Transactions on Signal Processing, Vol. 54, Issue 11 https://doi.org/10.1109/TSP.2006.881199	journal	November 2006
Dictionary Learning for Noisy and Incomplete Hyperspectral Images Xing, Zhengming; Zhou, Mingyuan; Castrodad, Alexey SIAM Journal on Imaging Sciences, Vol. 5, Issue 1, p. 33-56 https://doi.org/10.1137/110837486	journal	January 2012
Nonparametric Bayesian Dictionary Learning for Analysis of Noisy and Incomplete Images Zhou, Mingyuan; Chen, Haojun; Paisley, John IEEE Transactions on Image Processing, Vol. 21, Issue 1, p. 130-144 https://doi.org/10.1109/TIP.2011.2160072	journal	January 2012
Atomic-scale electron microscopy at ambient pressure Creemer, J. F.; Helveg, S.; Hoveling, G. H. Ultramicroscopy, Vol. 108, Issue 9 https://doi.org/10.1016/j.ultramic.2008.04.014	journal	August 2008
Emergence of simple-cell receptive field properties by learning a sparse code for natural images Olshausen, Bruno A.; Field, David J. Nature, Vol. 381, Issue 6583 https://doi.org/10.1038/381607a0	journal	June 1996
Quantitative Measurement of the Surface Self-Diffusion on Au Nanoparticles by Aberration-Corrected Transmission Electron Microscopy Surrey, A.; Pohl, D.; Schultz, L. Nano Letters, Vol. 12, Issue 12, p. 6071-6077 https://doi.org/10.1021/nl302280x	journal	February 2012
Advances in the environmental transmission electron microscope (ETEM) for nanoscale in situ studies of gas–solid interactions Jinschek, J. R. Chemical Communications, Vol. 50, Issue 21, p. 2696-2706 https://doi.org/10.1039/C3CC49092K	journal	January 2014
Adaptive temporal compressive sensing for video Yuan, Xin; Yang, Jianbo; Llull, Patrick 2013 IEEE International Conference on Image Processing, p. 14-18 https://doi.org/10.1109/ICIP.2013.6738004	conference	February 2013
Gaussian mixture model for video compressive sensing Yang, Jianbo; Yuan, Xin; Liao, Xuejun 2013 20th IEEE International Conference on Image Processing (ICIP), 2013 IEEE International Conference on Image Processing https://doi.org/10.1109/ICIP.2013.6738005	conference	September 2013
Generalized Alternating Projection for Weighted-$\ell_{2,1}$ Minimization with Applications to Model-Based Compressive Sensing Liao, Xuejun; Li, Hui; Carin, Lawrence SIAM Journal on Imaging Sciences, Vol. 7, Issue 2, p. 797-823 https://doi.org/10.1137/130936658	journal	January 2014
Low-Cost Compressive Sensing for Color Video and Depth Yuan, Xin; Llull, Patrick; Liao, Xuejun 2014 IEEE Conference on Computer Vision and Pattern Recognition, p. 3318-3325 https://doi.org/10.1109/CVPR.2014.424	conference	September 2014
A Tutorial on Bayesian Nonparametric Models Gershman, Samuel J.; Blei, David M. arXiv https://doi.org/10.48550/arxiv.1106.2697	preprint	January 2011
Coded aperture compressive temporal imaging Llull, Patrick; Liao, Xuejun; Yuan, Xin arXiv https://doi.org/10.48550/arxiv.1302.2575	text	January 2013
Robust Uncertainty Principles: Exact Signal Reconstruction from Highly Incomplete Frequency Information Candes, Emmanuel; Romberg, Justin; Tao, Terence arXiv https://doi.org/10.48550/arxiv.math/0409186	preprint	January 2004
Manifold-Based Signal Recovery and Parameter Estimation from Compressive Measurements Wakin, Michael B. arXiv https://doi.org/10.48550/arxiv.1002.1247	preprint	January 2010
Comparison of optimal performance at 300 keV of three direct electron detectors for use in low dose electron microscopy McMullan, G.; Faruqi, A. R.; Clare, D. arXiv https://doi.org/10.48550/arxiv.1406.1389	text	January 2014

Cited By (10)

Automating material image analysis for material discovery Park, Chiwoo; Ding, Yu MRS Communications, Vol. 9, Issue 02 https://doi.org/10.1557/mrc.2019.48	journal	April 2019
Subsampled STEM-ptychography Stevens, Andrew; Yang, Hao; Hao, Weituo Applied Physics Letters, Vol. 113, Issue 3 https://doi.org/10.1063/1.5040496	journal	July 2018
Quantitative Mapping of Nanoscale Chemical Dynamics in Sub-Sampled Operando (S)TEM Images using Spatio-Temporal Analytics Stanfill, Bryan A.; Reehl, Sarah M.; Johnson, Margaret C. ChemCatChem, Vol. 10, Issue 14 https://doi.org/10.1002/cctc.201800333	journal	May 2018
Investigating materials formation with liquid-phase and cryogenic TEM De Yoreo, J. J.; N. A. J. M., Sommerdijk Nature Reviews Materials, Vol. 1, Issue 8 https://doi.org/10.1038/natrevmats.2016.35	journal	June 2016
Methane Mapping with Future Satellite Imaging Spectrometers Ayasse, Alana K.; Dennison, Philip E.; Foote, Markus Remote Sensing, Vol. 11, Issue 24 https://doi.org/10.3390/rs11243054	journal	December 2019
Acquisition of STEM Images by Adaptive Compressive Sensing Xie, Weiyi; Feng, Qianli; Srinivasan, Ramprakash Microscopy and Microanalysis, Vol. 23, Issue S1 https://doi.org/10.1017/s1431927617001167	journal	July 2017
Implementing an accurate and rapid sparse sampling approach for low-dose atomic resolution STEM imaging Kovarik, L.; Stevens, A.; Liyu, A. Applied Physics Letters, Vol. 109, Issue 16 https://doi.org/10.1063/1.4965720	journal	October 2016
A sub-sampled approach to extremely low-dose STEM Stevens, A.; Luzi, L.; Yang, H. Applied Physics Letters, Vol. 112, Issue 4 https://doi.org/10.1063/1.5016192	journal	January 2018
Median Filter Based Compressed Sensing Model with Application to MR Image Reconstruction Yang, Yunyun; Qin, Xuxu; Wu, Boying Mathematical Problems in Engineering, Vol. 2018 https://doi.org/10.1155/2018/8316194	journal	September 2018
Detecting structural variances of Co3O4 catalysts by controlling beam-induced sample alterations in the vacuum of a transmission electron microscope Kisielowski, C.; Frei, H.; Specht, P. Advanced Structural and Chemical Imaging, Vol. 2, Issue 1 https://doi.org/10.1186/s40679-016-0027-9	journal	November 2016

Similar Records

TEM Video Compressive Sensing

Journal Article · Sat Aug 01 00:00:00 EDT 2015 · Microscopy and Microanalysis · OSTI ID:1214819

Stevens, Andrew; Kovarik, Libor; Abellan, Patricia; +3 more

High-Dimensional Compressive Scanning Transmission Electron Microscopy

Technical Report · Mon Jul 09 00:00:00 EDT 2018 · OSTI ID:1214819

Reed, Bryan Walter

Application of Two Phase (Liquid/Gas) Xenon Gamma-Camera for the Detection of Special Nuclear Material and PET Medical Imaging

Technical Report · Tue Aug 27 00:00:00 EDT 2013 · OSTI ID:1214819

McKinsey, Daniel Nicholas

Related Subjects

36 MATERIALS SCIENCE
compressive sensing
transmission electron microscopy
video
coded aperture
nanoparticles
chemical dynamics

Title: Applying compressive sensing to TEM video: A substantial frame rate increase on any camera

Citation Formats

References (43)

Cited By (10)

Similar Records

Related Subjects