Applying compressive sensing to TEM video: A substantial frame rate increase on any camera
Abstract
One of the main limitations of imaging at high spatial and temporal resolution during in-situ transmission electron microscopy (TEM) experiments is the frame rate of the camera being used to image the dynamic process. While the recent development of direct detectors has provided the hardware to achieve frame rates approaching 0.1 ms, the cameras are expensive and must replace existing detectors. In this paper, we examine the use of coded aperture compressive sensing (CS) methods to increase the frame rate of any camera with simple, low-cost hardware modifications. The coded aperture approach allows multiple sub-frames to be coded and integrated into a single camera frame during the acquisition process, and then extracted upon readout using statistical CS inversion. Here we describe the background of CS and statistical methods in depth and simulate the frame rates and efficiencies for in-situ TEM experiments. Depending on the resolution and signal/noise of the image, it should be possible to increase the speed of any camera by more than an order of magnitude using this approach.
- Authors:
-
[1];
- Pacific Northwest National Lab. (PNNL), Richland, WA (United States); Duke Univ., Durham, NC (United States)
- Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
- Duke Univ., Durham, NC (United States)
- Publication Date:
- Research Org.:
- Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC), Biological and Environmental Research (BER)
- OSTI Identifier:
- 1214819
- Grant/Contract Number:
- AC05-76RL01830
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Advanced Structural and Chemical Imaging
- Additional Journal Information:
- Journal Volume: 1; Journal Issue: 1; Journal ID: ISSN 2198-0926
- Publisher:
- Springer
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 36 MATERIALS SCIENCE; compressive sensing; transmission electron microscopy; video; coded aperture; nanoparticles; chemical dynamics
Citation Formats
Stevens, Andrew, Kovarik, Libor, Abellan, Patricia, Yuan, Xin, Carin, Lawrence, and Browning, Nigel D. Applying compressive sensing to TEM video: A substantial frame rate increase on any camera. United States: N. p., 2015.
Web. doi:10.1186/s40679-015-0009-3.
Stevens, Andrew, Kovarik, Libor, Abellan, Patricia, Yuan, Xin, Carin, Lawrence, & Browning, Nigel D. Applying compressive sensing to TEM video: A substantial frame rate increase on any camera. United States. https://doi.org/10.1186/s40679-015-0009-3
Stevens, Andrew, Kovarik, Libor, Abellan, Patricia, Yuan, Xin, Carin, Lawrence, and Browning, Nigel D. Thu .
"Applying compressive sensing to TEM video: A substantial frame rate increase on any camera". United States. https://doi.org/10.1186/s40679-015-0009-3. https://www.osti.gov/servlets/purl/1214819.
@article{osti_1214819,
title = {Applying compressive sensing to TEM video: A substantial frame rate increase on any camera},
author = {Stevens, Andrew and Kovarik, Libor and Abellan, Patricia and Yuan, Xin and Carin, Lawrence and Browning, Nigel D.},
abstractNote = {One of the main limitations of imaging at high spatial and temporal resolution during in-situ transmission electron microscopy (TEM) experiments is the frame rate of the camera being used to image the dynamic process. While the recent development of direct detectors has provided the hardware to achieve frame rates approaching 0.1 ms, the cameras are expensive and must replace existing detectors. In this paper, we examine the use of coded aperture compressive sensing (CS) methods to increase the frame rate of any camera with simple, low-cost hardware modifications. The coded aperture approach allows multiple sub-frames to be coded and integrated into a single camera frame during the acquisition process, and then extracted upon readout using statistical CS inversion. Here we describe the background of CS and statistical methods in depth and simulate the frame rates and efficiencies for in-situ TEM experiments. Depending on the resolution and signal/noise of the image, it should be possible to increase the speed of any camera by more than an order of magnitude using this approach.},
doi = {10.1186/s40679-015-0009-3},
journal = {Advanced Structural and Chemical Imaging},
number = 1,
volume = 1,
place = {United States},
year = {Thu Aug 13 00:00:00 EDT 2015},
month = {Thu Aug 13 00:00:00 EDT 2015}
}
Free Publicly Available Full Text
Publisher's Version of Record
Other availability
Search WorldCat to find libraries that may hold this journal
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
The potential for Bayesian compressive sensing to significantly reduce electron dose in high-resolution STEM images
journal, October 2013
- Stevens, Andrew; Yang, Hao; Carin, Lawrence
- Microscopy, Vol. 63, Issue 1
Towards sub-Å electron beams
journal, June 1999
- Krivanek, O. L.; Dellby, N.; Lupini, A. R.
- Ultramicroscopy, Vol. 78, Issue 1-4
A New TwIST: Two-Step Iterative Shrinkage/Thresholding Algorithms for Image Restoration
journal, December 2007
- Bioucas-Dias, J. M.; Figueiredo, M. A. T.
- IEEE Transactions on Image Processing, Vol. 16, Issue 12
Probabilistic Principal Component Analysis
journal, August 1999
- Tipping, Michael E.; Bishop, Christopher M.
- Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 61, Issue 3, p. 611-622
Electron tomography based on a total variation minimization reconstruction technique
journal, February 2012
- Goris, B.; Van den Broek, W.; Batenburg, K. J.
- Ultramicroscopy, Vol. 113
The restricted isometry property and its implications for compressed sensing
journal, May 2008
- Candès, Emmanuel J.
- Comptes Rendus Mathematique, Vol. 346, Issue 9-10, p. 589-592
Image resolution and sensitivity in an environmental transmission electron microscope
journal, November 2012
- Jinschek, J. R.; Helveg, S.
- Micron, Vol. 43, Issue 11
Compressive Sensing [Lecture Notes]
journal, August 2007
- Baraniuk, Richard G.
- IEEE Signal Processing Magazine, Vol. 24, Issue 4, p. 118-121
In situ Transmission Electron Microscopy of catalyst sintering
journal, December 2013
- DeLaRiva, Andrew T.; Hansen, Thomas W.; Challa, Sivakumar R.
- Journal of Catalysis, Vol. 308, p. 291-305
Compressed sensing
journal, April 2006
- Donoho, D. L.
- IEEE Transactions on Information Theory, Vol. 52, Issue 4
In Situ Observation of the Electrochemical Lithiation of a Single SnO2 Nanowire Electrode
journal, December 2010
- Huang, J. Y.; Zhong, L.; Wang, C. M.
- Science, Vol. 330, Issue 6010, p. 1515-1520
Visualizing Gas Molecules Interacting with Supported Nanoparticulate Catalysts at Reaction Conditions
journal, January 2012
- Yoshida, H.; Kuwauchi, Y.; Jinschek, J. R.
- Science, Vol. 335, Issue 6066
Mixtures of Probabilistic Principal Component Analyzers
journal, February 1999
- Tipping, Michael E.; Bishop, Christopher M.
- Neural Computation, Vol. 11, Issue 2, p. 443-482
Towards 0.1 nm resolution with the first spherically corrected transmission electron microscope
journal, January 1998
- Haider, M.; Rose, H.; Uhlemann, S.
- Journal of Electron Microscopy, Vol. 47, Issue 5, p. 395-405
In Situ Transmission Electron Microscopy
journal, February 2008
- Ferreira, P. J.; Mitsuishi, K.; Stach, E. A.
- MRS Bulletin, Vol. 33, Issue 02, p. 83-90
Reduced-dose and high-speed acquisition strategies for multi-dimensional electron microscopy
journal, May 2015
- Saghi, Zineb; Benning, Martin; Leary, Rowan
- Advanced Structural and Chemical Imaging, Vol. 1, Issue 1
Compressive Sensing on Manifolds Using a Nonparametric Mixture of Factor Analyzers: Algorithm and Performance Bounds
journal, December 2010
- Chen, Minhua; Silva, Jorge; Paisley, John
- IEEE Transactions on Signal Processing, Vol. 58, Issue 12, p. 6140-6155
A (S)TEM Gas Cell Holder with Localized Laser Heating for In Situ Experiments
journal, March 2013
- Mehraeen, Shareghe; McKeown, Joseph T.; Deshmukh, Pushkarraj V.
- Microscopy and Microanalysis, Vol. 19, Issue 2
Coded aperture compressive temporal imaging
journal, January 2013
- Llull, Patrick; Liao, Xuejun; Yuan, Xin
- Optics Express, Vol. 21, Issue 9, p. 10526-10545
Identification of variation in nutritional practice in neonatal units in England and association with clinical outcomes using agnostic machine learning
journal, March 2021
- Greenbury, Sam F.; Ougham, Kayleigh; Wu, Jinyi
- Scientific Reports, Vol. 11, Issue 1
A tutorial on Bayesian nonparametric models
journal, February 2012
- Gershman, Samuel J.; Blei, David M.
- Journal of Mathematical Psychology, Vol. 56, Issue 1, p. 1-12
Comparison of optimal performance at 300keV of three direct electron detectors for use in low dose electron microscopy
journal, December 2014
- McMullan, G.; Faruqi, A. R.; Clare, D.
- Ultramicroscopy, Vol. 147
Face recognition using Laplacianfaces
journal, March 2005
- Xiaofei He,
- IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 27, Issue 3
Analysis of Ni nanoparticle gas phase sintering
journal, January 2007
- Tsyganov, Sergej; Kästner, Jochen; Rellinghaus, Bernd
- Physical Review B, Vol. 75, Issue 4, Article No. 045421
Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information
journal, February 2006
- Candes, E.J.; Romberg, J.; Tao, T.
- IEEE Transactions on Information Theory, Vol. 52, Issue 2, p. 489-509
Controlled Growth of Nanoparticles from Solution with In Situ Liquid Transmission Electron Microscopy
journal, July 2011
- Evans, James E.; Jungjohann, Katherine L.; Browning, Nigel D.
- Nano Letters, Vol. 11, Issue 7
$rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation
journal, November 2006
- Aharon, M.; Elad, M.; Bruckstein, A.
- IEEE Transactions on Signal Processing, Vol. 54, Issue 11
Dictionary Learning for Noisy and Incomplete Hyperspectral Images
journal, January 2012
- Xing, Zhengming; Zhou, Mingyuan; Castrodad, Alexey
- SIAM Journal on Imaging Sciences, Vol. 5, Issue 1, p. 33-56
Nonparametric Bayesian Dictionary Learning for Analysis of Noisy and Incomplete Images
journal, January 2012
- Zhou, Mingyuan; Chen, Haojun; Paisley, John
- IEEE Transactions on Image Processing, Vol. 21, Issue 1, p. 130-144
Atomic-scale electron microscopy at ambient pressure
journal, August 2008
- Creemer, J. F.; Helveg, S.; Hoveling, G. H.
- Ultramicroscopy, Vol. 108, Issue 9
Emergence of simple-cell receptive field properties by learning a sparse code for natural images
journal, June 1996
- Olshausen, Bruno A.; Field, David J.
- Nature, Vol. 381, Issue 6583
Quantitative Measurement of the Surface Self-Diffusion on Au Nanoparticles by Aberration-Corrected Transmission Electron Microscopy
journal, February 2012
- Surrey, A.; Pohl, D.; Schultz, L.
- Nano Letters, Vol. 12, Issue 12, p. 6071-6077
Advances in the environmental transmission electron microscope (ETEM) for nanoscale in situ studies of gas–solid interactions
journal, January 2014
- Jinschek, J. R.
- Chemical Communications, Vol. 50, Issue 21, p. 2696-2706
Adaptive temporal compressive sensing for video
conference, February 2013
- Yuan, Xin; Yang, Jianbo; Llull, Patrick
- 2013 IEEE International Conference on Image Processing, p. 14-18
Gaussian mixture model for video compressive sensing
conference, September 2013
- Yang, Jianbo; Yuan, Xin; Liao, Xuejun
- 2013 20th IEEE International Conference on Image Processing (ICIP), 2013 IEEE International Conference on Image Processing
Generalized Alternating Projection for Weighted-$\ell_{2,1}$ Minimization with Applications to Model-Based Compressive Sensing
journal, January 2014
- Liao, Xuejun; Li, Hui; Carin, Lawrence
- SIAM Journal on Imaging Sciences, Vol. 7, Issue 2, p. 797-823
Low-Cost Compressive Sensing for Color Video and Depth
conference, September 2014
- Yuan, Xin; Llull, Patrick; Liao, Xuejun
- 2014 IEEE Conference on Computer Vision and Pattern Recognition, p. 3318-3325
A Tutorial on Bayesian Nonparametric Models
preprint, January 2011
- Gershman, Samuel J.; Blei, David M.
- arXiv
Coded aperture compressive temporal imaging
text, January 2013
- Llull, Patrick; Liao, Xuejun; Yuan, Xin
- arXiv
Robust Uncertainty Principles: Exact Signal Reconstruction from Highly Incomplete Frequency Information
preprint, January 2004
- Candes, Emmanuel; Romberg, Justin; Tao, Terence
- arXiv
Image resolution and sensitivity in an environmental transmission electron microscope
journal, November 2012
- Jinschek, J. R.; Helveg, S.
- Micron, Vol. 43, Issue 11
Atomic-scale electron microscopy at ambient pressure
journal, August 2008
- Creemer, J. F.; Helveg, S.; Hoveling, G. H.
- Ultramicroscopy, Vol. 108, Issue 9
Manifold-Based Signal Recovery and Parameter Estimation from Compressive Measurements
preprint, January 2010
- Wakin, Michael B.
- arXiv
Comparison of optimal performance at 300 keV of three direct electron detectors for use in low dose electron microscopy
text, January 2014
- McMullan, G.; Faruqi, A. R.; Clare, D.
- arXiv
Works referencing / citing this record:
Automating material image analysis for material discovery
journal, April 2019
- Park, Chiwoo; Ding, Yu
- MRS Communications, Vol. 9, Issue 02
Subsampled STEM-ptychography
journal, July 2018
- Stevens, Andrew; Yang, Hao; Hao, Weituo
- Applied Physics Letters, Vol. 113, Issue 3
Quantitative Mapping of Nanoscale Chemical Dynamics in Sub-Sampled Operando (S)TEM Images using Spatio-Temporal Analytics
journal, May 2018
- Stanfill, Bryan A.; Reehl, Sarah M.; Johnson, Margaret C.
- ChemCatChem, Vol. 10, Issue 14
Investigating materials formation with liquid-phase and cryogenic TEM
journal, June 2016
- De Yoreo, J. J.; N. A. J. M., Sommerdijk
- Nature Reviews Materials, Vol. 1, Issue 8
Methane Mapping with Future Satellite Imaging Spectrometers
journal, December 2019
- Ayasse, Alana K.; Dennison, Philip E.; Foote, Markus
- Remote Sensing, Vol. 11, Issue 24
Acquisition of STEM Images by Adaptive Compressive Sensing
journal, July 2017
- Xie, Weiyi; Feng, Qianli; Srinivasan, Ramprakash
- Microscopy and Microanalysis, Vol. 23, Issue S1
Implementing an accurate and rapid sparse sampling approach for low-dose atomic resolution STEM imaging
journal, October 2016
- Kovarik, L.; Stevens, A.; Liyu, A.
- Applied Physics Letters, Vol. 109, Issue 16
A sub-sampled approach to extremely low-dose STEM
journal, January 2018
- Stevens, A.; Luzi, L.; Yang, H.
- Applied Physics Letters, Vol. 112, Issue 4
Median Filter Based Compressed Sensing Model with Application to MR Image Reconstruction
journal, September 2018
- Yang, Yunyun; Qin, Xuxu; Wu, Boying
- Mathematical Problems in Engineering, Vol. 2018
Detecting structural variances of Co3O4 catalysts by controlling beam-induced sample alterations in the vacuum of a transmission electron microscope
journal, November 2016
- Kisielowski, C.; Frei, H.; Specht, P.
- Advanced Structural and Chemical Imaging, Vol. 2, Issue 1