skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Applying compressive sensing to TEM video: A substantial frame rate increase on any camera

Abstract

One of the main limitations of imaging at high spatial and temporal resolution during in-situ transmission electron microscopy (TEM) experiments is the frame rate of the camera being used to image the dynamic process. While the recent development of direct detectors has provided the hardware to achieve frame rates approaching 0.1 ms, the cameras are expensive and must replace existing detectors. In this paper, we examine the use of coded aperture compressive sensing (CS) methods to increase the frame rate of any camera with simple, low-cost hardware modifications. The coded aperture approach allows multiple sub-frames to be coded and integrated into a single camera frame during the acquisition process, and then extracted upon readout using statistical CS inversion. Here we describe the background of CS and statistical methods in depth and simulate the frame rates and efficiencies for in-situ TEM experiments. Depending on the resolution and signal/noise of the image, it should be possible to increase the speed of any camera by more than an order of magnitude using this approach.

Authors:
ORCiD logo [1];  [2];  [2];  [3];  [3];  [2]
  1. Pacific Northwest National Lab. (PNNL), Richland, WA (United States); Duke Univ., Durham, NC (United States)
  2. Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
  3. Duke Univ., Durham, NC (United States)
Publication Date:
Research Org.:
Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Biological and Environmental Research (BER) (SC-23)
OSTI Identifier:
1214819
Grant/Contract Number:  
AC05-76RL01830
Resource Type:
Accepted Manuscript
Journal Name:
Advanced Structural and Chemical Imaging
Additional Journal Information:
Journal Volume: 1; Journal Issue: 1; Journal ID: ISSN 2198-0926
Publisher:
Springer
Country of Publication:
United States
Language:
English
Subject:
36 MATERIALS SCIENCE; compressive sensing; transmission electron microscopy; video; coded aperture; nanoparticles; chemical dynamics

Citation Formats

Stevens, Andrew, Kovarik, Libor, Abellan, Patricia, Yuan, Xin, Carin, Lawrence, and Browning, Nigel D. Applying compressive sensing to TEM video: A substantial frame rate increase on any camera. United States: N. p., 2015. Web. doi:10.1186/s40679-015-0009-3.
Stevens, Andrew, Kovarik, Libor, Abellan, Patricia, Yuan, Xin, Carin, Lawrence, & Browning, Nigel D. Applying compressive sensing to TEM video: A substantial frame rate increase on any camera. United States. doi:10.1186/s40679-015-0009-3.
Stevens, Andrew, Kovarik, Libor, Abellan, Patricia, Yuan, Xin, Carin, Lawrence, and Browning, Nigel D. Thu . "Applying compressive sensing to TEM video: A substantial frame rate increase on any camera". United States. doi:10.1186/s40679-015-0009-3. https://www.osti.gov/servlets/purl/1214819.
@article{osti_1214819,
title = {Applying compressive sensing to TEM video: A substantial frame rate increase on any camera},
author = {Stevens, Andrew and Kovarik, Libor and Abellan, Patricia and Yuan, Xin and Carin, Lawrence and Browning, Nigel D.},
abstractNote = {One of the main limitations of imaging at high spatial and temporal resolution during in-situ transmission electron microscopy (TEM) experiments is the frame rate of the camera being used to image the dynamic process. While the recent development of direct detectors has provided the hardware to achieve frame rates approaching 0.1 ms, the cameras are expensive and must replace existing detectors. In this paper, we examine the use of coded aperture compressive sensing (CS) methods to increase the frame rate of any camera with simple, low-cost hardware modifications. The coded aperture approach allows multiple sub-frames to be coded and integrated into a single camera frame during the acquisition process, and then extracted upon readout using statistical CS inversion. Here we describe the background of CS and statistical methods in depth and simulate the frame rates and efficiencies for in-situ TEM experiments. Depending on the resolution and signal/noise of the image, it should be possible to increase the speed of any camera by more than an order of magnitude using this approach.},
doi = {10.1186/s40679-015-0009-3},
journal = {Advanced Structural and Chemical Imaging},
number = 1,
volume = 1,
place = {United States},
year = {2015},
month = {8}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Save / Share:

Works referenced in this record:

In Situ Transmission Electron Microscopy
journal, February 2008

  • Ferreira, P. J.; Mitsuishi, K.; Stach, E. A.
  • MRS Bulletin, Vol. 33, Issue 02, p. 83-90
  • DOI: 10.1557/mrs2008.20

Advances in the environmental transmission electron microscope (ETEM) for nanoscale in situ studies of gas–solid interactions
journal, January 2014

  • Jinschek, J. R.
  • Chemical Communications, Vol. 50, Issue 21, p. 2696-2706
  • DOI: 10.1039/C3CC49092K

In Situ Observation of the Electrochemical Lithiation of a Single SnO2 Nanowire Electrode
journal, December 2010


Controlled Growth of Nanoparticles from Solution with In Situ Liquid Transmission Electron Microscopy
journal, July 2011

  • Evans, James E.; Jungjohann, Katherine L.; Browning, Nigel D.
  • Nano Letters, Vol. 11, Issue 7
  • DOI: 10.1021/nl201166k

Towards sub-Å electron beams
journal, June 1999


Towards 0.1 nm resolution with the first spherically corrected transmission electron microscope
journal, January 1998


Image resolution and sensitivity in an environmental transmission electron microscope
journal, November 2012


Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information
journal, February 2006

  • Candes, E.J.; Romberg, J.; Tao, T.
  • IEEE Transactions on Information Theory, Vol. 52, Issue 2, p. 489-509
  • DOI: 10.1109/TIT.2005.862083

Compressed sensing
journal, April 2006


The potential for Bayesian compressive sensing to significantly reduce electron dose in high-resolution STEM images
journal, October 2013


Generalized Alternating Projection for Weighted-$\ell_{2,1}$ Minimization with Applications to Model-Based Compressive Sensing
journal, January 2014

  • Liao, Xuejun; Li, Hui; Carin, Lawrence
  • SIAM Journal on Imaging Sciences, Vol. 7, Issue 2, p. 797-823
  • DOI: 10.1137/130936658

A New TwIST: Two-Step Iterative Shrinkage/Thresholding Algorithms for Image Restoration
journal, December 2007

  • Bioucas-Dias, J. M.; Figueiredo, M. A. T.
  • IEEE Transactions on Image Processing, Vol. 16, Issue 12
  • DOI: 10.1109/TIP.2007.909319

Compressive Sensing [Lecture Notes]
journal, August 2007


Adaptive temporal compressive sensing for video
conference, February 2013

  • Yuan, Xin; Yang, Jianbo; Llull, Patrick
  • 2013 IEEE International Conference on Image Processing, p. 14-18
  • DOI: 10.1109/ICIP.2013.6738004

Reduced-dose and high-speed acquisition strategies for multi-dimensional electron microscopy
journal, May 2015

  • Saghi, Zineb; Benning, Martin; Leary, Rowan
  • Advanced Structural and Chemical Imaging, Vol. 1, Issue 1
  • DOI: 10.1186/s40679-015-0007-5

Nonparametric Bayesian Dictionary Learning for Analysis of Noisy and Incomplete Images
journal, January 2012

  • Zhou, Mingyuan; Chen, Haojun; Paisley, John
  • IEEE Transactions on Image Processing, Vol. 21, Issue 1, p. 130-144
  • DOI: 10.1109/TIP.2011.2160072

Emergence of simple-cell receptive field properties by learning a sparse code for natural images
journal, June 1996

  • Olshausen, Bruno A.; Field, David J.
  • Nature, Vol. 381, Issue 6583
  • DOI: 10.1038/381607a0

$rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation
journal, November 2006

  • Aharon, M.; Elad, M.; Bruckstein, A.
  • IEEE Transactions on Signal Processing, Vol. 54, Issue 11
  • DOI: 10.1109/TSP.2006.881199

Electron tomography based on a total variation minimization reconstruction technique
journal, February 2012


Compressive Sensing on Manifolds Using a Nonparametric Mixture of Factor Analyzers: Algorithm and Performance Bounds
journal, December 2010

  • Chen, Minhua; Silva, Jorge; Paisley, John
  • IEEE Transactions on Signal Processing, Vol. 58, Issue 12, p. 6140-6155
  • DOI: 10.1109/TSP.2010.2070796

Face recognition using Laplacianfaces
journal, March 2005

  • Xiaofei He,
  • IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 27, Issue 3
  • DOI: 10.1109/TPAMI.2005.55

A tutorial on Bayesian nonparametric models
journal, February 2012

  • Gershman, Samuel J.; Blei, David M.
  • Journal of Mathematical Psychology, Vol. 56, Issue 1, p. 1-12
  • DOI: 10.1016/j.jmp.2011.08.004

Probabilistic Principal Component Analysis
journal, August 1999

  • Tipping, Michael E.; Bishop, Christopher M.
  • Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 61, Issue 3, p. 611-622
  • DOI: 10.1111/1467-9868.00196

Dictionary Learning for Noisy and Incomplete Hyperspectral Images
journal, January 2012

  • Xing, Zhengming; Zhou, Mingyuan; Castrodad, Alexey
  • SIAM Journal on Imaging Sciences, Vol. 5, Issue 1, p. 33-56
  • DOI: 10.1137/110837486

Mixtures of Probabilistic Principal Component Analyzers
journal, February 1999


Coded aperture compressive temporal imaging
journal, January 2013

  • Llull, Patrick; Liao, Xuejun; Yuan, Xin
  • Optics Express, Vol. 21, Issue 9, p. 10526-10545
  • DOI: 10.1364/OE.21.010526

Visualizing Gas Molecules Interacting with Supported Nanoparticulate Catalysts at Reaction Conditions
journal, January 2012


In situ Transmission Electron Microscopy of catalyst sintering
journal, December 2013

  • DeLaRiva, Andrew T.; Hansen, Thomas W.; Challa, Sivakumar R.
  • Journal of Catalysis, Vol. 308, p. 291-305
  • DOI: 10.1016/j.jcat.2013.08.018

Atomic-scale electron microscopy at ambient pressure
journal, August 2008


A (S)TEM Gas Cell Holder with Localized Laser Heating for In Situ Experiments
journal, March 2013

  • Mehraeen, Shareghe; McKeown, Joseph T.; Deshmukh, Pushkarraj V.
  • Microscopy and Microanalysis, Vol. 19, Issue 2
  • DOI: 10.1017/S1431927612014419

Analysis of Ni nanoparticle gas phase sintering
journal, January 2007

  • Tsyganov, Sergej; Kästner, Jochen; Rellinghaus, Bernd
  • Physical Review B, Vol. 75, Issue 4, Article No. 045421
  • DOI: 10.1103/PhysRevB.75.045421

Quantitative Measurement of the Surface Self-Diffusion on Au Nanoparticles by Aberration-Corrected Transmission Electron Microscopy
journal, February 2012

  • Surrey, A.; Pohl, D.; Schultz, L.
  • Nano Letters, Vol. 12, Issue 12, p. 6071-6077
  • DOI: 10.1021/nl302280x

The restricted isometry property and its implications for compressed sensing
journal, May 2008


    Works referencing / citing this record:

    Automating material image analysis for material discovery
    journal, April 2019