Big Data Analytics for Scanning Transmission Electron Microscopy Ptychography
Abstract
Electron microscopy is undergoing a transition; from the model of producing only a few micrographs, through the current state where many images and spectra can be digitally recorded, to a new mode where very large volumes of data (movies, ptychographic and multi-dimensional series) can be rapidly obtained. In this paper, we discuss the application of so-called “big-data” methods to high dimensional microscopy data, using unsupervised multivariate statistical techniques, in order to explore salient image features in a specific example of BiFeO3 domains. Remarkably, k-means clustering reveals domain differentiation despite the fact that the algorithm is purely statistical in nature and does not require any prior information regarding the material, any coexisting phases, or any differentiating structures. While this is a somewhat trivial case, this example signifies the extraction of useful physical and structural information without any prior bias regarding the sample or the instrumental modality. Further interpretation of these types of results may still require human intervention. Finally, however, the open nature of this algorithm and its wide availability, enable broad collaborations and exploratory work necessary to enable efficient data analysis in electron microscopy.
- Authors:
-
- Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Inst. for Functional Imaging of Materials; Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Center for Nanophase Materials Sciences
- Florida State Univ., Tallahassee, FL (United States)
- Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Inst. for Functional Imaging of Materials; Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Materials Sciences and Technology Division
- Publication Date:
- Research Org.:
- Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC), Basic Energy Sciences (BES). Materials Sciences & Engineering Division; USDOE Laboratory Directed Research and Development (LDRD) Program
- OSTI Identifier:
- 1261280
- Grant/Contract Number:
- AC05-00OR22725
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Scientific Reports
- Additional Journal Information:
- Journal Volume: 6; Journal ID: ISSN 2045-2322
- Publisher:
- Nature Publishing Group
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 36 MATERIALS SCIENCE; imaging techniques; transmission electron microscopy
Citation Formats
Jesse, S., Chi, M., Belianinov, A., Beekman, C., Kalinin, S. V., Borisevich, A. Y., and Lupini, A. R. Big Data Analytics for Scanning Transmission Electron Microscopy Ptychography. United States: N. p., 2016.
Web. doi:10.1038/srep26348.
Jesse, S., Chi, M., Belianinov, A., Beekman, C., Kalinin, S. V., Borisevich, A. Y., & Lupini, A. R. Big Data Analytics for Scanning Transmission Electron Microscopy Ptychography. United States. https://doi.org/10.1038/srep26348
Jesse, S., Chi, M., Belianinov, A., Beekman, C., Kalinin, S. V., Borisevich, A. Y., and Lupini, A. R. Mon .
"Big Data Analytics for Scanning Transmission Electron Microscopy Ptychography". United States. https://doi.org/10.1038/srep26348. https://www.osti.gov/servlets/purl/1261280.
@article{osti_1261280,
title = {Big Data Analytics for Scanning Transmission Electron Microscopy Ptychography},
author = {Jesse, S. and Chi, M. and Belianinov, A. and Beekman, C. and Kalinin, S. V. and Borisevich, A. Y. and Lupini, A. R.},
abstractNote = {Electron microscopy is undergoing a transition; from the model of producing only a few micrographs, through the current state where many images and spectra can be digitally recorded, to a new mode where very large volumes of data (movies, ptychographic and multi-dimensional series) can be rapidly obtained. In this paper, we discuss the application of so-called “big-data” methods to high dimensional microscopy data, using unsupervised multivariate statistical techniques, in order to explore salient image features in a specific example of BiFeO3 domains. Remarkably, k-means clustering reveals domain differentiation despite the fact that the algorithm is purely statistical in nature and does not require any prior information regarding the material, any coexisting phases, or any differentiating structures. While this is a somewhat trivial case, this example signifies the extraction of useful physical and structural information without any prior bias regarding the sample or the instrumental modality. Further interpretation of these types of results may still require human intervention. Finally, however, the open nature of this algorithm and its wide availability, enable broad collaborations and exploratory work necessary to enable efficient data analysis in electron microscopy.},
doi = {10.1038/srep26348},
journal = {Scientific Reports},
number = ,
volume = 6,
place = {United States},
year = {Mon May 23 00:00:00 EDT 2016},
month = {Mon May 23 00:00:00 EDT 2016}
}
Web of Science
Works referenced in this record:
Mapping Octahedral Tilts and Polarization Across a Domain Wall in BiFeO 3 from Z-Contrast Scanning Transmission Electron Microscopy Image Atomic Column Shape Analysis
journal, October 2010
- Borisevich, AlbinaY.; Ovchinnikov, Oleg S.; Chang, Hye Jung
- ACS Nano, Vol. 4, Issue 10
Research Update: Spatially resolved mapping of electronic structure on atomic level by multivariate statistical analysis
journal, December 2014
- Belianinov, Alex; Ganesh, Panchapakesan; Lin, Wenzhi
- APL Materials, Vol. 2, Issue 12
Multivariate statistical methods for the analysis of microscope image series: applications in materials science
journal, April 1998
- Bonnet, N.
- Journal of Microscopy, Vol. 190, Issue 1‐2
Ptychographic microscope for three-dimensional imaging
journal, January 2014
- Godden, T. M.; Suman, R.; Humphry, M. J.
- Optics Express, Vol. 22, Issue 10
Principal component and spatial correlation analysis of spectroscopic-imaging data in scanning probe microscopy
journal, February 2009
- Jesse, Stephen; Kalinin, Sergei V.
- Nanotechnology, Vol. 20, Issue 8
A Strain-Driven Morphotropic Phase Boundary in BiFeO3
journal, November 2009
- Zeches, R. J.; Rossell, M. D.; Zhang, J. X.
- Science, Vol. 326, Issue 5955, p. 977-980
Aberration measurement using the Ronchigram contrast transfer function
journal, June 2010
- Lupini, A. R.; Wang, P.; Nellist, P. D.
- Ultramicroscopy, Vol. 110, Issue 7
Mapping internal structure of coal by confocal micro-Raman spectroscopy and scanning microwave microscopy
journal, June 2014
- Tselev, Alexander; Ivanov, Ilia N.; Lavrik, Nickolay V.
- Fuel, Vol. 126
Ptychography and Related Diffractive Imaging Methods
book, May 2008
- Rodenburg, J. M.
- Advances in Imaging and Electron Physics
Interplay of Octahedral Tilts and Polar Order in BiFeO 3 Films
journal, March 2013
- Kim, Young-Min; Kumar, Amit; Hatt, Alison
- Advanced Materials, Vol. 25, Issue 17
Ptychographic electron microscopy using high-angle dark-field scattering for sub-nanometre resolution imaging
journal, January 2012
- Humphry, M. J.; Kraus, B.; Hurst, A. C.
- Nature Communications, Vol. 3, Issue 1
STEM imaging of 47-pm-separated atomic columns by a spherical aberration-corrected electron microscope with a 300-kV cold field emission gun
journal, June 2009
- Sawada, H.; Tanishiro, Y.; Ohashi, N.
- Journal of Electron Microscopy, Vol. 58, Issue 6
Big–deep–smart data in imaging for guiding materials design
journal, September 2015
- Kalinin, Sergei V.; Sumpter, Bobby G.; Archibald, Richard K.
- Nature Materials, Vol. 14, Issue 10
Mechanisms for microstructure enhancement in flux-assisted growth of barium titanate on sapphire
journal, March 2014
- Burch, Matthew J.; Li, Jing; Harris, David T.
- Journal of Materials Research, Vol. 29, Issue 7
Suppression of Octahedral Tilts and Associated Changes in Electronic Properties at Epitaxial Oxide Heterostructure Interfaces
journal, August 2010
- Borisevich, A. Y.; Chang, H. J.; Huijben, M.
- Physical Review Letters, Vol. 105, Issue 8
Complete information acquisition in dynamic force microscopy
journal, March 2015
- Belianinov, Alexei; Kalinin, Sergei V.; Jesse, Stephen
- Nature Communications, Vol. 6, Issue 1
Letter to the Editor: Limitations to the Measurement of Oxygen Concentrations by HRTEM Imposed by Surface Roughness
journal, March 2005
- Lupini, Andrew R.; Chisholm, Matthew F.; van Benthem, Klaus
- Microscopy and Microanalysis, Vol. 11, Issue 2
Direct observation of ferroelectric field effect and vacancy-controlled screening at the BiFeO3/LaxSr1−xMnO3 interface
journal, August 2014
- Kim, Young-Min; Morozovska, Anna; Eliseev, Eugene
- Nature Materials, Vol. 13, Issue 11
Constraining Data Mining with Physical Models: Voltage- and Oxygen Pressure-Dependent Transport in Multiferroic Nanostructures
journal, August 2015
- Strelcov, Evgheni; Belianinov, Alexei; Hsieh, Ying-Hui
- Nano Letters, Vol. 15, Issue 10
Atomic electric fields revealed by a quantum mechanical approach to electron picodiffraction
journal, December 2014
- Müller, Knut; Krause, Florian F.; Béché, Armand
- Nature Communications, Vol. 5, Issue 1
Combined Scanning Transmission Electron Microscopy Tilt- and Focal Series
journal, February 2014
- Dahmen, Tim; Baudoin, Jean-Pierre; Lupini, Andrew R.
- Microscopy and Microanalysis, Vol. 20, Issue 2
Direct Observation of Continuous Electric Dipole Rotation in Flux-Closure Domains in Ferroelectric Pb(Zr,Ti)O3
journal, March 2011
- Jia, C. -L.; Urban, K. W.; Alexe, M.
- Science, Vol. 331, Issue 6023
Spontaneous Vortex Nanodomain Arrays at Ferroelectric Heterointerfaces
journal, February 2011
- Nelson, Christopher T.; Winchester, Benjamin; Zhang, Yi
- Nano Letters, Vol. 11, Issue 2
Artificial intelligence and pattern recognition techniques in microscope image processing and analysis
book, January 2000
- Bonnet, Noël
- Advances in Imaging and Electron Physics
Oxygen octahedron reconstruction in the heterointerfaces investigated using aberration-corrected ultrahigh-resolution transmission electron microscopy
journal, February 2009
- Jia, C. L.; Mi, S. B.; Faley, M.
- Physical Review B, Vol. 79, Issue 8
Graphene engineering by neon ion beams
journal, February 2016
- Iberi, Vighter; Ievlev, Anton V.; Vlassiouk, Ivan
- Nanotechnology, Vol. 27, Issue 12
Prominent electrochromism through vacancy-order melting in a complex oxide
journal, January 2012
- Seidel, J.; Luo, W.; Suresha, S. J.
- Nature Communications, Vol. 3, Issue 1
3D electron microscopy in the physical sciences: the development of Z-contrast and EFTEM tomography
journal, September 2003
- Midgley, P. A.; Weyland, M.
- Ultramicroscopy, Vol. 96, Issue 3-4
Picometre-precision analysis of scanning transmission electron microscopy images of platinum nanocatalysts
journal, June 2014
- Yankovich, Andrew B.; Berkels, Benjamin; Dahmen, W.
- Nature Communications, Vol. 5, Issue 1
Efficient phase contrast imaging in STEM using a pixelated detector. Part 1: Experimental demonstration at atomic resolution
journal, April 2015
- Pennycook, Timothy J.; Lupini, Andrew R.; Yang, Hao
- Ultramicroscopy, Vol. 151, p. 160-167
Mapping chemical and bonding information using multivariate analysis of electron energy-loss spectrum images
journal, October 2006
- Bosman, M.; Watanabe, M.; Alexander, D. T. L.
- Ultramicroscopy, Vol. 106, Issue 11-12, p. 1024-1032
Phase Transitions, Phase Coexistence, and Piezoelectric Switching Behavior in Highly Strained BiFeO 3 Films
journal, July 2013
- Beekman, C.; Siemons, W.; Ward, T. Z.
- Advanced Materials, Vol. 25, Issue 39
Full information acquisition in piezoresponse force microscopy
journal, December 2015
- Somnath, Suhas; Belianinov, Alexei; Kalinin, Sergei V.
- Applied Physics Letters, Vol. 107, Issue 26
Deep Data Analysis of Conductive Phenomena on Complex Oxide Interfaces: Physics from Data Mining
journal, May 2014
- Strelcov, Evgheni; Belianinov, Alexei; Hsieh, Ying-Hui
- ACS Nano, Vol. 8, Issue 6
Comment on "Single Crystals of Single-Walled Carbon Nanotubes Formed by Self-Assembly"
journal, May 2003
- Chisholm, M. F.
- Science, Vol. 300, Issue 5623
Algorithm AS 136: A K-Means Clustering Algorithm
journal, January 1979
- Hartigan, J. A.; Wong, M. A.
- Applied Statistics, Vol. 28, Issue 1
Control of Octahedral Tilts and Magnetic Properties of Perovskite Oxide Heterostructures by Substrate Symmetry
journal, November 2010
- He, Jun; Borisevich, Albina; Kalinin, Sergei V.
- Physical Review Letters, Vol. 105, Issue 22
Identification of phases, symmetries and defects through local crystallography
journal, July 2015
- Belianinov, Alex; He, Qian; Kravchenko, Mikhail
- Nature Communications, Vol. 6, Issue 1
Big data in reciprocal space: Sliding fast Fourier transforms for determining periodicity
journal, March 2015
- Vasudevan, Rama K.; Belianinov, Alex; Gianfrancesco, Anthony G.
- Applied Physics Letters, Vol. 106, Issue 9
Multifrequency spectrum analysis using fully digital G Mode-Kelvin probe force microscopy
journal, February 2016
- Collins, Liam; Belianinov, Alex; Somnath, Suhas
- Nanotechnology, Vol. 27, Issue 10
Stress-induced symmetry changes in BiFeO films
journal, April 2011
- Christen, H. M.; Nam, J. H.; Kim, H. S.
- Physical Review B, Vol. 83, Issue 14
Unit-cell scale mapping of ferroelectricity and tetragonality in epitaxial ultrathin ferroelectric films
journal, December 2006
- Jia, Chun-Lin; Nagarajan, Valanoor; He, Jia-Qing
- Nature Materials, Vol. 6, Issue 1
Big-Data Reflection High Energy Electron Diffraction Analysis for Understanding Epitaxial Film Growth Processes
journal, October 2014
- Vasudevan, Rama K.; Tselev, Alexander; Baddorf, Arthur P.
- ACS Nano, Vol. 8, Issue 10
Atomically Resolved Mapping of Polarization and Electric Fields Across Ferroelectric/Oxide Interfaces by Z-contrast Imaging
journal, April 2011
- Chang, Hye Jung; Kalinin, Sergei V.; Morozovska, Anna N.
- Advanced Materials, Vol. 23, Issue 21
Blind analysis: Hide results to seek the truth
journal, October 2015
- MacCoun, Robert; Perlmutter, Saul
- Nature, Vol. 526, Issue 7572
Detection of Single Atoms and Buried Defects in Three Dimensions by Aberration-Corrected Electron Microscope with 0.5-Å Information Limit
journal, September 2008
- Kisielowski, C.; Freitag, B.; Bischoff, M.
- Microscopy and Microanalysis, Vol. 14, Issue 5
Probing oxygen vacancy concentration and homogeneity in solid-oxide fuel-cell cathode materials on the subunit-cell level
journal, August 2012
- Kim, Young-Min; He, Jun; Biegalski, Michael D.
- Nature Materials, Vol. 11, Issue 10
Interplay of Octahedral Tilts and Polar Order in BiFeO 3 Films
journal, March 2013
- Kim, Young-Min; Kumar, Amit; Hatt, Alison
- Advanced Materials, Vol. 25, Issue 17
Phase Transitions, Phase Coexistence, and Piezoelectric Switching Behavior in Highly Strained BiFeO 3 Films
journal, July 2013
- Beekman, C.; Siemons, W.; Ward, T. Z.
- Advanced Materials, Vol. 25, Issue 39
Mapping chemical and bonding information using multivariate analysis of electron energy-loss spectrum images
journal, October 2006
- Bosman, M.; Watanabe, M.; Alexander, D. T. L.
- Ultramicroscopy, Vol. 106, Issue 11-12, p. 1024-1032
Aberration measurement using the Ronchigram contrast transfer function
journal, June 2010
- Lupini, A. R.; Wang, P.; Nellist, P. D.
- Ultramicroscopy, Vol. 110, Issue 7
Efficient phase contrast imaging in STEM using a pixelated detector. Part 1: Experimental demonstration at atomic resolution
journal, April 2015
- Pennycook, Timothy J.; Lupini, Andrew R.; Yang, Hao
- Ultramicroscopy, Vol. 151, p. 160-167
Deep Data Analysis of Conductive Phenomena on Complex Oxide Interfaces: Physics from Data Mining
journal, May 2014
- Strelcov, Evgheni; Belianinov, Alexei; Hsieh, Ying-Hui
- ACS Nano, Vol. 8, Issue 6
Blind analysis: Hide results to seek the truth
journal, October 2015
- MacCoun, Robert; Perlmutter, Saul
- Nature, Vol. 526, Issue 7572
Ptychographic electron microscopy using high-angle dark-field scattering for sub-nanometre resolution imaging
journal, January 2012
- Humphry, M. J.; Kraus, B.; Hurst, A. C.
- Nature Communications, Vol. 3, Issue 1
Picometre-precision analysis of scanning transmission electron microscopy images of platinum nanocatalysts
journal, June 2014
- Yankovich, Andrew B.; Berkels, Benjamin; Dahmen, W.
- Nature Communications, Vol. 5, Issue 1
Unit-cell scale mapping of ferroelectricity and tetragonality in epitaxial ultrathin ferroelectric films
journal, December 2006
- Jia, Chun-Lin; Nagarajan, Valanoor; He, Jia-Qing
- Nature Materials, Vol. 6, Issue 1
Probing oxygen vacancy concentration and homogeneity in solid-oxide fuel-cell cathode materials on the subunit-cell level
journal, August 2012
- Kim, Young-Min; He, Jun; Biegalski, Michael D.
- Nature Materials, Vol. 11, Issue 10
Big–deep–smart data in imaging for guiding materials design
journal, September 2015
- Kalinin, Sergei V.; Sumpter, Bobby G.; Archibald, Richard K.
- Nature Materials, Vol. 14, Issue 10
Multivariate statistical methods for the analysis of microscope image series: applications in materials science
journal, April 1998
- Bonnet, N.
- Journal of Microscopy, Vol. 190, Issue 1‐2
Research Update: Spatially resolved mapping of electronic structure on atomic level by multivariate statistical analysis
journal, December 2014
- Belianinov, Alex; Ganesh, Panchapakesan; Lin, Wenzhi
- APL Materials, Vol. 2, Issue 12
Multifrequency spectrum analysis using fully digital G Mode-Kelvin probe force microscopy
journal, February 2016
- Collins, Liam; Belianinov, Alex; Somnath, Suhas
- Nanotechnology, Vol. 27, Issue 10
Graphene engineering by neon ion beams
journal, February 2016
- Iberi, Vighter; Ievlev, Anton V.; Vlassiouk, Ivan
- Nanotechnology, Vol. 27, Issue 12
Single Crystals of Single-Walled Carbon Nanotubes Formed by Self-Assembly
journal, April 2001
- Schlittler, R. R.
- Science, Vol. 292, Issue 5519
A Strain-Driven Morphotropic Phase Boundary in BiFeO3
journal, November 2009
- Zeches, R. J.; Rossell, M. D.; Zhang, J. X.
- Science, Vol. 326, Issue 5955, p. 977-980
Direct Observation of Continuous Electric Dipole Rotation in Flux-Closure Domains in Ferroelectric Pb(Zr,Ti)O3
journal, March 2011
- Jia, C. -L.; Urban, K. W.; Alexe, M.
- Science, Vol. 331, Issue 6023
Erratum: Big data and deep data in scanning and electron microscopies: deriving functionality from multidimensional data sets
journal, August 2015
- Belianinov, Alex; Vasudevan, Rama; Strelcov, Evgheni
- Advanced Structural and Chemical Imaging, Vol. 1, Issue 1
Algorithm AS 136: A K-Means Clustering Algorithm
journal, January 1979
- Hartigan, J. A.; Wong, M. A.
- Applied Statistics, Vol. 28, Issue 1
Works referencing / citing this record:
LiberTEM/LiberTEM: 0.4.1
software, February 2020
- Clausen, Alexander; Weber, Dieter; Ruzaeva, Karina
- Zenodo
Direct and continuous strain control of catalysts with tunable battery electrode materials
journal, November 2016
- Wang, Haotian; Xu, Shicheng; Tsai, Charlie
- Science, Vol. 354, Issue 6315
LiberTEM/LiberTEM: 0.3.0
software, December 2019
- Clausen, Alexander; Weber, Dieter; Ruzaeva, Karina
- Zenodo
Frontiers in strain-engineered multifunctional ferroic materials
journal, August 2016
- Agar, Joshua C.; Pandya, Shishir; Xu, Ruijuan
- MRS Communications, Vol. 6, Issue 3
LiberTEM/LiberTEM: 0.2.0
software, October 2019
- Clausen, Alexander; Weber, Dieter; Caron, Jan
- Zenodo
LiberTEM/LiberTEM-blobfinder: 0.4.0
software, February 2020
- Clausen, Alexander; Weber, Dieter; Ruzaeva, Karina
- Zenodo
Automated defect analysis in electron microscopic images
journal, July 2018
- Li, Wei; Field, Kevin G.; Morgan, Dane
- npj Computational Materials, Vol. 4, Issue 1
LiberTEM/LiberTEM-blobfinder: 0.4.0
software, February 2020
- Clausen, Alexander; Weber, Dieter; Ruzaeva, Karina
- Zenodo
Machine Detection of Enhanced Electromechanical Energy Conversion in PbZr 0.2 Ti 0.8 O 3 Thin Films
journal, May 2018
- Agar, Joshua C.; Cao, Ye; Naul, Brett
- Advanced Materials, Vol. 30, Issue 28
LiberTEM/LiberTEM: 0.2.2
software, October 2019
- Clausen, Alexander; Weber, Dieter; Caron, Jan
- Zenodo
Machine learning-based multidomain processing for texture-based image segmentation and analysis
journal, January 2020
- Borodinov, Nikolay; Tsai, Wan-Yu; Korolkov, Vladimir V.
- Applied Physics Letters, Vol. 116, Issue 4
Manifold learning of four-dimensional scanning transmission electron microscopy
journal, January 2019
- Li, Xin; Dyck, Ondrej E.; Oxley, Mark P.
- npj Computational Materials, Vol. 5, Issue 1
Functional Materials Under Stress: In Situ TEM Observations of Structural Evolution
journal, November 2019
- Deng, Yu; Zhang, Ruopeng; Pekin, Thomas C.
- Advanced Materials
Ptychography 4.0: 0.1.0
software, January 2021
- Weber, Dieter; Lesnichaia, Anastasiia; Strauch, Achim
LiberTEM/LiberTEM: 0.2.1
software, October 2019
- Clausen, Alexander; Weber, Dieter; Caron, Jan
- Zenodo
LiberTEM/LiberTEM: 0.4.0
software, February 2020
- Clausen, Alexander; Weber, Dieter; Ruzaeva, Karina
- Zenodo
Smart machine learning or discovering meaningful physical and chemical contributions through dimensional stacking
journal, August 2019
- Griffin, Lee A.; Gaponenko, Iaroslav; Zhang, Shujun
- npj Computational Materials, Vol. 5, Issue 1
Extensive deep neural networks for transferring small scale learning to large scale systems
text, January 2017
- Mills, Kyle; Ryczko, Kevin; Luchak, Iryna
- arXiv
LiberTEM/LiberTEM: 0.5.1
software, August 2020
- Clausen, Alexander; Weber, Dieter; Ruzaeva, Karina
- Zenodo
LiberTEM/LiberTEM: 0.5.0
software, April 2020
- Clausen, Alexander; Weber, Dieter; Ruzaeva, Karina
- Zenodo
Correcting the linear and nonlinear distortions for atomically resolved STEM spectrum and diffraction imaging
journal, January 2018
- Wang, Yi; Suyolcu, Y. Eren; Salzberger, Ute
- Microscopy, Vol. 67, Issue suppl_1
Nonuniform Sparse Data Clustering Cascade Algorithm Based on Dynamic Cumulative Entropy
journal, January 2016
- Li, Ning; Gu, Yunxia; Deng, Zhongliang
- Mathematical Problems in Engineering, Vol. 2016