Search Menu
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information
Search Scholarly Publications

Title: Automated defect analysis in electron microscopic images

Abstract

Electron microscopy and defect analysis are a cornerstone of materials science, as they offer detailed insights on the microstructure and performance of a wide range of materials and material systems. Building a robust and flexible platform for automated defect recognition and classification in electron microscopy will result in the completion of analysis orders of magnitude faster after images are recorded, or even online during image acquisition. Automated analysis has the potential to be significantly more efficient, accurate, and repeatable than human analysis, and it can scale with the increasingly important methods of automated data generation. Herein, an automated recognition tool is developed based on a computer vison–based approach; it sequentially applies a cascade object detector, convolutional neural network, and local image analysis methods. We demonstrate that the automated tool performs as well as or better than manual human detection in terms of recall and precision and achieves quantitative image/defect analysis metrics close to the human average. The proposed approach works for images of varying contrast, brightness, and magnification. Furthermore, these promising results suggest that this and similar approaches are worth exploring for detecting multiple defect types and have the potential to locate, classify, and measure quantitative features for a rangemore » of defect types, materials, and electron microscopic techniques.« less

Authors:
ORCiD logo [1]; ORCiD logo [2]; ORCiD logo [1]
+ Show Author Affiliations
  1. Univ. of Wisconsin, Madison, WI (United States)
  2. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
1462882
Grant/Contract Number:  
AC05-00OR22725
Resource Type:
Accepted Manuscript
Journal Name:
npj Computational Materials
Additional Journal Information:
Journal Volume: 4; Journal Issue: 1; Journal ID: ISSN 2057-3960
Publisher:
Nature Publishing Group
Country of Publication:
United States
Language:
English
Subject:
47 OTHER INSTRUMENTATION

Citation Formats

Li, Wei, Field, Kevin G., and Morgan, Dane. Automated defect analysis in electron microscopic images. United States: N. p., 2018. Web. doi:10.1038/s41524-018-0093-8.
Copy to clipboard
Li, Wei, Field, Kevin G., & Morgan, Dane. Automated defect analysis in electron microscopic images. United States. https://doi.org/10.1038/s41524-018-0093-8
Copy to clipboard
Li, Wei, Field, Kevin G., and Morgan, Dane. Wed . "Automated defect analysis in electron microscopic images". United States. https://doi.org/10.1038/s41524-018-0093-8. https://www.osti.gov/servlets/purl/1462882.
Copy to clipboard
@article{osti_1462882,
title = {Automated defect analysis in electron microscopic images},
author = {Li, Wei and Field, Kevin G. and Morgan, Dane},
abstractNote = {Electron microscopy and defect analysis are a cornerstone of materials science, as they offer detailed insights on the microstructure and performance of a wide range of materials and material systems. Building a robust and flexible platform for automated defect recognition and classification in electron microscopy will result in the completion of analysis orders of magnitude faster after images are recorded, or even online during image acquisition. Automated analysis has the potential to be significantly more efficient, accurate, and repeatable than human analysis, and it can scale with the increasingly important methods of automated data generation. Herein, an automated recognition tool is developed based on a computer vison–based approach; it sequentially applies a cascade object detector, convolutional neural network, and local image analysis methods. We demonstrate that the automated tool performs as well as or better than manual human detection in terms of recall and precision and achieves quantitative image/defect analysis metrics close to the human average. The proposed approach works for images of varying contrast, brightness, and magnification. Furthermore, these promising results suggest that this and similar approaches are worth exploring for detecting multiple defect types and have the potential to locate, classify, and measure quantitative features for a range of defect types, materials, and electron microscopic techniques.},
doi = {10.1038/s41524-018-0093-8},
journal = {npj Computational Materials},
number = 1,
volume = 4,
place = {United States},
year = {Wed Jul 18 00:00:00 EDT 2018},
month = {Wed Jul 18 00:00:00 EDT 2018}
}
Copy to clipboard
Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
https://doi.org/10.1038/s41524-018-0093-8
Copyright Statement
Other availability
Search WorldCat Search WorldCat to find libraries that may hold this journal
Citation Metrics:
Cited by: 66 works
Citation information provided by
Web of Science
Save / Share:
Export Metadata
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.

Works referenced in this record:

Deep learning
journal, May 2015

  • LeCun, Yann; Bengio, Yoshua; Hinton, Geoffrey
  • Nature, Vol. 521, Issue 7553
  • DOI: 10.1038/nature14539

Face Description with Local Binary Patterns: Application to Face Recognition
journal, December 2006

  • Ahonen, T.; Hadid, A.; Pietikainen, M.
  • IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 28, Issue 12
  • DOI: 10.1109/TPAMI.2006.244

A practical implementation of face detection by using Matlab cascade object detector
conference, October 2015

  • Alionte, Elena; Lazar, Corneliu
  • 2015 19th International Conference on System Theory, Control and Computing (ICSTCC)
  • DOI: 10.1109/ICSTCC.2015.7321390

Image driven machine learning methods for microstructure recognition
journal, October 2016

Robust Real-Time Face Detection
journal, May 2004

Histograms of Oriented Gradients for Human Detection
conference, January 2005

  • Dalal, N.; Triggs, B.
  • 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05)
  • DOI: 10.1109/CVPR.2005.177

Exploring the microstructure manifold: Image texture representations applied to ultrahigh carbon steel microstructures
journal, July 2017

A general review of human face detection including a study of neural networks and Haar feature-based cascade classifier in face detection
conference, August 2014

  • Sharifara, Ali; Mohd Rahim, Mohd Shafry; Anisi, Yasaman
  • 2014 International Symposium on Biometrics and Security Technologies (ISBAST)
  • DOI: 10.1109/ISBAST.2014.7013097

Object recognition from local scale-invariant features
conference, January 1999

Rethinking the Inception Architecture for Computer Vision
conference, June 2016

  • Szegedy, Christian; Vanhoucke, Vincent; Ioffe, Sergey
  • 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • DOI: 10.1109/CVPR.2016.308

Characterization of irradiation defect structures and densities by transmission electron microscopy
journal, February 2015

  • Kirk, Marquis; Yi, Xiaoou; Jenkins, Michael
  • Journal of Materials Research, Vol. 30, Issue 9
  • DOI: 10.1557/jmr.2015.19

Fully convolutional networks for semantic segmentation
conference, June 2015

  • Long, Jonathan; Shelhamer, Evan; Darrell, Trevor
  • 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • DOI: 10.1109/CVPR.2015.7298965

Dislocation loop formation in model FeCrAl alloys after neutron irradiation below 1 dpa
journal, November 2017

Big Data Analytics for Scanning Transmission Electron Microscopy Ptychography
journal, May 2016

  • Jesse, S.; Chi, M.; Belianinov, A.
  • Scientific Reports, Vol. 6, Issue 1
  • DOI: 10.1038/srep26348

Inferring low-dimensional microstructure representations using convolutional neural networks
journal, November 2017

Watersheds in digital spaces: an efficient algorithm based on immersion simulations
journal, June 1991

  • Vincent, L.; Soille, P.
  • IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 13, Issue 6
  • DOI: 10.1109/34.87344

Structural materials for fission & fusion energy
journal, November 2009

A Method for Detection and Classification of Glass Defects in Low Resolution Images
conference, August 2011

  • Zhao, Jie; Kong, Qing-Jie; Zhao, Xu
  • Graphics (ICIG), 2011 Sixth International Conference on Image and Graphics
  • DOI: 10.1109/ICIG.2011.187

Shape matching and object recognition using shape contexts
journal, April 2002

  • Belongie, S.; Malik, J.; Puzicha, J.
  • IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 24, Issue 4
  • DOI: 10.1109/34.993558

TEM characterization of dislocation loops in irradiated bcc Fe-based steels
journal, March 2013

Automated Detection and Segmentation of Synaptic Contacts in Nearly Isotropic Serial Electron Microscopy Images
journal, October 2011

Diffraction contrast STEM of dislocations: Imaging and simulations
journal, August 2011

Face Detection with the Faster R-CNN
conference, May 2017

  • Jiang, Huaizu; Learned-Miller, Erik
  • 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017)
  • DOI: 10.1109/FG.2017.82

Object Detection with Discriminatively Trained Part-Based Models
journal, September 2010

  • Felzenszwalb, P. F.; Girshick, R. B.; McAllester, D.
  • IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 32, Issue 9
  • DOI: 10.1109/TPAMI.2009.167

Rapid object detection using a boosted cascade of simple features
conference, January 2001

  • Viola, P.; Jones, M.
  • Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001
  • DOI: 10.1109/CVPR.2001.990517

Heterogeneous dislocation loop formation near grain boundaries in a neutron-irradiated commercial FeCrAl alloy
journal, January 2017

Application of STEM characterization for investigating radiation effects in BCC Fe-based alloys
journal, April 2015

  • Parish, Chad M.; Field, Kevin G.; Certain, Alicia G.
  • Journal of Materials Research, Vol. 30, Issue 9
  • DOI: 10.1557/jmr.2015.32

Radiation tolerance of neutron-irradiated model Fe–Cr–Al alloys
journal, October 2015

Mask R-CNN
conference, October 2017

  • He, Kaiming; Gkioxari, Georgia; Dollar, Piotr
  • 2017 IEEE International Conference on Computer Vision (ICCV)
  • DOI: 10.1109/ICCV.2017.322

Big data and deep data in scanning and electron microscopies: deriving functionality from multidimensional data sets
journal, May 2015

  • Belianinov, Alex; Vasudevan, Rama; Strelcov, Evgheni
  • Advanced Structural and Chemical Imaging, Vol. 1, Issue 1
  • DOI: 10.1186/s40679-015-0006-6

Deep Learning
text, January 2018

Robust real-time face detection
conference, January 2001

  • Viola, P.; Jones, M.
  • Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001
  • DOI: 10.1109/iccv.2001.937709

Rethinking the Inception Architecture for Computer Vision
preprint, January 2015

Fully Convolutional Networks for Semantic Segmentation
preprint, January 2016

Detection of Open Loop Defects in STEM Images of Irradiation-Damaged Alloys – Source Code for Detection and Image Dataset
dataset, January 2018

  • Li, Wei; Field, Kevin G.; Morgan, Dane
  • Materials Data Facility
  • DOI: 10.18126/m2692z
    Sort by:
    [ × clear filter / sort ]

    Works referencing / citing this record:

    Interplay Between Grain Boundaries and Radiation Damage
    journal, March 2019

    Deep Learning for Semantic Segmentation of Defects in Advanced STEM Images of Steels
    journal, September 2019

    Machine-learning-assisted screening of pure-silica zeolites for effective removal of linear siloxanes and derivatives
    journal, January 2020

    • Lin, Shiru; Wang, Yekun; Zhao, Yinghe
    • Journal of Materials Chemistry A, Vol. 8, Issue 6
    • DOI: 10.1039/c9ta11909d

    Data-driven approach for synchrotron X-ray Laue microdiffraction scan analysis
    journal, October 2019

    • Song, Yintao; Tamura, Nobumichi; Zhang, Chenbo
    • Acta Crystallographica Section A Foundations and Advances, Vol. 75, Issue 6
    • DOI: 10.1107/s2053273319012804
      Sort by:
      [ × clear filter / sort ]

      Similar Records in DOE PAGES and OSTI.GOV collections:

      Other research related to this record: