Automated defect analysis in electron microscopic images

Li, Wei; Field, Kevin G.; Morgan, Dane

doi:10.1038/s41524-018-0093-8

Title: Automated defect analysis in electron microscopic images

Journal Article · 18 July 2018 · npj Computational Materials

DOI: https://doi.org/10.1038/s41524-018-0093-8 · OSTI ID:1462882

^[1];

^[2];

^[1]

Univ. of Wisconsin, Madison, WI (United States)
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)

Electron microscopy and defect analysis are a cornerstone of materials science, as they offer detailed insights on the microstructure and performance of a wide range of materials and material systems. Building a robust and flexible platform for automated defect recognition and classification in electron microscopy will result in the completion of analysis orders of magnitude faster after images are recorded, or even online during image acquisition. Automated analysis has the potential to be significantly more efficient, accurate, and repeatable than human analysis, and it can scale with the increasingly important methods of automated data generation. Herein, an automated recognition tool is developed based on a computer vison–based approach; it sequentially applies a cascade object detector, convolutional neural network, and local image analysis methods. We demonstrate that the automated tool performs as well as or better than manual human detection in terms of recall and precision and achieves quantitative image/defect analysis metrics close to the human average. The proposed approach works for images of varying contrast, brightness, and magnification. Furthermore, these promising results suggest that this and similar approaches are worth exploring for detecting multiple defect types and have the potential to locate, classify, and measure quantitative features for a range of defect types, materials, and electron microscopic techniques.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Organization:: USDOE

Grant/Contract Number:: AC05-00OR22725

OSTI ID:: 1462882

Journal Information:: npj Computational Materials, Vol. 4, Issue 1; ISSN 2057-3960

Publisher:: Nature Publishing GroupCopyright Statement

Country of Publication:: United States

Language:: English

Citation Metrics:

Cited by: 76 works

Citation information provided by
Web of Science

References (39)

Deep learning LeCun, Yann; Bengio, Yoshua; Hinton, Geoffrey Nature, Vol. 521, Issue 7553 https://doi.org/10.1038/nature14539	journal	May 2015
Face Description with Local Binary Patterns: Application to Face Recognition Ahonen, T.; Hadid, A.; Pietikainen, M. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 28, Issue 12 https://doi.org/10.1109/TPAMI.2006.244	journal	December 2006
Deep Learning of Atomically Resolved Scanning Transmission Electron Microscopy Images: Chemical Identification and Tracking Local Transformations Ziatdinov, Maxim; Dyck, Ondrej; Maksov, Artem ACS Nano, Vol. 11, Issue 12 https://doi.org/10.1021/acsnano.7b07504	journal	October 2016
A practical implementation of face detection by using Matlab cascade object detector Alionte, Elena; Lazar, Corneliu 2015 19th International Conference on System Theory, Control and Computing (ICSTCC) https://doi.org/10.1109/ICSTCC.2015.7321390	conference	October 2015
Image driven machine learning methods for microstructure recognition Chowdhury, Aritra; Kautz, Elizabeth; Yener, Bülent Computational Materials Science, Vol. 123 https://doi.org/10.1016/j.commatsci.2016.05.034	journal	October 2016
Robust Real-Time Face Detection Viola, Paul; Jones, Michael J. International Journal of Computer Vision, Vol. 57, Issue 2 https://doi.org/10.1023/B:VISI.0000013087.49260.fb	journal	May 2004
Histograms of Oriented Gradients for Human Detection Dalal, N.; Triggs, B. 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) https://doi.org/10.1109/CVPR.2005.177	conference	January 2005
Exploring the microstructure manifold: Image texture representations applied to ultrahigh carbon steel microstructures DeCost, Brian L.; Francis, Toby; Holm, Elizabeth A. Acta Materialia, Vol. 133 https://doi.org/10.1016/j.actamat.2017.05.014	journal	July 2017
A general review of human face detection including a study of neural networks and Haar feature-based cascade classifier in face detection Sharifara, Ali; Mohd Rahim, Mohd Shafry; Anisi, Yasaman 2014 International Symposium on Biometrics and Security Technologies (ISBAST) https://doi.org/10.1109/ISBAST.2014.7013097	conference	August 2014
Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks Cireşan, Dan C.; Giusti, Alessandro; Gambardella, Luca M. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2013 https://doi.org/10.1007/978-3-642-40763-5_51	book	January 2013
Rethinking the Inception Architecture for Computer Vision Szegedy, Christian; Vanhoucke, Vincent; Ioffe, Sergey 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2016.308	conference	June 2016
Characterization of irradiation defect structures and densities by transmission electron microscopy Kirk, Marquis; Yi, Xiaoou; Jenkins, Michael Journal of Materials Research, Vol. 30, Issue 9 https://doi.org/10.1557/jmr.2015.19	journal	February 2015
Fully convolutional networks for semantic segmentation Long, Jonathan; Shelhamer, Evan; Darrell, Trevor 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2015.7298965	conference	June 2015
Dislocation loop formation in model FeCrAl alloys after neutron irradiation below 1 dpa Field, Kevin G.; Briggs, Samuel A.; Sridharan, Kumar Journal of Nuclear Materials, Vol. 495 https://doi.org/10.1016/j.jnucmat.2017.07.061	journal	November 2017
Big Data Analytics for Scanning Transmission Electron Microscopy Ptychography Jesse, S.; Chi, M.; Belianinov, A. Scientific Reports, Vol. 6, Issue 1 https://doi.org/10.1038/srep26348	journal	May 2016
Inferring low-dimensional microstructure representations using convolutional neural networks Lubbers, Nicholas; Lookman, Turab; Barros, Kipton Physical Review E, Vol. 96, Issue 5 https://doi.org/10.1103/PhysRevE.96.052111	journal	November 2017
Watersheds in digital spaces: an efficient algorithm based on immersion simulations Vincent, L.; Soille, P. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 13, Issue 6 https://doi.org/10.1109/34.87344	journal	June 1991
Structural materials for fission & fusion energy Zinkle, Steven J.; Busby, Jeremy T. Materials Today, Vol. 12, Issue 11 https://doi.org/10.1016/S1369-7021(09)70294-9	journal	November 2009
A Method for Detection and Classification of Glass Defects in Low Resolution Images Zhao, Jie; Kong, Qing-Jie; Zhao, Xu Graphics (ICIG), 2011 Sixth International Conference on Image and Graphics https://doi.org/10.1109/ICIG.2011.187	conference	August 2011
Visualizing and Understanding Convolutional Networks Zeiler, Matthew D.; Fergus, Rob Computer Vision – ECCV 2014 https://doi.org/10.1007/978-3-319-10590-1_53	book	January 2014
Shape matching and object recognition using shape contexts Belongie, S.; Malik, J.; Puzicha, J. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 24, Issue 4 https://doi.org/10.1109/34.993558	journal	April 2002
TEM characterization of dislocation loops in irradiated bcc Fe-based steels Yao, B.; Edwards, D. J.; Kurtz, R. J. Journal of Nuclear Materials, Vol. 434, Issue 1-3 https://doi.org/10.1016/j.jnucmat.2012.12.002	journal	March 2013
Automated Detection and Segmentation of Synaptic Contacts in Nearly Isotropic Serial Electron Microscopy Images Kreshuk, Anna; Straehle, Christoph N.; Sommer, Christoph PLoS ONE, Vol. 6, Issue 10 https://doi.org/10.1371/journal.pone.0024899	journal	October 2011
Diffraction contrast STEM of dislocations: Imaging and simulations Phillips, P. J.; Brandes, M. C.; Mills, M. J. Ultramicroscopy, Vol. 111, Issue 9-10 https://doi.org/10.1016/j.ultramic.2011.07.001	journal	August 2011
Face Detection with the Faster R-CNN Jiang, Huaizu; Learned-Miller, Erik 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017) https://doi.org/10.1109/FG.2017.82	conference	May 2017
Object Detection with Discriminatively Trained Part-Based Models Felzenszwalb, P. F.; Girshick, R. B.; McAllester, D. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 32, Issue 9 https://doi.org/10.1109/TPAMI.2009.167	journal	September 2010
Rapid object detection using a boosted cascade of simple features Viola, P.; Jones, M. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001 https://doi.org/10.1109/CVPR.2001.990517	conference	January 2001
Heterogeneous dislocation loop formation near grain boundaries in a neutron-irradiated commercial FeCrAl alloy Field, Kevin G.; Briggs, Samuel A.; Hu, Xunxiang Journal of Nuclear Materials, Vol. 483 https://doi.org/10.1016/j.jnucmat.2016.10.050	journal	January 2017
Application of STEM characterization for investigating radiation effects in BCC Fe-based alloys Parish, Chad M.; Field, Kevin G.; Certain, Alicia G. Journal of Materials Research, Vol. 30, Issue 9 https://doi.org/10.1557/jmr.2015.32	journal	April 2015
Radiation tolerance of neutron-irradiated model Fe–Cr–Al alloys Field, Kevin G.; Hu, Xunxiang; Littrell, Kenneth C. Journal of Nuclear Materials, Vol. 465 https://doi.org/10.1016/j.jnucmat.2015.06.023	journal	October 2015
Big data and deep data in scanning and electron microscopies: deriving functionality from multidimensional data sets Belianinov, Alex; Vasudevan, Rama; Strelcov, Evgheni Advanced Structural and Chemical Imaging, Vol. 1, Issue 1 https://doi.org/10.1186/s40679-015-0006-6	journal	May 2015
A deep convolutional neural network to analyze position averaged convergent beam electron diffraction patterns Xu, W.; LeBeau, J. M. Ultramicroscopy, Vol. 188 https://doi.org/10.1016/j.ultramic.2018.03.004	journal	May 2018
Deep Learning Ganatra, Sanket Zenodo https://doi.org/10.5281/zenodo.1189359	text	January 2018
Robust real-time face detection Viola, P.; Jones, M. Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001 https://doi.org/10.1109/iccv.2001.937709	conference	January 2001
Rethinking the Inception Architecture for Computer Vision Szegedy, Christian; Vanhoucke, Vincent; Ioffe, Sergey arXiv https://doi.org/10.48550/arxiv.1512.00567	preprint	January 2015
Fully Convolutional Networks for Semantic Segmentation Shelhamer, Evan; Long, Jonathan; Darrell, Trevor arXiv https://doi.org/10.48550/arxiv.1605.06211	preprint	January 2016
A Deep Convolutional Neural Network to Analyze Position Averaged Convergent Beam Electron Diffraction Patterns Xu, Weizong; LeBeau, James M. arXiv https://doi.org/10.48550/arxiv.1708.00855	text	January 2017
Deep Learning of Atomically Resolved Scanning Transmission Electron Microscopy Images: Chemical Identification and Tracking Local Transformations Ziatdinov, Maxim; Dyck, Ondrej; Maksov, Artem arXiv https://doi.org/10.48550/arxiv.1801.05860	text	January 2018
Detection of Open Loop Defects in STEM Images of Irradiation-Damaged Alloys – Source Code for Detection and Image Dataset Li, Wei; Field, Kevin G.; Morgan, Dane Materials Data Facility https://doi.org/10.18126/m2692z	dataset	January 2018

Cited By (4)

Interplay Between Grain Boundaries and Radiation Damage Barr, Christopher M.; El-Atwani, Osman; Kaoumi, Djamel JOM, Vol. 71, Issue 4 https://doi.org/10.1007/s11837-019-03386-y	journal	March 2019
Deep Learning for Semantic Segmentation of Defects in Advanced STEM Images of Steels Roberts, Graham; Haile, Simon Y.; Sainju, Rajat Scientific Reports, Vol. 9, Issue 1 https://doi.org/10.1038/s41598-019-49105-0	journal	September 2019
Machine-learning-assisted screening of pure-silica zeolites for effective removal of linear siloxanes and derivatives Lin, Shiru; Wang, Yekun; Zhao, Yinghe Journal of Materials Chemistry A, Vol. 8, Issue 6 https://doi.org/10.1039/c9ta11909d	journal	January 2020
Data-driven approach for synchrotron X-ray Laue microdiffraction scan analysis Song, Yintao; Tamura, Nobumichi; Zhang, Chenbo Acta Crystallographica Section A Foundations and Advances, Vol. 75, Issue 6 https://doi.org/10.1107/s2053273319012804	journal	October 2019

Similar Records

Automated Quantitative Analysis of Extended Irradiation Defects - Dislocations, Voids and Precipitates in Neutron Irradiated HT-9 Steel

Journal Article · 2019 · Microscopy and Microanalysis · OSTI ID:1765324

Sainju, Rajat; Ophus, Colin; Toloczko, Mychailo B.; +2 more

Automated phase segmentation and quantification of high-resolution TEM image for alloy design

Journal Article · 2023 · Materials Characterization · OSTI ID:2004578

Liu, Sen; Amin-Ahmadi, Behnam; Liu, Rui; +2 more

An Open Analysis Platform with In-Situ Machine Vision for Electron Microscopy Final Technical Report

Technical Report · 2021 · OSTI ID:1833354

Field, Christopher Ryan; Sharaf, Nasir; Field, Kevin G; +1 more

Related Subjects

47 OTHER INSTRUMENTATION

Title: Automated defect analysis in electron microscopic images

Citation Formats

References (39)

Cited By (4)

Linked Research (1)

Similar Records

Related Subjects