Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

Deep Learning for Semantic Segmentation of Defects in Advanced STEM Images of Steels

Journal Article · · Scientific Reports
 [1];  [2];  [3];  [1];  [4];  [5]
  1. Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
  2. Western Washington Univ., Bellingham, WA (United States)
  3. Univ. of Connecticut, Storrs, CT (United States)
  4. Western Washington Univ., Bellingham, WA (United States); Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
  5. Pacific Northwest National Lab. (PNNL), Richland, WA (United States); Univ. of Connecticut, Storrs, CT (United States)
+ Show Author Affiliations
Crystalline materials exhibit long-range ordered lattice unit, within which resides nonperiodic structural features called defects. These crystallographic defects play a vital role in determining the physical and mechanical properties of a wide range of material systems. While computer vision has demonstrated success in recognizing feature patterns in images with well defined contrast, automated identification of nanometer scale crystallographic defects in electron micrographs governed by complex contrast mechanism is still a challenging task. Here, building upon an advanced defect imaging mode that offers high feature clarity, we introduce DefectSegNet - a new convolutional neural network (CNN) architecture that performs semantic segmentation of three common crystallographic defects in structural alloys: dislocation lines, precipitates and voids. Results from supervised training of a small set of high-quality defect images of steels show high pixel-wise accuracy across all three types of defects: 91.60 ± 1.77% on dislocations, 93.39 ± 1.00% on precipitates, and 98.85 ± 0.56% on voids. We discuss the sources of uncertainties in CNN prediction, and the training data in terms of feature density, representation and homogeneity and their effects on deep learning performance. Further defect quantification using DefectSegNet prediction outperforms human expert average, presenting a promising new workflow for fast and statistically meaningful quantification of materials defects.
Research Organization:
Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
Sponsoring Organization:
USDOE Office of Science (SC), Fusion Energy Sciences (FES) (SC-24)
Grant/Contract Number:
AC05-76RL01830
OSTI ID:
1559992
Report Number(s):
PNNL-SA--146546
Journal Information:
Scientific Reports, Journal Name: Scientific Reports Journal Issue: 1 Vol. 9; ISSN 2045-2322
Publisher:
Nature Publishing GroupCopyright Statement
Country of Publication:
United States
Language:
English

References (46)

A comparative study of texture measures with classification based on featured distributions journal January 1996
The formation and interpretation of defect images from crystalline materials in a scanning transmission electron microscope journal January 1976
Phase stability and structural defects in high-temperature Mo–Si–B alloys journal October 2008
Microstructure recognition using convolutional neural networks for prediction of ionic conductivity in ceramics journal December 2017
A computer vision approach for automated analysis and classification of microstructural image data journal December 2015
Image driven machine learning methods for microstructure recognition journal October 2016
Diffraction contrast STEM of dislocations: Imaging and simulations journal August 2011
Enhancing radiation tolerance by controlling defect mobility and migration pathways in multicomponent single-phase alloys journal December 2016
Advanced Steel Microstructural Classification by Deep Learning Methods journal February 2018
The relationship between grain boundary structure, defect mobility and grain boundary sink efficiency journal March 2015
A systematic error in the determination of dislocation densities in thin films journal September 1961
Dislocation movement through random arrays of obstacles journal October 1966
A Kinematical Theory of Diffraction Contrast of Electron Transmission Microscope Images of Dislocations and other Defects
  • Hirsch, P. B.; Howie, A.; Whelan, M. J.
  • Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 252, Issue 1017 https://doi.org/10.1098/rsta.1960.0013
journal May 1960
Image Enhancement via Subimage Histogram Equalization Based on Mean and Variance journal January 2017
Analytical Transmission Electron Microscopy datapaper January 2009
Beyond Pixels: A Comprehensive Survey from Bottom-up to Semantic Image Segmentation and Cosegmentation preprint January 2015
A deep learning approach to identify local structures in atomic-resolution transmission electron microscopy images text January 2018
A Deep Learning Approach to Identify Local Structures in Atomic-Resolution Transmission Electron Microscopy Images journal July 2018
U-Net: Convolutional Networks for Biomedical Image Segmentation
  • Ronneberger, Olaf; Fischer, Philipp; Brox, Thomas
  • Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III https://doi.org/10.1007/978-3-319-24574-4_28
book November 2015
Practical Recommendations for Gradient-Based Training of Deep Architectures book January 2012
A review of semantic segmentation using deep neural networks journal November 2017
Classification of corrosion defects in NiAl bronze through image analysis journal November 2010
Beyond pixels: A comprehensive survey from bottom-up to semantic image segmentation and cosegmentation journal January 2016
Disconnections and other defects associated with twin interfaces journal October 2016
Towards bend-contour-free dislocation imaging via diffraction contrast STEM journal October 2018
High Throughput Quantitative Metallography for Complex Microstructures Using Deep Learning: A Case Study in Ultrahigh Carbon Steel journal February 2019
Deep Learning of Atomically Resolved Scanning Transmission Electron Microscopy Images: Chemical Identification and Tracking Local Transformations journal October 2016
Automated defect analysis in electron microscopic images journal July 2018
AxonDeepSeg: automatic axon and myelin segmentation from microscopy data using convolutional neural networks journal February 2018
Fully convolutional networks for semantic segmentation conference June 2015
Densely Connected Convolutional Networks conference July 2017
The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation conference July 2017
Best practices for convolutional neural networks applied to visual document analysis conference January 2003
A Method for Detection and Classification of Glass Defects in Low Resolution Images conference August 2011
Steel defect classification with Max-Pooling Convolutional Neural Networks
  • Masci, Jonathan; Meier, Ueli; Ciresan, Dan
  • 2012 International Joint Conference on Neural Networks (IJCNN 2012 - Brisbane), The 2012 International Joint Conference on Neural Networks (IJCNN) https://doi.org/10.1109/IJCNN.2012.6252468
conference June 2012
Survey on semantic image segmentation techniques conference December 2017
Importance-Aware Semantic Segmentation for Autonomous Vehicles journal January 2019
Fully Convolutional Networks for Semantic Segmentation journal April 2017
Control and Use of Defects in Materials journal August 1998
Dislocation Movement Through Random Arrays of Obstacles journal February 1967
Analytical Transmission Electron Microscopy journal August 2005
Fully Convolutional Networks for Semantic Segmentation preprint January 2016
Densely Connected Convolutional Networks preprint January 2016
Advanced Steel Microstructural Classification by Deep Learning Methods text January 2017
Deep Learning of Atomically Resolved Scanning Transmission Electron Microscopy Images: Chemical Identification and Tracking Local Transformations text January 2018
High throughput quantitative metallography for complex microstructures using deep learning: A case study in ultrahigh carbon steel text January 2018

Cited By (1)

Learning-based Defect Recognition for Quasi-Periodic Microscope Images text January 2020

Similar Records

DefectSegNet
Software · Thu Oct 31 20:00:00 EDT 2019 · OSTI ID:code-31987
ADVANCED-STEM-BASED DEEP LEARNING FOR SEMANTIC SEGMENTATION OF DEFECTS IN STEELS
Book · Fri Mar 29 00:00:00 EDT 2019 · OSTI ID:1569022
Application of a deep learning semantic segmentation model to helium bubbles and voids in nuclear materials
Journal Article · Mon Jul 31 20:00:00 EDT 2023 · Engineering Applications of Artificial Intelligence · OSTI ID:2229233

Related Subjects

36 MATERIALS SCIENCE
STEM Imaging
convolutional neural network
machine leaning
semantic segmentation