Deep Learning for Semantic Segmentation of Defects in Advanced STEM Images of Steels

doi:10.1038/s41598-019-49105-0

Title: Deep Learning for Semantic Segmentation of Defects in Advanced STEM Images of Steels

Full Record
Other Related Research

Abstract

Crystalline materials exhibit long-range ordered lattice unit, within which resides nonperiodic structural features called defects. These crystallographic defects play a vital role in determining the physical and mechanical properties of a wide range of material systems. While computer vision has demonstrated success in recognizing feature patterns in images with well defined contrast, automated identification of nanometer scale crystallographic defects in electron micrographs governed by complex contrast mechanism is still a challenging task. Here, building upon an advanced defect imaging mode that offers high feature clarity, we introduce DefectSegNet - a new convolutional neural network (CNN) architecture that performs semantic segmentation of three common crystallographic defects in structural alloys: dislocation lines, precipitates and voids. Results from supervised training of a small set of high-quality defect images of steels show high pixel-wise accuracy across all three types of defects: 91.60 ± 1.77% on dislocations, 93.39 ± 1.00% on precipitates, and 98.85 ± 0.56% on voids. We discuss the sources of uncertainties in CNN prediction, and the training data in terms of feature density, representation and homogeneity and their effects on deep learning performance. Further defect quantification using DefectSegNet prediction outperforms human expert average, presenting a promising new workflow for fast and statisticallymore »« less

Authors:

Roberts, Graham ^[1]; Haile, Simon ^[2]; Sainju, Rajat ^[3];

^[1]; Hutchinson, Brian J. ^[4];

^[5]

Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
Western Washington Univ., Bellingham, WA (United States)
Univ. of Connecticut, Storrs, CT (United States)
Western Washington Univ., Bellingham, WA (United States); Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
Pacific Northwest National Lab. (PNNL), Richland, WA (United States); Univ. of Connecticut, Storrs, CT (United States)

Publication Date:: 2019-09-04

Research Org.:: Pacific Northwest National Lab. (PNNL), Richland, WA (United States)

Sponsoring Org.:: USDOE Office of Science (SC), Fusion Energy Sciences (FES) (SC-24)

OSTI Identifier:: 1559992

Report Number(s):: PNNL-SA-146546
Journal ID: ISSN 2045-2322

Grant/Contract Number:: AC05-76RL01830

Resource Type:: Accepted Manuscript

Journal Name:: Scientific Reports

Additional Journal Information:: Journal Volume: 9; Journal Issue: 1; Journal ID: ISSN 2045-2322

Publisher:: Nature Publishing Group

Country of Publication:: United States

Language:: English

Subject:: machine leaning; STEM Imaging; convolutional neural network; semantic segmentation

Citation Formats


                    Roberts, Graham, Haile, Simon, Sainju, Rajat, Edwards, Danny J., Hutchinson, Brian J., and Zhu, Yuanyuan. Deep Learning for Semantic Segmentation of Defects in Advanced STEM Images of Steels.  United States: N. p., 2019. 
        Web.  doi:10.1038/s41598-019-49105-0.

Copy to clipboard


                    Roberts, Graham, Haile, Simon, Sainju, Rajat, Edwards, Danny J., Hutchinson, Brian J., & Zhu, Yuanyuan. Deep Learning for Semantic Segmentation of Defects in Advanced STEM Images of Steels.  United States.  doi:10.1038/s41598-019-49105-0.

Copy to clipboard


                    Roberts, Graham, Haile, Simon, Sainju, Rajat, Edwards, Danny J., Hutchinson, Brian J., and Zhu, Yuanyuan. Wed .  
        "Deep Learning for Semantic Segmentation of Defects in Advanced STEM Images of Steels".  United States.  doi:10.1038/s41598-019-49105-0.  https://www.osti.gov/servlets/purl/1559992.

Copy to clipboard


                    
@article{osti_1559992,

  title        = {Deep Learning for Semantic Segmentation of Defects in Advanced STEM Images of Steels},

  author       = {Roberts, Graham and Haile, Simon and Sainju, Rajat and Edwards, Danny J. and Hutchinson, Brian J. and Zhu, Yuanyuan},

  abstractNote = {Crystalline materials exhibit long-range ordered lattice unit, within which resides nonperiodic structural features called defects. These crystallographic defects play a vital role in determining the physical and mechanical properties of a wide range of material systems. While computer vision has demonstrated success in recognizing feature patterns in images with well defined contrast, automated identification of nanometer scale crystallographic defects in electron micrographs governed by complex contrast mechanism is still a challenging task. Here, building upon an advanced defect imaging mode that offers high feature clarity, we introduce DefectSegNet - a new convolutional neural network (CNN) architecture that performs semantic segmentation of three common crystallographic defects in structural alloys: dislocation lines, precipitates and voids. Results from supervised training of a small set of high-quality defect images of steels show high pixel-wise accuracy across all three types of defects: 91.60 ± 1.77% on dislocations, 93.39 ± 1.00% on precipitates, and 98.85 ± 0.56% on voids. We discuss the sources of uncertainties in CNN prediction, and the training data in terms of feature density, representation and homogeneity and their effects on deep learning performance. Further defect quantification using DefectSegNet prediction outperforms human expert average, presenting a promising new workflow for fast and statistically meaningful quantification of materials defects.},

  doi          = {10.1038/s41598-019-49105-0},

  journal      = {Scientific Reports},

  number       = 1,

  volume       = 9,

  place        = {United States},

  year         = {2019},

  month        = {9}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

DOI: 10.1038/s41598-019-49105-0

Other availability

Search WorldCat to find libraries that may hold this journal

Save / Share:

Export Metadata

Save to My Library

Similar Records in DOE PAGES and OSTI.GOV collections:

ADVANCED-STEM-BASED DEEP LEARNING FOR SEMANTIC SEGMENTATION OF DEFECTS IN STEELS

Book Zhu, Yuanyuan ; Roberts, Graham W. ; Sainju, Rajat ; ...

In this work, we developed a novel deep convolutional neural network (DCNN) model, called DefectNet, for robust and automated semantic segmentation of three crystallographic defects including line dislocations, precipitates and voids commonly observed in structural metals and alloys. In previous work, we established an experimental protocol for a diffraction contrast imaging scanning transmission electron microscopy (DCI STEM) technique and tailored it specifically for imaging defects in popular iron-based structural alloy. Thus, the DefectNet was trained over a small but high-quality DCI STEM defect image sets of a HT-9 martensitic steel before and after neutron irradiation for 111.8 dpa at 412°C.more »« less
Automated correlative segmentation of large Transmission X-ray Microscopy (TXM) tomograms using deep learning

Journal Article Shashank Kaira, C. ; Yang, Xiaogang ; De Andrade, Vincent ; ... - Materials Characterization

A unique correlative approach for automated segmentation of large 3D nanotomography datasets obtained using Transmission X-ray Microscopy (TXM) in an Al-Cu alloy has been introduced. Automated segmentation using a Convolutional Neural Network (CNN) architecture based on a deep learning approach was employed. This extremely versatile technique is capable of emulating the manual segmentation process effectively. Coupling this technique with post-scanning SEM imaging ensured precise estimation of 3D morphological parameters from nanotomography. The segmentation process as well as subsequent analysis was expedited by several orders of magnitude. Quantitative comparison between segmentation performed manually and using the CNN architecture established the accuracymore »« less
Cited by 1
DOI: 10.1016/j.matchar.2018.05.053

Full Text Available
Jet-images — deep learning edition

Journal Article de Oliveira, Luke ; Kagan, Michael ; Mackey, Lester ; ... - Journal of High Energy Physics (Online)

Building on the notion of a particle physics detector as a camera and the collimated streams of high energy particles, or jets, it measures as an image, we investigate the potential of machine learning techniques based on deep learning architectures to identify highly boosted W bosons. Modern deep learning algorithms trained on jet images can out-perform standard physically-motivated feature driven approaches to jet tagging. We develop techniques for visualizing how these features are learned by the network and what additional information is used to improve performance. Finally, this interplay between physically-motivated feature driven tools and supervised learning algorithms is generalmore »« less
Cited by 13
DOI: 10.1007/JHEP07(2016)069

Full Text Available
DeepWelding: a Deep Learning Enhanced Approach to GTAW Using Multi-source Sensing Images

Journal Article Feng, Yunhe ; Chen, Zongyao ; Wang, Dali ; ... - IEEE Transactions on Industrial Informatics

Deep learning has great potential to reshape manufacturing industries. In this paper, we present DeepWelding, a novel framework that applies deep learning techniques to improve gas tungsten arc welding (GTAW) process monitoring and penetration detection using multi-source sensing images. The framework is capable of analyzing multiple types of optical sensing images synchronously and consists of three deep learning enhanced consecutive phases: image preprocessing, image selection, and weld penetration classification. Specifically, we adopted generative adversarial networks (pix2pix) for image denoising and classic convolutional neural networks (AlexNet) for image selection. Both pix2pix and AlexNet delivered satisfactory performance. However, five individual neural networksmore »« less
DOI: 10.1109/TII.2019.2937563
Transfer learning for galaxy morphology from one survey to another [Knowledge transfer of Deep Learning for galaxy morphology from one survey to another]

Journal Article Domínguez Sánchez, H. ; Huertas-Company, M. ; Bernardi, M. ; ... - Monthly Notices of the Royal Astronomical Society

Deep Learning (DL) algorithms for morphological classification of galaxies have proven very successful, mimicking (or even improving) visual classifications. However, these algorithms rely on large training samples of labeled galaxies (typically thousands of them). A key question for using DL classifications in future Big Data surveys is how much of the knowledge acquired from an existing survey can be exported to a new dataset, i.e. if the features learned by the machines are meaningful for different data. Here, we test the performance of DL models, trained with Sloan Digital Sky Survey (SDSS) data, on Dark Energy survey (DES) using images for a sample of 5000 galaxies with a similar redshift distribution to SDSS. Applying the models directly to DES data provides a reasonable global accuracy (more »« less
DOI: 10.1093/mnras/sty3497

Similar Records