DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Machine-Learning-based Algorithms for Automated Image Segmentation Techniques of Transmission X-ray Microscopy (TXM)

Abstract

Four state-of-the-art Deep Learning-based Convolutional Neural Networks (DCNN) were applied to automate the semantic segmentation of a 3D Transmission x-ray Microscopy (TXM) nanotomography image data. The standard U-Net architecture as baseline along with UNet++, PSPNet, and DeepLab v3+ networks were trained to segment the microstructural features of an AA7075 micropillar. A workflow was established to evaluate and compare the DCNN prediction dataset with the manually segmented features using the Intersection of Union (IoU) scores, time of training, confusion matrix, and visual assessment. Comparing all model segmentation accuracy metrics, it was found that using pre-trained models as a backbone along with appropriate training encoder-decoder architecture of the Unet++ can robustly handle large volumes of x-ray radiographic images in a reasonable amount of time. This opens a new window for handling accurate and efficient image segmentation of in situ time-dependent 4D x-ray microscopy experimental datasets.

Authors:
 [1];  [1];  [2];  [2];  [3];  [2];  [1]
  1. Purdue Univ., West Lafayette, IN (United States)
  2. Arizona State Univ., Tempe, AZ (United States)
  3. Argonne National Lab. (ANL), Argonne, IL (United States). Advanced Photon Source (APS)
Publication Date:
Research Org.:
Argonne National Laboratory (ANL), Argonne, IL (United States)
Sponsoring Org.:
US Department of the Navy, Office of Naval Research (ONR); USDOE Office of Science (SC)
OSTI Identifier:
1834598
Grant/Contract Number:  
AC02-06CH11357; N00014-10-1-0350
Resource Type:
Accepted Manuscript
Journal Name:
JOM. Journal of the Minerals, Metals & Materials Society
Additional Journal Information:
Journal Volume: 73; Journal Issue: 7; Journal ID: ISSN 1047-4838
Publisher:
Springer
Country of Publication:
United States
Language:
English
Subject:
36 MATERIALS SCIENCE

Citation Formats

Torbati-Sarraf, Hamidreza, Niverty, Sridhar, Singh, Rajhans, Barboza, Daniel, De Andrade, Vincent, Turaga, Pavan, and Chawla, Nikhilesh. Machine-Learning-based Algorithms for Automated Image Segmentation Techniques of Transmission X-ray Microscopy (TXM). United States: N. p., 2021. Web. doi:10.1007/s11837-021-04706-x.
Torbati-Sarraf, Hamidreza, Niverty, Sridhar, Singh, Rajhans, Barboza, Daniel, De Andrade, Vincent, Turaga, Pavan, & Chawla, Nikhilesh. Machine-Learning-based Algorithms for Automated Image Segmentation Techniques of Transmission X-ray Microscopy (TXM). United States. https://doi.org/10.1007/s11837-021-04706-x
Torbati-Sarraf, Hamidreza, Niverty, Sridhar, Singh, Rajhans, Barboza, Daniel, De Andrade, Vincent, Turaga, Pavan, and Chawla, Nikhilesh. Tue . "Machine-Learning-based Algorithms for Automated Image Segmentation Techniques of Transmission X-ray Microscopy (TXM)". United States. https://doi.org/10.1007/s11837-021-04706-x. https://www.osti.gov/servlets/purl/1834598.
@article{osti_1834598,
title = {Machine-Learning-based Algorithms for Automated Image Segmentation Techniques of Transmission X-ray Microscopy (TXM)},
author = {Torbati-Sarraf, Hamidreza and Niverty, Sridhar and Singh, Rajhans and Barboza, Daniel and De Andrade, Vincent and Turaga, Pavan and Chawla, Nikhilesh},
abstractNote = {Four state-of-the-art Deep Learning-based Convolutional Neural Networks (DCNN) were applied to automate the semantic segmentation of a 3D Transmission x-ray Microscopy (TXM) nanotomography image data. The standard U-Net architecture as baseline along with UNet++, PSPNet, and DeepLab v3+ networks were trained to segment the microstructural features of an AA7075 micropillar. A workflow was established to evaluate and compare the DCNN prediction dataset with the manually segmented features using the Intersection of Union (IoU) scores, time of training, confusion matrix, and visual assessment. Comparing all model segmentation accuracy metrics, it was found that using pre-trained models as a backbone along with appropriate training encoder-decoder architecture of the Unet++ can robustly handle large volumes of x-ray radiographic images in a reasonable amount of time. This opens a new window for handling accurate and efficient image segmentation of in situ time-dependent 4D x-ray microscopy experimental datasets.},
doi = {10.1007/s11837-021-04706-x},
journal = {JOM. Journal of the Minerals, Metals & Materials Society},
number = 7,
volume = 73,
place = {United States},
year = {Tue May 11 00:00:00 EDT 2021},
month = {Tue May 11 00:00:00 EDT 2021}
}

Works referenced in this record:

Nanoscale Three-Dimensional Microstructural Characterization of an Sn-Rich Solder Alloy Using High-Resolution Transmission X-Ray Microscopy (TXM)
journal, July 2016

  • Kaira, Chandrashekara S.; Mayer, Carl R.; De Andrade, V.
  • Microscopy and Microanalysis, Vol. 22, Issue 4
  • DOI: 10.1017/S1431927616011429

Advanced Deep Learning‐Based 3D Microstructural Characterization of Multiphase Metal Matrix Composites
journal, February 2020

  • Evsevleev, Sergei; Paciornik, Sidnei; Bruno, Giovanni
  • Advanced Engineering Materials, Vol. 22, Issue 4
  • DOI: 10.1002/adem.201901197

U-Net: Convolutional Networks for Biomedical Image Segmentation
book, November 2015

  • Ronneberger, Olaf; Fischer, Philipp; Brox, Thomas
  • Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III
  • DOI: 10.1007/978-3-319-24574-4_28

Real-time quantitative imaging of failure events in materials under load at temperatures above 1,600 °C
journal, December 2012

  • Bale, Hrishikesh A.; Haboub, Abdel; MacDowell, Alastair A.
  • Nature Materials, Vol. 12, Issue 1
  • DOI: 10.1038/nmat3497

TomoPy: a framework for the analysis of synchrotron tomographic data
journal, August 2014

  • Gürsoy, Dogˇa; De Carlo, Francesco; Xiao, Xianghui
  • Journal of Synchrotron Radiation, Vol. 21, Issue 5
  • DOI: 10.1107/S1600577514013939

Learning From Scanning Transmission Electron Microscopy to Enhance Transmission X-ray Microscopy: How We Can Merge STEM and TXM Datasets?
journal, July 2016


Following the phase transitions of iron in 3D with X-ray tomography and diffraction under extreme conditions
journal, June 2020


A method for zinger artifact reduction in high-energy x-ray computed tomography
journal, November 2015

  • Mertens, J. C. E.; Williams, J. J.; Chawla, Nikhilesh
  • Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, Vol. 800
  • DOI: 10.1016/j.nima.2015.08.012

Defect detection in textured materials using optimized filters
journal, October 2002

  • Kumar, A.; Pang, G. K. H.
  • IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics), Vol. 32, Issue 5
  • DOI: 10.1109/TSMCB.2002.1033176

In situ 3D quantification of the evolution of creep cavity size, shape, and spatial orientation using synchrotron X-ray tomography
journal, April 2008


Understanding Nanoscale 4D Microstructural Evolution in Aluminum Alloys using Transmission X-Ray Microscopy (TXM)
journal, July 2017

  • Kaira, C. Shashank; De Andrade, V.; Singh, S. S.
  • Microscopy and Microanalysis, Vol. 23, Issue S1
  • DOI: 10.1017/S143192761701176X

Optimizing convolutional neural networks to perform semantic segmentation on large materials imaging datasets: X-ray tomography and serial sectioning
journal, February 2020


Automated correlative segmentation of large Transmission X-ray Microscopy (TXM) tomograms using deep learning
journal, August 2018


Multiscale investigation of corrosion damage initiation and propagation in AA7075-T651 alloy using correlative microscopy
journal, June 2021


Pitting corrosion of naturally aged AA 7075 aluminum alloys with bimodal grain size
journal, December 2016


In situ and real-time 3-D microtomography investigation of dendritic solidification in an Al–10wt.% Cu alloy
journal, April 2009


UNet++: A Nested U-Net Architecture for Medical Image Segmentation
book, January 2018

  • Zhou, Zongwei; Rahman Siddiquee, Md Mahfuzur; Tajbakhsh, Nima
  • Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support
  • DOI: 10.1007/978-3-030-00889-5_1

DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
journal, April 2018

  • Chen, Liang-Chieh; Papandreou, George; Kokkinos, Iasonas
  • IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 40, Issue 4
  • DOI: 10.1109/TPAMI.2017.2699184

Microstructural evolution and deformation behavior of Al-Cu alloys: A Transmission X-ray Microscopy (TXM) and micropillar compression study
journal, February 2018


Microstructure and micropore formation in a centrifugally-cast duplex stainless steel via X-ray microtomography
journal, February 2019


An integrated ship segmentation method based on discriminator and extractor
journal, January 2020


Probing Novel Microstructural Evolution Mechanisms in Aluminum Alloys Using 4D Nanoscale Characterization
journal, September 2017

  • Kaira, C. Shashank; De Andrade, V.; Singh, Sudhanshu S.
  • Advanced Materials, Vol. 29, Issue 41
  • DOI: 10.1002/adma.201703482

3D Time-Resolved Observations of Fatigue Crack Initiation and Growth from Corrosion Pits in Al 7XXX Alloys Using In Situ Synchrotron X-ray Tomography
journal, November 2019

  • Singaravelu, Arun Sundar Sundaram; Williams, Jason J.; Goyal, Harsh Dev
  • Metallurgical and Materials Transactions A, Vol. 51, Issue 1
  • DOI: 10.1007/s11661-019-05519-z

In Situ X-ray Microtomography of Stress Corrosion Cracking and Corrosion Fatigue in Aluminum Alloys
journal, June 2017


Elemental and Chemical Mapping of High Capacity Intermetallic Li-ion Anodes with Transmission X-ray Microscopy
journal, June 2017


Application of machine learning techniques in mineral phase segmentation for X-ray microcomputed tomography (µCT) data
journal, October 2019


Recent progress in semantic image segmentation
journal, June 2018


Microstructural design for damage tolerance in high strength steels
journal, June 2020


Effect of temper on the distribution of pits in AA7075 alloys
journal, October 2008


Microstructure Instance Segmentation from Aluminum Alloy Metallographic Image Using Different Loss Functions
journal, April 2020


Damage investigation and modeling of 3D woven ceramic matrix composites from X-ray tomography in-situ tensile tests
journal, November 2017


Synchrotron CT imaging of lattice structures with engineered defects
journal, May 2020

  • Patterson, Brian M.; Kuettner, Lindsey; Shear, Trevor
  • Journal of Materials Science, Vol. 55, Issue 25
  • DOI: 10.1007/s10853-020-04840-y

Mini Review: Deep Learning for Atrial Segmentation From Late Gadolinium-Enhanced MRIs
journal, May 2020

  • Jamart, Kevin; Xiong, Zhaohan; Maso Talou, Gonzalo D.
  • Frontiers in Cardiovascular Medicine, Vol. 7
  • DOI: 10.3389/fcvm.2020.00086

An analytical electron microscopy study of constituent particles in commercial 7075-T6 and 2024-T3 alloys
journal, April 1998

  • Gao, Ming; Feng, C. R.; Wei, Robert P.
  • Metallurgical and Materials Transactions A, Vol. 29, Issue 4
  • DOI: 10.1007/s11661-998-0240-9

Deep Learning-Based Image Segmentation for Al-La Alloy Microscopic Images
journal, April 2018


Electromigration in Bi-crystal pure Sn solder joints: Elucidating the role of grain orientation
journal, March 2020


Low-dose x-ray tomography through a deep convolutional neural network
journal, February 2018


3D microstructural characterization and mechanical properties of constituent particles in Al 7075 alloys using X-ray synchrotron tomography and nanoindentation
journal, July 2014


Damage evolution in SiC particle reinforced Al alloy matrix composites by X-ray synchrotron tomography
journal, October 2010


A convolutional neural network approach to calibrating the rotation axis for X-ray computed tomography
journal, January 2017

  • Yang, Xiaogang; De Carlo, Francesco; Phatak, Charudatta
  • Journal of Synchrotron Radiation, Vol. 24, Issue 2
  • DOI: 10.1107/S1600577516020117