Machine-Learning-based Algorithms for Automated Image Segmentation Techniques of Transmission X-ray Microscopy (TXM)
Abstract
Four state-of-the-art Deep Learning-based Convolutional Neural Networks (DCNN) were applied to automate the semantic segmentation of a 3D Transmission x-ray Microscopy (TXM) nanotomography image data. The standard U-Net architecture as baseline along with UNet++, PSPNet, and DeepLab v3+ networks were trained to segment the microstructural features of an AA7075 micropillar. A workflow was established to evaluate and compare the DCNN prediction dataset with the manually segmented features using the Intersection of Union (IoU) scores, time of training, confusion matrix, and visual assessment. Comparing all model segmentation accuracy metrics, it was found that using pre-trained models as a backbone along with appropriate training encoder-decoder architecture of the Unet++ can robustly handle large volumes of x-ray radiographic images in a reasonable amount of time. This opens a new window for handling accurate and efficient image segmentation of in situ time-dependent 4D x-ray microscopy experimental datasets.
- Authors:
-
- Purdue Univ., West Lafayette, IN (United States)
- Arizona State Univ., Tempe, AZ (United States)
- Argonne National Lab. (ANL), Argonne, IL (United States). Advanced Photon Source (APS)
- Publication Date:
- Research Org.:
- Argonne National Laboratory (ANL), Argonne, IL (United States)
- Sponsoring Org.:
- US Department of the Navy, Office of Naval Research (ONR); USDOE Office of Science (SC)
- OSTI Identifier:
- 1834598
- Grant/Contract Number:
- AC02-06CH11357; N00014-10-1-0350
- Resource Type:
- Accepted Manuscript
- Journal Name:
- JOM. Journal of the Minerals, Metals & Materials Society
- Additional Journal Information:
- Journal Volume: 73; Journal Issue: 7; Journal ID: ISSN 1047-4838
- Publisher:
- Springer
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 36 MATERIALS SCIENCE
Citation Formats
Torbati-Sarraf, Hamidreza, Niverty, Sridhar, Singh, Rajhans, Barboza, Daniel, De Andrade, Vincent, Turaga, Pavan, and Chawla, Nikhilesh. Machine-Learning-based Algorithms for Automated Image Segmentation Techniques of Transmission X-ray Microscopy (TXM). United States: N. p., 2021.
Web. doi:10.1007/s11837-021-04706-x.
Torbati-Sarraf, Hamidreza, Niverty, Sridhar, Singh, Rajhans, Barboza, Daniel, De Andrade, Vincent, Turaga, Pavan, & Chawla, Nikhilesh. Machine-Learning-based Algorithms for Automated Image Segmentation Techniques of Transmission X-ray Microscopy (TXM). United States. https://doi.org/10.1007/s11837-021-04706-x
Torbati-Sarraf, Hamidreza, Niverty, Sridhar, Singh, Rajhans, Barboza, Daniel, De Andrade, Vincent, Turaga, Pavan, and Chawla, Nikhilesh. Tue .
"Machine-Learning-based Algorithms for Automated Image Segmentation Techniques of Transmission X-ray Microscopy (TXM)". United States. https://doi.org/10.1007/s11837-021-04706-x. https://www.osti.gov/servlets/purl/1834598.
@article{osti_1834598,
title = {Machine-Learning-based Algorithms for Automated Image Segmentation Techniques of Transmission X-ray Microscopy (TXM)},
author = {Torbati-Sarraf, Hamidreza and Niverty, Sridhar and Singh, Rajhans and Barboza, Daniel and De Andrade, Vincent and Turaga, Pavan and Chawla, Nikhilesh},
abstractNote = {Four state-of-the-art Deep Learning-based Convolutional Neural Networks (DCNN) were applied to automate the semantic segmentation of a 3D Transmission x-ray Microscopy (TXM) nanotomography image data. The standard U-Net architecture as baseline along with UNet++, PSPNet, and DeepLab v3+ networks were trained to segment the microstructural features of an AA7075 micropillar. A workflow was established to evaluate and compare the DCNN prediction dataset with the manually segmented features using the Intersection of Union (IoU) scores, time of training, confusion matrix, and visual assessment. Comparing all model segmentation accuracy metrics, it was found that using pre-trained models as a backbone along with appropriate training encoder-decoder architecture of the Unet++ can robustly handle large volumes of x-ray radiographic images in a reasonable amount of time. This opens a new window for handling accurate and efficient image segmentation of in situ time-dependent 4D x-ray microscopy experimental datasets.},
doi = {10.1007/s11837-021-04706-x},
journal = {JOM. Journal of the Minerals, Metals & Materials Society},
number = 7,
volume = 73,
place = {United States},
year = {Tue May 11 00:00:00 EDT 2021},
month = {Tue May 11 00:00:00 EDT 2021}
}
Works referenced in this record:
Nanoscale Three-Dimensional Microstructural Characterization of an Sn-Rich Solder Alloy Using High-Resolution Transmission X-Ray Microscopy (TXM)
journal, July 2016
- Kaira, Chandrashekara S.; Mayer, Carl R.; De Andrade, V.
- Microscopy and Microanalysis, Vol. 22, Issue 4
Advanced Deep Learning‐Based 3D Microstructural Characterization of Multiphase Metal Matrix Composites
journal, February 2020
- Evsevleev, Sergei; Paciornik, Sidnei; Bruno, Giovanni
- Advanced Engineering Materials, Vol. 22, Issue 4
U-Net: Convolutional Networks for Biomedical Image Segmentation
book, November 2015
- Ronneberger, Olaf; Fischer, Philipp; Brox, Thomas
- Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III
Formation of intermetallic δ phase in Al-10Si-0.3Fe alloy investigated by in-situ 4D X-ray synchrotron tomography
journal, May 2017
- Yu, J. M.; Wanderka, N.; Rack, A.
- Acta Materialia, Vol. 129
Real-time quantitative imaging of failure events in materials under load at temperatures above 1,600 °C
journal, December 2012
- Bale, Hrishikesh A.; Haboub, Abdel; MacDowell, Alastair A.
- Nature Materials, Vol. 12, Issue 1
TomoPy: a framework for the analysis of synchrotron tomographic data
journal, August 2014
- Gürsoy, Dogˇa; De Carlo, Francesco; Xiao, Xianghui
- Journal of Synchrotron Radiation, Vol. 21, Issue 5
Learning From Scanning Transmission Electron Microscopy to Enhance Transmission X-ray Microscopy: How We Can Merge STEM and TXM Datasets?
journal, July 2016
- Yang, X.; Gürsoy, D.; Phatak, C.
- Microscopy and Microanalysis, Vol. 22, Issue S3
Formation of a Three-Phase Spiral Structure Due to Competitive Growth of a Peritectic Phase with a Metastable Eutectic
journal, June 2020
- Wang, Yeqing; Gao, Jianrong; Ren, Yang
- JOM, Vol. 72, Issue 8
Following the phase transitions of iron in 3D with X-ray tomography and diffraction under extreme conditions
journal, June 2020
- Boulard, Eglantine; Denoual, Christophe; Dewaele, Agnès
- Acta Materialia, Vol. 192
A method for zinger artifact reduction in high-energy x-ray computed tomography
journal, November 2015
- Mertens, J. C. E.; Williams, J. J.; Chawla, Nikhilesh
- Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, Vol. 800
Defect detection in textured materials using optimized filters
journal, October 2002
- Kumar, A.; Pang, G. K. H.
- IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics), Vol. 32, Issue 5
In situ 3D quantification of the evolution of creep cavity size, shape, and spatial orientation using synchrotron X-ray tomography
journal, April 2008
- Isaac, A.; Sket, F.; Reimers, W.
- Materials Science and Engineering: A, Vol. 478, Issue 1-2
Understanding Nanoscale 4D Microstructural Evolution in Aluminum Alloys using Transmission X-Ray Microscopy (TXM)
journal, July 2017
- Kaira, C. Shashank; De Andrade, V.; Singh, S. S.
- Microscopy and Microanalysis, Vol. 23, Issue S1
Optimizing convolutional neural networks to perform semantic segmentation on large materials imaging datasets: X-ray tomography and serial sectioning
journal, February 2020
- Stan, Tiberiu; Thompson, Zachary T.; Voorhees, Peter W.
- Materials Characterization, Vol. 160
Automated correlative segmentation of large Transmission X-ray Microscopy (TXM) tomograms using deep learning
journal, August 2018
- Shashank Kaira, C.; Yang, Xiaogang; De Andrade, Vincent
- Materials Characterization, Vol. 142
Multiscale investigation of corrosion damage initiation and propagation in AA7075-T651 alloy using correlative microscopy
journal, June 2021
- Niverty, Sridhar; Kale, Chaitanya; Solanki, Kiran N.
- Corrosion Science, Vol. 185
Pitting corrosion of naturally aged AA 7075 aluminum alloys with bimodal grain size
journal, December 2016
- Tian, Wenming; Li, Songmei; Wang, Bo
- Corrosion Science, Vol. 113
In situ and real-time 3-D microtomography investigation of dendritic solidification in an Al–10wt.% Cu alloy
journal, April 2009
- Limodin, N.; Salvo, L.; Boller, E.
- Acta Materialia, Vol. 57, Issue 7
UNet++: A Nested U-Net Architecture for Medical Image Segmentation
book, January 2018
- Zhou, Zongwei; Rahman Siddiquee, Md Mahfuzur; Tajbakhsh, Nima
- Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support
DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
journal, April 2018
- Chen, Liang-Chieh; Papandreou, George; Kokkinos, Iasonas
- IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 40, Issue 4
Microstructural evolution and deformation behavior of Al-Cu alloys: A Transmission X-ray Microscopy (TXM) and micropillar compression study
journal, February 2018
- Kaira, C. Shashank; Kantzos, Christopher; Williams, Jason J.
- Acta Materialia, Vol. 144
Microstructure and micropore formation in a centrifugally-cast duplex stainless steel via X-ray microtomography
journal, February 2019
- Zhang, Qingdong; Niverty, Sridhar; Singaravelu, Arun Sundar S.
- Materials Characterization, Vol. 148
An integrated ship segmentation method based on discriminator and extractor
journal, January 2020
- Zhang, Wen; He, Xujie; Li, Wanyi
- Image and Vision Computing, Vol. 93
Probing Novel Microstructural Evolution Mechanisms in Aluminum Alloys Using 4D Nanoscale Characterization
journal, September 2017
- Kaira, C. Shashank; De Andrade, V.; Singh, Sudhanshu S.
- Advanced Materials, Vol. 29, Issue 41
3D Time-Resolved Observations of Fatigue Crack Initiation and Growth from Corrosion Pits in Al 7XXX Alloys Using In Situ Synchrotron X-ray Tomography
journal, November 2019
- Singaravelu, Arun Sundar Sundaram; Williams, Jason J.; Goyal, Harsh Dev
- Metallurgical and Materials Transactions A, Vol. 51, Issue 1
In Situ X-ray Microtomography of Stress Corrosion Cracking and Corrosion Fatigue in Aluminum Alloys
journal, June 2017
- Singh, Sudhanshu S.; Stannard, Tyler J.; Xiao, Xianghui
- JOM, Vol. 69, Issue 8
Elemental and Chemical Mapping of High Capacity Intermetallic Li-ion Anodes with Transmission X-ray Microscopy
journal, June 2017
- Ausderau, Logan J.; Gonzalez Malabet, Hernando J.; Buckley, Joseph R.
- JOM, Vol. 69, Issue 9
Application of machine learning techniques in mineral phase segmentation for X-ray microcomputed tomography (µCT) data
journal, October 2019
- Guntoro, Pratama Istiadi; Tiu, Glacialle; Ghorbani, Yousef
- Minerals Engineering, Vol. 142
Recent progress in semantic image segmentation
journal, June 2018
- Liu, Xiaolong; Deng, Zhidong; Yang, Yuhan
- Artificial Intelligence Review, Vol. 52, Issue 2
Four dimensional (4D) microstructural evolution of Cu6Sn5 intermetallic and voids under electromigration in bi-crystal pure Sn solder joints
journal, May 2020
- Kelly, Marion Branch; Niverty, Sridhar; Chawla, Nikhilesh
- Acta Materialia, Vol. 189
Microstructural design for damage tolerance in high strength steels
journal, June 2020
- Samei, Javad; Pelligra, Concetta; Amirmaleki, M.
- Materials Letters, Vol. 269
Effect of temper on the distribution of pits in AA7075 alloys
journal, October 2008
- Dey, Swapna; Gunjan, Manoj K.; Chattoraj, Indranil
- Corrosion Science, Vol. 50, Issue 10
Direct observation of the displacement field and microcracking in a glass by means of X-ray tomography during in situ Vickers indentation experiment
journal, October 2019
- Lacondemine, Tanguy; Réthoré, Julien; Maire, Éric
- Acta Materialia, Vol. 179
Microstructure Instance Segmentation from Aluminum Alloy Metallographic Image Using Different Loss Functions
journal, April 2020
- Chen, Dali; Guo, Dinghao; Liu, Shixin
- Symmetry, Vol. 12, Issue 4
Damage investigation and modeling of 3D woven ceramic matrix composites from X-ray tomography in-situ tensile tests
journal, November 2017
- Mazars, Vincent; Caty, Olivier; Couégnat, Guillaume
- Acta Materialia, Vol. 140
Synchrotron CT imaging of lattice structures with engineered defects
journal, May 2020
- Patterson, Brian M.; Kuettner, Lindsey; Shear, Trevor
- Journal of Materials Science, Vol. 55, Issue 25
Mini Review: Deep Learning for Atrial Segmentation From Late Gadolinium-Enhanced MRIs
journal, May 2020
- Jamart, Kevin; Xiong, Zhaohan; Maso Talou, Gonzalo D.
- Frontiers in Cardiovascular Medicine, Vol. 7
An analytical electron microscopy study of constituent particles in commercial 7075-T6 and 2024-T3 alloys
journal, April 1998
- Gao, Ming; Feng, C. R.; Wei, Robert P.
- Metallurgical and Materials Transactions A, Vol. 29, Issue 4
Upscaling ice crystal growth dynamics in snow: Rigorous modeling and comparison to 4D X-ray tomography data
journal, June 2018
- Krol, Quirine; Löwe, Henning
- Acta Materialia, Vol. 151
Deep Learning-Based Image Segmentation for Al-La Alloy Microscopic Images
journal, April 2018
- Ma, Boyuan; Ban, Xiaojuan; Huang, Haiyou
- Symmetry, Vol. 10, Issue 4
Electromigration in Bi-crystal pure Sn solder joints: Elucidating the role of grain orientation
journal, March 2020
- Kelly, Marion Branch; Niverty, Sridhar; Chawla, Nikhilesh
- Journal of Alloys and Compounds, Vol. 818
Low-dose x-ray tomography through a deep convolutional neural network
journal, February 2018
- Yang, Xiaogang; De Andrade, Vincent; Scullin, William
- Scientific Reports, Vol. 8, Issue 1
3D microstructural characterization and mechanical properties of constituent particles in Al 7075 alloys using X-ray synchrotron tomography and nanoindentation
journal, July 2014
- Singh, Sudhanshu S.; Schwartzstein, Cary; Williams, Jason J.
- Journal of Alloys and Compounds, Vol. 602
Damage evolution in SiC particle reinforced Al alloy matrix composites by X-ray synchrotron tomography
journal, October 2010
- Williams, J. J.; Flom, Z.; Amell, A. A.
- Acta Materialia, Vol. 58, Issue 18
A convolutional neural network approach to calibrating the rotation axis for X-ray computed tomography
journal, January 2017
- Yang, Xiaogang; De Carlo, Francesco; Phatak, Charudatta
- Journal of Synchrotron Radiation, Vol. 24, Issue 2