Machine learning-based multidomain processing for texture-based image segmentation and analysis

Borodinov, Nikolay; Tsai, Wan-Yu; Korolkov, Vladimir V.; Wisinger, Nina Balke; Kalinin, Sergei V.; Ovchinnikova, Olga S.

doi:10.1063/1.5135328

Title: Machine learning-based multidomain processing for texture-based image segmentation and analysis

Journal Article · 27 January 2020 · Applied Physics Letters

DOI: https://doi.org/10.1063/1.5135328 · OSTI ID:1609049

^[1];

^[2];

^[1];

^[1]

Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Univ. of Nottingham (United Kingdom)

Atomic and molecular resolved atomic force microscopy (AFM) images offer unique insights into materials' properties such as local ordering, molecular orientation, and topological defects, which can be used to pinpoint physical and chemical interactions occurring at the surface. Utilizing machine learning for extracting underlying physical parameters increases the throughput of AFM data processing and eliminates inconsistencies intrinsic to manual image analysis, thus enabling the creation of reliable frameworks for qualitative and quantitative evaluation of experimental data. Here, we present a robust and scalable approach to the segmentation of AFM images based on flexible pre-selected classification criteria. The usage of supervised learning and feature extraction allows us to retain the consideration of specific problem-dependent features (such as types of periodical structures observed in the images and the associated numerical parameters: spacing, orientation, etc.). We highlight the applicability of this approach for the segmentation of molecular resolved AFM images based on the crystal orientation of the observed domains, automated selection of boundaries, and collection of relevant statistics. Finally, we outline a general strategy for machine learning-enabled analysis of nanoscale systems exhibiting periodic order that could be applied to any analytical imaging technique

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Organization:: USDOE

Grant/Contract Number:: AC05-00OR22725

OSTI ID:: 1609049

Journal Information:: Applied Physics Letters, Journal Name: Applied Physics Letters Journal Issue: 4 Vol. 116; ISSN 0003-6951

Publisher:: American Institute of Physics (AIP)Copyright Statement

Country of Publication:: United States

Language:: English

References (36)

Texture-based segmentation and a new cell shape index for quantitative analysis of cell spreading in AFM images: A New AFM-based Cell Shape Index Tiryaki, Volkan Müjdat; Adia-Nimuwa, Usienemnfon; Ayres, Virginia M. Cytometry Part A, Vol. 87, Issue 12 https://doi.org/10.1002/cyto.a.22739	journal	November 2015
The conformation and dynamics of P-glycoprotein in a lipid bilayer investigated by atomic force microscopy Sigdel, K. P.; Wilt, L. A.; Marsh, B. P. Biochemical Pharmacology, Vol. 156 https://doi.org/10.1016/j.bcp.2018.08.017	journal	October 2018
Supramolecular Interfacial Polymerization of Miscible Monomers: Fabricating Supramolecular Polymers with Tailor-Made Structures Qin, Bo; Zhang, Shuai; Huang, Zehuan Macromolecules, Vol. 51, Issue 5 https://doi.org/10.1021/acs.macromol.8b00289	journal	February 2018
Constraining Data Mining with Physical Models: Voltage- and Oxygen Pressure-Dependent Transport in Multiferroic Nanostructures Strelcov, Evgheni; Belianinov, Alexei; Hsieh, Ying-Hui Nano Letters, Vol. 15, Issue 10 https://doi.org/10.1021/acs.nanolett.5b02472	journal	August 2015
Phases and Interfaces from Real Space Atomically Resolved Data: Physics-Based Deep Data Image Analysis Vasudevan, Rama K.; Ziatdinov, Maxim; Jesse, Stephen Nano Letters, Vol. 16, Issue 9 https://doi.org/10.1021/acs.nanolett.6b02130	journal	August 2016
Quantitative Analysis of the Local Phase Transitions Induced by Laser Heating Ievlev, Anton V.; Susner, Michael A.; McGuire, Michael A. ACS Nano, Vol. 9, Issue 12 https://doi.org/10.1021/acsnano.5b05818	journal	October 2015
Big, Deep, and Smart Data in Scanning Probe Microscopy Kalinin, Sergei V.; Strelcov, Evgheni; Belianinov, Alex ACS Nano, Vol. 10, Issue 10 https://doi.org/10.1021/acsnano.6b04212	journal	September 2016
Direct Probe of Interplay between Local Structure and Superconductivity in FeTe _0.55 Se _0.45 Lin, Wenzhi; Li, Qing; Sales, Brian C. ACS Nano, Vol. 7, Issue 3 https://doi.org/10.1021/nn400012q	journal	March 2013
Deep Data Analysis of Conductive Phenomena on Complex Oxide Interfaces: Physics from Data Mining Strelcov, Evgheni; Belianinov, Alexei; Hsieh, Ying-Hui ACS Nano, Vol. 8, Issue 6 https://doi.org/10.1021/nn502029b	journal	May 2014
Big-Data Reflection High Energy Electron Diffraction Analysis for Understanding Epitaxial Film Growth Processes Vasudevan, Rama K.; Tselev, Alexander; Baddorf, Arthur P. ACS Nano, Vol. 8, Issue 10 https://doi.org/10.1021/nn504730n	journal	October 2014
Chemically induced Jahn–Teller ordering on manganite surfaces Gai, Zheng; Lin, Wenzhi; Burton, J. D. Nature Communications, Vol. 5, Issue 1 https://doi.org/10.1038/ncomms5528	journal	July 2014
Bond competition and phase evolution on the IrTe2 surface Li, Qing; Lin, Wenzhi; Yan, Jiaqiang Nature Communications, Vol. 5, Issue 1 https://doi.org/10.1038/ncomms6358	journal	November 2014
Identification of phases, symmetries and defects through local crystallography Belianinov, Alex; He, Qian; Kravchenko, Mikhail Nature Communications, Vol. 6, Issue 1 https://doi.org/10.1038/ncomms8801	journal	July 2015
A metal-free polymeric photocatalyst for hydrogen production from water under visible light Wang, Xinchen; Maeda, Kazuhiko; Thomas, Arne Nature Materials, Vol. 8, Issue 1 https://doi.org/10.1038/nmat2317	journal	November 2008
Ultra-high resolution imaging of thin films and single strands of polythiophene using atomic force microscopy Korolkov, Vladimir V.; Summerfield, Alex; Murphy, Alanna Nature Communications, Vol. 10, Issue 1 https://doi.org/10.1038/s41467-019-09571-6	journal	April 2019
Deep neural networks for understanding noisy data applied to physical property extraction in scanning probe microscopy Borodinov, Nikolay; Neumayer, Sabine; Kalinin, Sergei V. npj Computational Materials, Vol. 5, Issue 1 https://doi.org/10.1038/s41524-019-0148-5	journal	February 2019
Diffraction imaging of nanocrystalline structures in organic semiconductor molecular thin films Panova, Ouliana; Ophus, Colin; Takacs, Christopher J. Nature Materials, Vol. 18, Issue 8 https://doi.org/10.1038/s41563-019-0387-3	journal	June 2019
Big Data Analytics for Scanning Transmission Electron Microscopy Ptychography Jesse, S.; Chi, M.; Belianinov, A. Scientific Reports, Vol. 6, Issue 1 https://doi.org/10.1038/srep26348	journal	May 2016
Nano-domains segmentation on AFM images Luciani, X.; Patrone, L.; Courmontagne, P. Journal de Physique IV (Proceedings), Vol. 132 https://doi.org/10.1051/jp4:2006132045	journal	March 2006
Research Update: Spatially resolved mapping of electronic structure on atomic level by multivariate statistical analysis Belianinov, Alex; Ganesh, Panchapakesan; Lin, Wenzhi APL Materials, Vol. 2, Issue 12 https://doi.org/10.1063/1.4902996	journal	December 2014
Big data in reciprocal space: Sliding fast Fourier transforms for determining periodicity Vasudevan, Rama K.; Belianinov, Alex; Gianfrancesco, Anthony G. Applied Physics Letters, Vol. 106, Issue 9 https://doi.org/10.1063/1.4914016	journal	March 2015
Functional recognition imaging using artificial neural networks: applications to rapid cellular identification via broadband electromechanical response Nikiforov, M. P.; Reukov, V. V.; Thompson, G. L. Nanotechnology, Vol. 20, Issue 40 https://doi.org/10.1088/0957-4484/20/40/405708	journal	September 2009
Principal component and spatial correlation analysis of spectroscopic-imaging data in scanning probe microscopy Jesse, Stephen; Kalinin, Sergei V. Nanotechnology, Vol. 20, Issue 8 https://doi.org/10.1088/0957-4484/20/8/085714	journal	February 2009
Local crystallography analysis for atomically resolved scanning tunneling microscopy images Lin, Wenzhi; Li, Qing; Belianinov, Alexei Nanotechnology, Vol. 24, Issue 41 https://doi.org/10.1088/0957-4484/24/41/415707	journal	September 2013
Deep data mining in a real space: separation of intertwined electronic responses in a lightly doped BaFe ₂ As ₂ Ziatdinov, Maxim; Maksov, Artem; Li, Li Nanotechnology, Vol. 27, Issue 47 https://doi.org/10.1088/0957-4484/27/47/475706	journal	October 2016
Data mining graphene: correlative analysis of structure and electronic degrees of freedom in graphenic monolayers with defects Ziatdinov, Maxim; Fujii, Shintaro; Kiguchi, Manabu Nanotechnology, Vol. 27, Issue 49 https://doi.org/10.1088/0957-4484/27/49/495703	journal	November 2016
Spatially resolved mapping of disorder type and distribution in random systems using artificial neural network recognition Kumar, A.; Ovchinnikov, O.; Guo, S. Physical Review B, Vol. 84, Issue 2 https://doi.org/10.1103/PhysRevB.84.024203	journal	July 2011
Disorder Identification in Hysteresis Data: Recognition Analysis of the Random-Bond–Random-Field Ising Model Ovchinnikov, O. S.; Jesse, S.; Bintacchit, P. Physical Review Letters, Vol. 103, Issue 15 https://doi.org/10.1103/PhysRevLett.103.157203	journal	October 2009
Wavelet-driven knowledge-based MRI calf muscle segmentation Essafi, Salma; Langs, G.; Deux, J-F. 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro (ISBI) https://doi.org/10.1109/ISBI.2009.5193024	conference	June 2009
Combining Spiral Scanning and Internal Model Control for Sequential AFM Imaging at Video Rate Bazaei, Ali; Yong, Yuen Kuan; Moheimani, S. O. Reza IEEE/ASME Transactions on Mechatronics, Vol. 22, Issue 1 https://doi.org/10.1109/TMECH.2016.2574892	journal	February 2017
Video-Rate Lissajous-Scan Atomic Force Microscopy Yong, Yuen Kuan; Bazaei, Ali; Moheimani, S. O. Reza IEEE Transactions on Nanotechnology, Vol. 13, Issue 1 https://doi.org/10.1109/TNANO.2013.2292610	journal	January 2014
Big data and deep data in scanning and electron microscopies: deriving functionality from multidimensional data sets Belianinov, Alex; Vasudevan, Rama; Strelcov, Evgheni Advanced Structural and Chemical Imaging, Vol. 1, Issue 1 https://doi.org/10.1186/s40679-015-0006-6	journal	May 2015
Feature extraction via similarity search: application to atom finding and denoising in electron and scanning probe microscopy imaging Somnath, Suhas; Smith, Christopher R.; Kalinin, Sergei V. Advanced Structural and Chemical Imaging, Vol. 4, Issue 1 https://doi.org/10.1186/s40679-018-0052-y	journal	March 2018
Deep data analysis via physically constrained linear unmixing: universal framework, domain examples, and a community-wide platform Kannan, R.; Ievlev, A. V.; Laanait, N. Advanced Structural and Chemical Imaging, Vol. 4, Issue 1 https://doi.org/10.1186/s40679-018-0055-8	journal	April 2018
Deep data mining in a real space: Separation of intertwined electronic responses in a lightly-doped BaFe2As2 Ziatdinov, Maxim; Maksov, Artem; Li, Li arXiv https://doi.org/10.48550/arxiv.1609.00846	text	January 2016
Data Mining Graphene: Correlative Analysis of Structure and Electronic Degrees of Freedom in Graphenic Monolayers with Defects Ziatdinov, Maxim; Fujii, Shintaro; Kiguchi, Manabu arXiv https://doi.org/10.48550/arxiv.1609.00865	text	January 2016

Similar Records

Texture segmentation by clustering of Gabor feature vectors

Conference · 1990 · OSTI ID:5903298

Lu, Shin-yee; Hernandez, J E; Clark, G A

Leveraging unlabeled SEM datasets with self-supervised learning for enhanced particle segmentation

Journal Article · 2024 · npj Computational Materials · OSTI ID:2997777

Rettenberger, Luca; Szymanski, Nathan J; Giunto, Andrea; +5 more

Factorization-based texture segmentation

Journal Article · 2015 · IEEE Transactions on Image Processing · OSTI ID:1286702

Yuan, Jiangye; Wang, Deliang; Cheriyadat, Anil M.

Related Subjects

36 MATERIALS SCIENCE
atomic force microscopy
image processing
image segmentation
multicomponent analysis

Title: Machine learning-based multidomain processing for texture-based image segmentation and analysis

Citation Formats

References (36)

Similar Records

Related Subjects