Design of a graphical user interface for few-shot machine learning classification of electron microscopy data

Doty, Christina; Gallagher, Shaun; Cui, Wenqi; Chen, Wenya; Bhushan, Shweta; Oostrom, Marjolein T.; Akers, Sarah M.; Spurgeon, Steven R.

doi:10.1016/j.commatsci.2021.111121

Title: Design of a graphical user interface for few-shot machine learning classification of electron microscopy data

Journal Article · 03 January 2022 · Computational Materials Science

DOI: https://doi.org/10.1016/j.commatsci.2021.111121 · OSTI ID:1838120

Doty, Christina ^[1]; Gallagher, Shaun ^[1];

^[1]; Chen, Wenya ^[1]; Bhushan, Shweta ^[1]; Oostrom, Marjolein T. ^[2];

^[2];

^[2]

Univ. of Washington, Seattle, WA (United States)
Pacific Northwest National Lab. (PNNL), Richland, WA (United States)

The recent growth in data generation by modern electron microscopes requires rapid, scalable, and flexible approaches to image segmentation and analysis. Few-shot machine learning, which can richly classify images from a handful of user-provided examples, is a promising route to high-throughput analysis. However, current command-line implementations of such approaches can be slow and unintuitive to use, lacking the real-time feedback necessary to perform effective classification. Here we report on the development of a Python-based graphical user interface that enables end users to easily conduct and visualize the output of few-shot learning models. This interface is portable and can be hosted locally or on the web, providing the opportunity to reproducibly conduct, share, and crowd-source few-shot analyses.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)

Sponsoring Organization:: USDOE

Grant/Contract Number:: AC05-76RL01830

OSTI ID:: 1838120

Report Number(s):: PNNL-SA--164356

Journal Information:: Computational Materials Science, Journal Name: Computational Materials Science Vol. 203; ISSN 0927-0256

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

References (39)

Possibility of an integrated transmission electron microscope: enabling complex in-situ experiments Hattar, Khalid; Jungjohann, Katherine L. Journal of Materials Science, Vol. 56, Issue 9 https://doi.org/10.1007/s10853-020-05598-z	journal	January 2021
A computer vision approach for automated analysis and classification of microstructural image data DeCost, Brian L.; Holm, Elizabeth A. Computational Materials Science, Vol. 110 https://doi.org/10.1016/j.commatsci.2015.08.011	journal	December 2015
Informatics and data science in materials microscopy Voyles, Paul M. Current Opinion in Solid State and Materials Science, Vol. 21, Issue 3 https://doi.org/10.1016/j.cossms.2016.10.001	journal	June 2017
Role of transmission electron microscopy in the semiconductor industry for process development and failure analysis Rai, Raghaw S.; Subramanian, Swaminathan Progress in Crystal Growth and Characterization of Materials, Vol. 55, Issue 3-4 https://doi.org/10.1016/j.pcrysgrow.2009.09.002	journal	September 2009
Microstructure sensitive design for performance optimization Fullwood, David T.; Niezgoda, Stephen R.; Adams, Brent L. Progress in Materials Science, Vol. 55, Issue 6 https://doi.org/10.1016/j.pmatsci.2009.08.002	journal	August 2010
Current status and future directions for in situ transmission electron microscopy Taheri, Mitra L.; Stach, Eric A.; Arslan, Ilke Ultramicroscopy, Vol. 170 https://doi.org/10.1016/j.ultramic.2016.08.007	journal	November 2016
The impact of STEM aberration correction on materials science Pennycook, Stephen J. Ultramicroscopy, Vol. 180 https://doi.org/10.1016/j.ultramic.2017.03.020	journal	September 2017
Characterisation of the Medipix3 detector for 60 and 80 keV electrons Mir, J. A.; Clough, R.; MacInnes, R. Ultramicroscopy, Vol. 182 https://doi.org/10.1016/j.ultramic.2017.06.010	journal	November 2017
Hybrid pixel direct detector for electron energy loss spectroscopy Plotkin-Swing, Benjamin; Corbin, George J.; De Carlo, Sacha Ultramicroscopy, Vol. 217 https://doi.org/10.1016/j.ultramic.2020.113067	journal	October 2020
High Dynamic Range Pixel Array Detector for Scanning Transmission Electron Microscopy Tate, Mark W.; Purohit, Prafull; Chamberlain, Darol Microscopy and Microanalysis, Vol. 22, Issue 1 https://doi.org/10.1017/S1431927615015664	journal	January 2016
High Throughput Quantitative Metallography for Complex Microstructures Using Deep Learning: A Case Study in Ultrahigh Carbon Steel DeCost, Brian L.; Lei, Bo; Francis, Toby Microscopy and Microanalysis, Vol. 25, Issue 1 https://doi.org/10.1017/S1431927618015635	journal	February 2019
Phases and Interfaces from Real Space Atomically Resolved Data: Physics-Based Deep Data Image Analysis Vasudevan, Rama K.; Ziatdinov, Maxim; Jesse, Stephen Nano Letters, Vol. 16, Issue 9 https://doi.org/10.1021/acs.nanolett.6b02130	journal	August 2016
Knowledge Extraction from Atomically Resolved Images Vlcek, Lukas; Maksov, Artem; Pan, Minghu ACS Nano, Vol. 11, Issue 10 https://doi.org/10.1021/acsnano.7b05036	journal	October 2017
Integration of SrTiO ₃ on Crystallographically Oriented Epitaxial Germanium for Low-Power Device Applications Hudait, Mantu K.; Clavel, Michael; Zhu, Yan ACS Applied Materials & Interfaces, Vol. 7, Issue 9 https://doi.org/10.1021/am5091726	journal	February 2015
Improved reproducibility by assuring confidence in measurements in biomedical research Plant, Anne L.; Locascio, Laurie E.; May, Willie E. Nature Methods, Vol. 11, Issue 9 https://doi.org/10.1038/nmeth.3076	journal	August 2014
Reproducibility, sharing and progress in nanomaterial databases Tropsha, Alexander; Mills, Karmann C.; Hickey, Anthony J. Nature Nanotechnology, Vol. 12, Issue 12 https://doi.org/10.1038/nnano.2017.233	journal	December 2017
50 years of scanning electron microscopy of bone—a comprehensive overview of the important discoveries made and insights gained into bone material properties in health, disease, and taphonomy Shah, Furqan A.; Ruscsák, Krisztina; Palmquist, Anders Bone Research, Vol. 7, Issue 1 https://doi.org/10.1038/s41413-019-0053-z	journal	May 2019
Learning surface molecular structures via machine vision Ziatdinov, Maxim; Maksov, Artem; Kalinin, Sergei V. npj Computational Materials, Vol. 3, Issue 1 https://doi.org/10.1038/s41524-017-0038-7	journal	August 2017
Mapping mesoscopic phase evolution during E-beam induced transformations via deep learning of atomically resolved images Vasudevan, Rama K.; Laanait, Nouamane; Ferragut, Erik M. npj Computational Materials, Vol. 4, Issue 1 https://doi.org/10.1038/s41524-018-0086-7	journal	June 2018
Rapid and flexible segmentation of electron microscopy data using few-shot machine learning Akers, Sarah; Kautz, Elizabeth; Trevino-Gavito, Andrea npj Computational Materials, Vol. 7, Issue 1 https://doi.org/10.1038/s41524-021-00652-z	journal	November 2021
Towards data-driven next-generation transmission electron microscopy Spurgeon, Steven R.; Ophus, Colin; Jones, Lewys Nature Materials https://doi.org/10.1038/s41563-020-00833-z	journal	October 2020
Software tools for automated transmission electron microscopy Schorb, Martin; Haberbosch, Isabella; Hagen, Wim J. H. Nature Methods, Vol. 16, Issue 6 https://doi.org/10.1038/s41592-019-0396-9	journal	May 2019
Aachen-Heerlen annotated steel microstructure dataset Iren, Deniz; Ackermann, Marc; Gorfer, Julian Scientific Data, Vol. 8, Issue 1 https://doi.org/10.1038/s41597-021-00926-7	journal	May 2021
Diverse applications of MoO ₃ for high performance organic photovoltaics: fundamentals, processes and optimization strategies Gong, Yongshuai; Dong, Yiman; Zhao, Biao Journal of Materials Chemistry A, Vol. 8, Issue 3 https://doi.org/10.1039/C9TA12005J	journal	January 2020
The effects of core-level broadening in determining band alignment at the epitaxial SrTiO ₃ (001)/ p -Ge(001) heterojunction Chambers, Scott A.; Du, Yingge; Comes, Ryan B. Applied Physics Letters, Vol. 110, Issue 8 https://doi.org/10.1063/1.4977422	journal	February 2017
Detectors—The ongoing revolution in scanning transmission electron microscopy and why this important to material characterization MacLaren, Ian; Macgregor, Thomas A.; Allen, Christopher S. APL Materials, Vol. 8, Issue 11 https://doi.org/10.1063/5.0026992	journal	November 2020
Rich multi-dimensional correlative imaging Withers, Philip J.; Burnett, Timothy L. IOP Conference Series: Materials Science and Engineering, Vol. 580, Issue 1 https://doi.org/10.1088/1757-899X/580/1/012014	journal	August 2019
Deep learning in electron microscopy Ede, Jeffrey M. Machine Learning: Science and Technology, Vol. 2, Issue 1 https://doi.org/10.1088/2632-2153/abd614	journal	March 2021
Trainable Weka Segmentation: a machine learning tool for microscopy pixel classification Arganda-Carreras, Ignacio; Kaynig, Verena; Rueden, Curtis Bioinformatics, Vol. 33, Issue 15 https://doi.org/10.1093/bioinformatics/btx180	journal	March 2017
D³ Data-Driven Documents Bostock, M.; Ogievetsky, V.; Heer, J. IEEE Transactions on Visualization and Computer Graphics, Vol. 17, Issue 12 https://doi.org/10.1109/TVCG.2011.185	journal	December 2011
Vega-Lite: A Grammar of Interactive Graphics Satyanarayan, Arvind; Moritz, Dominik; Wongsuphasawat, Kanit IEEE Transactions on Visualization and Computer Graphics, Vol. 23, Issue 1 https://doi.org/10.1109/TVCG.2016.2599030	journal	January 2017
Decoding crystallography from high-resolution electron imaging and diffraction datasets with deep learning Aguiar, J. A.; Gong, M. L.; Unocic, R. R. Science Advances, Vol. 5, Issue 10 https://doi.org/10.1126/sciadv.aaw1949	journal	October 2019
Telling humans and computers apart automatically von Ahn, Luis; Blum, Manuel; Langford, John Communications of the ACM, Vol. 47, Issue 2 https://doi.org/10.1145/966389.966390	journal	February 2004
Aberration-corrected scanning transmission electron microscopy for atomic-resolution studies of functional oxides MacLaren, Ian; Ramasse, Quentin M. International Materials Reviews, Vol. 59, Issue 3 https://doi.org/10.1179/1743280413Y.0000000026	journal	March 2014
Big data and deep data in scanning and electron microscopies: deriving functionality from multidimensional data sets Belianinov, Alex; Vasudevan, Rama; Strelcov, Evgheni Advanced Structural and Chemical Imaging, Vol. 1, Issue 1 https://doi.org/10.1186/s40679-015-0006-6	journal	May 2015
Deep data analysis via physically constrained linear unmixing: universal framework, domain examples, and a community-wide platform Kannan, R.; Ievlev, A. V.; Laanait, N. Advanced Structural and Chemical Imaging, Vol. 4, Issue 1 https://doi.org/10.1186/s40679-018-0055-8	journal	April 2018
Lab on a beam—Big data and artificial intelligence in scanning transmission electron microscopy Kalinin, Sergei V.; Lupini, Andrew R.; Dyck, Ondrej MRS Bulletin, Vol. 44, Issue 7 https://doi.org/10.1557/mrs.2019.159	journal	July 2019
Materials Advances through Aberration-Corrected Electron Microscopy Pennycook, S. J.; Varela, M.; Hetherington, C. J. D. MRS Bulletin, Vol. 31, Issue 1 https://doi.org/10.1557/mrs2006.4	journal	January 2006
Aachen-Heerlen Annotated Steel Microstructure Dataset Iren, Deniz; Ackerman, Marc; Gorfer, Julian figshare https://doi.org/10.6084/m9.figshare.c.5185004.v1	collection	January 2021

Similar Records

Rapid and flexible segmentation of electron microscopy data using few-shot machine learning

Journal Article · 2021 · npj Computational Materials · OSTI ID:1830837

Akers, Sarah; Kautz, Elizabeth; Trevino-Gavito, Andrea; +5 more

A hypertext display component for a graphical user interface development environment

Conference · 1995 · OSTI ID:195712

Love, R J

Improving Machine Learning Explainability with a Graphical User Interface

Conference · 2024 · OSTI ID:2371868

Walker, Cody McBroom

Related Subjects

97 MATHEMATICS AND COMPUTING
Few-shot
Graphical user interface
Machine learning
Segmentation
Sparse data analytics
Transmission electron microscopy

Title: Design of a graphical user interface for few-shot machine learning classification of electron microscopy data

Citation Formats

References (39)

Similar Records

Related Subjects