Ingrained: An Automated Framework for Fusing Atomic-Scale Image Simulations into Experiments

Schwenker, Eric; Kolluru, Venkata Surya Chaitanya; Guo, Jinglong; Zhang, Rui; Hu, Xiaobing; Li, Qiucheng; Paul, Joshua T.; Hersam, Mark C.; Dravid, Vinayak P.; Klie, Robert; Guest, Jeffrey R.; Chan, Maria K. Y.

doi:10.1002/smll.202102960

Title: Ingrained: An Automated Framework for Fusing Atomic-Scale Image Simulations into Experiments

Journal Article · 05 April 2022 · Small

DOI: https://doi.org/10.1002/smll.202102960 · OSTI ID:1894219

^[1];

^[2];

^[3];

^[4];

^[5]; Li, Qiucheng ^[5];

^[1];

^[5];

^[3];

^[4];

^[4]

Argonne National Lab. (ANL), Lemont, IL (United States). Center for Nanoscale Materials; Northwestern Univ., Evanston, IL (United States)
Argonne National Lab. (ANL), Lemont, IL (United States). Center for Nanoscale Materials; Univ. of Florida, Gainesville, FL (United States)
Univ. of Illinois, Chicago, IL (United States)
Argonne National Lab. (ANL), Lemont, IL (United States). Center for Nanoscale Materials
Northwestern Univ., Evanston, IL (United States)

Abstract To fully leverage the power of image simulation to corroborate and explain patterns and structures in atomic resolution microscopy, an initial correspondence between the simulation and experimental image must be established at the outset of further high accuracy simulations or calculations. Furthermore, if simulation is to be used in context of highly automated processes or high‐throughput optimization, the process of finding this correspondence itself must be automated. In this work, “ingrained,” an open‐source automation framework which solves for this correspondence and fuses atomic resolution image simulations into the experimental images to which they correspond, is introduced. Herein, the overall “ingrained” workflow, focusing on its application to interface structure approximations, and the development of an experimentally rationalized forward model for scanning tunneling microscopy simulation are described.

View Accepted Manuscript (DOE)

View Accepted Manuscript (Publisher)

Cite

Export

Save

Research Organization:: Argonne National Laboratory (ANL), Argonne, IL (United States). Center for Nanoscale Materials (CNM); Energy Frontier Research Centers (EFRC) (United States). Center for Electrochemical Energy Science (CEES); Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). National Energy Research Scientific Computing Center (NERSC); Argonne National Laboratory (ANL), Argonne, IL (United States). Laboratory Computing Resource Center (LCRC)

Sponsoring Organization:: Northwestern University - Materials Research Science and Engineering Center (NU-MRSEC); National Science Foundation (NSF); USDOE Office of Science (SC), Basic Energy Sciences (BES). Scientific User Facilities Division; USDOE

Grant/Contract Number:: AC02-06CH11357; AC02-05CH11231

OSTI ID:: 1894219

Alternate ID(s):: OSTI ID: 1863100

Journal Information:: Small, Vol. 18, Issue 19; ISSN 1613-6810

Publisher:: WileyCopyright Statement

Country of Publication:: United States

Language:: English

References (41)

SciPy 1.0: fundamental algorithms for scientific computing in Python Virtanen, Pauli; Gommers, Ralf; Oliphant, Travis E. Nature Methods https://doi.org/10.1038/s41592-019-0686-2	journal	February 2020
Robust nonrigid registration to capture brain shift from intraoperative MRI Clatz, O.; Delingette, H.; Talos, I. -F. IEEE Transactions on Medical Imaging, Vol. 24, Issue 11 https://doi.org/10.1109/TMI.2005.856734	journal	November 2005
Least-Squares Fitting of Two 3-D Point Sets Arun, K. S.; Huang, T. S.; Blostein, S. D. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. PAMI-9, Issue 5 https://doi.org/10.1109/TPAMI.1987.4767965	journal	September 1987
scikit-image: image processing in Python van der Walt, Stéfan; Schönberger, Johannes L.; Nunez-Iglesias, Juan PeerJ, Vol. 2 https://doi.org/10.7717/peerj.453	journal	January 2014
Imaging Catalytic Activation of CO ₂ on Cu ₂ O (110): A First-Principles Study Li, Liang; Zhang, Rui; Vinson, John Chemistry of Materials, Vol. 30, Issue 6 https://doi.org/10.1021/acs.chemmater.7b04803	journal	March 2018
USID and Pycroscopy – Open Source Frameworks for Storing and Analyzing Imaging and Spectroscopy Data Somnath, Suhas; Smith, Chris R.; Laanait, Nouamane Microscopy and Microanalysis, Vol. 25, Issue S2 https://doi.org/10.1017/S1431927619001831	journal	August 2019
An image registration algorithm based on phase correlation and the classical Lucas–Kanade technique Douini, Youssef; Riffi, Jamal; Mahraz, Adnane Mohamed Signal, Image and Video Processing, Vol. 11, Issue 7 https://doi.org/10.1007/s11760-017-1089-4	journal	May 2017
Efficiency of ab-initio total energy calculations for metals and semiconductors using a plane-wave basis set Kresse, G.; Furthmüller, J. Computational Materials Science, Vol. 6, Issue 1, p. 15-50 https://doi.org/10.1016/0927-0256(96)00008-0	journal	July 1996
Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography Fischler, Martin A.; Bolles, Robert C. Communications of the ACM, Vol. 24, Issue 6 https://doi.org/10.1145/358669.358692	journal	June 1981
CO on Pt(111): binding site assignment from the interplay between measured and calculated STM images Pedersen, M. Ø; Bocquet, M. -L; Sautet, P. Chemical Physics Letters, Vol. 299, Issue 5 https://doi.org/10.1016/S0009-2614(98)01318-9	journal	January 1999
Efficient and reliable method for the simulation of scanning tunneling images and spectra with local basis sets Paz, Óscar; Soler, José M. physica status solidi (b), Vol. 243, Issue 5 https://doi.org/10.1002/pssb.200541453	journal	April 2006
Quantitative composition determination at the atomic level using model-based high-angle annular dark field scanning transmission electron microscopy Martinez, G. T.; Rosenauer, A.; De Backer, A. Ultramicroscopy, Vol. 137 https://doi.org/10.1016/j.ultramic.2013.11.001	journal	February 2014
Efficient subpixel image registration algorithms Guizar-Sicairos, Manuel; Thurman, Samuel T.; Fienup, James R. Optics Letters, Vol. 33, Issue 2 https://doi.org/10.1364/OL.33.000156	journal	January 2008
Composition determination of semiconductor alloys towards atomic accuracy by HAADF-STEM Duschek, L.; Kükelhan, P.; Beyer, A. Ultramicroscopy, Vol. 200 https://doi.org/10.1016/j.ultramic.2019.02.009	journal	May 2019
Synthesis of borophane polymorphs through hydrogenation of borophene Li, Qiucheng; Kolluru, Venkata Surya Chaitanya; Rahn, Matthew S. Science, Vol. 371, Issue 6534 https://doi.org/10.1126/science.abg1874	journal	March 2021
Efficient iterative schemes for ab initio total-energy calculations using a plane-wave basis set Kresse, G.; Furthmüller, J. Physical Review B, Vol. 54, Issue 16, p. 11169-11186 https://doi.org/10.1103/PhysRevB.54.11169	journal	October 1996
A Simulated Atomic-resolution HAADF STEM Imaging Dataset Containing Unique ICSD Structure Prototypes Schwenker, Eric; Sen, Fatih; Wolverton, Chris Materials Data Facility https://doi.org/10.18126/SZEQ-YDE5	dataset	January 2020
Picometre-precision analysis of scanning transmission electron microscopy images of platinum nanocatalysts Yankovich, Andrew B.; Berkels, Benjamin; Dahmen, W. Nature Communications, Vol. 5, Issue 1 https://doi.org/10.1038/ncomms5155	journal	June 2014
Tip and Surface Determination from Experiments and Simulations of Scanning Tunneling Microscopy and Spectroscopy Paz, Óscar; Brihuega, Iván; Gómez-Rodríguez, José M. Physical Review Letters, Vol. 94, Issue 5 https://doi.org/10.1103/PhysRevLett.94.056103	journal	February 2005
Commentary: The Materials Project: A materials genome approach to accelerating materials innovation Jain, Anubhav; Ong, Shyue Ping; Hautier, Geoffroy APL Materials, Vol. 1, Issue 1 https://doi.org/10.1063/1.4812323	journal	July 2013
Distinctive Image Features from Scale-Invariant Keypoints Lowe, David G. International Journal of Computer Vision, Vol. 60, Issue 2, p. 91-110 https://doi.org/10.1023/B:VISI.0000029664.99615.94	journal	November 2004
Atomic-Scale Investigation of the Borides Precipitated in a Transient Liquid Phase-Bonded Ni-Based Superalloy Hu, X. B.; Sheng, N. C.; Zhu, Y. M. Metallurgical and Materials Transactions A, Vol. 51, Issue 4 https://doi.org/10.1007/s11661-020-05646-y	journal	January 2020
Strain-Energy Release in Bent Semiconductor Nanowires Occurring by Polygonization or Nanocrack Formation Sun, Zhiyuan; Huang, Chunyi; Guo, Jinglong ACS Nano, Vol. 13, Issue 3 https://doi.org/10.1021/acsnano.9b01231	journal	February 2019
Theory of the scanning tunneling microscope Tersoff, J.; Hamann, D. R. Physical Review B, Vol. 31, Issue 2 https://doi.org/10.1103/PhysRevB.31.805	journal	January 1985
A fast image simulation algorithm for scanning transmission electron microscopy Ophus, Colin Advanced Structural and Chemical Imaging, Vol. 3, Issue 1 https://doi.org/10.1186/s40679-017-0046-1	journal	May 2017
Electron Localization Determines Defect Formation on Ceria Substrates Esch, F. Science, Vol. 309, Issue 5735 https://doi.org/10.1126/science.1111568	journal	July 2005
Image Quality Assessment: From Error Visibility to Structural Similarity Wang, Z.; Bovik, A. C.; Sheikh, H. R. IEEE Transactions on Image Processing, Vol. 13, Issue 4 https://doi.org/10.1109/TIP.2003.819861	journal	April 2004
An efficient method for finding the minimum of a function of several variables without calculating derivatives Powell, M. J. D. The Computer Journal, Vol. 7, Issue 2 https://doi.org/10.1093/comjnl/7.2.155	journal	February 1964
Correcting the linear and nonlinear distortions for atomically resolved STEM spectrum and diffraction imaging Wang, Yi; Suyolcu, Y. Eren; Salzberger, Ute Microscopy, Vol. 67, Issue suppl_1 https://doi.org/10.1093/jmicro/dfy002	journal	January 2018
Effect of selenium and chlorine co-passivation in polycrystalline CdSeTe devices Guo, Jinglong; Mannodi-Kanakkithodi, Arun; Sen, Fatih G. Applied Physics Letters, Vol. 115, Issue 15 https://doi.org/10.1063/1.5123169	journal	October 2019
Advanced Computing in Electron Microscopy Kirkland, Earl J. https://doi.org/10.1007/978-1-4757-4406-4	book	January 1998
Theories of scanning probe microscopes at the atomic scale Hofer, Werner A.; Foster, Adam S.; Shluger, Alexander L. Reviews of Modern Physics, Vol. 75, Issue 4 https://doi.org/10.1103/RevModPhys.75.1287	journal	October 2003
A streaming multi-GPU implementation of image simulation algorithms for scanning transmission electron microscopy Pryor, Alan; Ophus, Colin; Miao, Jianwei Advanced Structural and Chemical Imaging, Vol. 3, Issue 1 https://doi.org/10.1186/s40679-017-0048-z	journal	October 2017
The scattering of electrons by atoms and crystals. I. A new theoretical approach Cowley, J. M.; Moodie, A. F. Acta Crystallographica, Vol. 10, Issue 10, p. 609-619 https://doi.org/10.1107/S0365110X57002194	journal	October 1957
Smart Align—a new tool for robust non-rigid registration of scanning microscope data Jones, Lewys; Yang, Hao; Pennycook, Timothy J. Advanced Structural and Chemical Imaging, Vol. 1, Issue 1 https://doi.org/10.1186/s40679-015-0008-4	journal	July 2015
Understanding the phase contrast optics to restore artifact-free microscopy images for segmentation Yin, Zhaozheng; Kanade, Takeo; Chen, Mei Medical Image Analysis, Vol. 16, Issue 5 https://doi.org/10.1016/j.media.2011.12.006	journal	July 2012
Modeling realistic tip structures: Scanning tunneling microscopy of NO adsorption on Rh(111) Hagelaar, J. H. A.; Flipse, C. F. J.; Cerdá, J. I. Physical Review B, Vol. 78, Issue 16 https://doi.org/10.1103/PhysRevB.78.161405	journal	October 2008
Measurement of specimen thickness and composition in AlxGa1-xN/GaN using high-angle annular dark field images Rosenauer, Andreas; Gries, Katharina; Müller, Knut Ultramicroscopy, Vol. 109, Issue 9 https://doi.org/10.1016/j.ultramic.2009.05.003	journal	August 2009
A pyramid approach to subpixel registration based on intensity Thevenaz, P.; Ruttimann, U. E.; Unser, M. IEEE Transactions on Image Processing, Vol. 7, Issue 1 https://doi.org/10.1109/83.650848	journal	January 1998
Determination of the chemical composition of GaNAs using STEM HAADF imaging and STEM strain state analysis Grieb, Tim; Müller, Knut; Fritz, Rafael Ultramicroscopy, Vol. 117 https://doi.org/10.1016/j.ultramic.2012.03.014	journal	June 2012
Atom counting in HAADF STEM using a statistical model-based approach: Methodology, possibilities, and inherent limitations De Backer, A.; Martinez, G. T.; Rosenauer, A. Ultramicroscopy, Vol. 134 https://doi.org/10.1016/j.ultramic.2013.05.003	journal	November 2013