Unraveling the impact of initial choices and in-loop interventions on learning dynamics in autonomous scanning probe microscopy

Slautin, Boris N.; Liu, Yongtao; Funakubo, Hiroshi; Kalinin, Sergei V.

doi:10.1063/5.0198316

Title: Unraveling the impact of initial choices and in-loop interventions on learning dynamics in autonomous scanning probe microscopy

Journal Article · 15 April 2024 · Journal of Applied Physics

DOI: https://doi.org/10.1063/5.0198316 · OSTI ID:2439881

^[1];

^[2];

^[3];

^[4]

Independent researcher, Belgrade (Serbia)
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Center for Nanophase Materials Sciences (CNMS)
Tokyo Institute of Technology (Japan)
Univ. of Tennessee, Knoxville, TN (United States)

The current focus in Autonomous Experimentation (AE) is on developing robust workflows to conduct the AE effectively. This entails the need for well-defined approaches to guide the AE process, including strategies for hyperparameter tuning and high-level human interventions within the workflow loop. This paper presents a comprehensive analysis of the influence of initial experimental conditions and in-loop interventions on the learning dynamics of Deep Kernel Learning (DKL) within the realm of AE in scanning probe microscopy. We explore the concept of the “seed effect,” where the initial experiment setup has a substantial impact on the subsequent learning trajectory. Additionally, we introduce an approach of the seed point interventions in AE allowing the operator to influence the exploration process. Using a dataset from Piezoresponse Force Microscopy on PbTiO₃ thin films, we illustrate the impact of the “seed effect” and in-loop seed interventions on the effectiveness of DKL in predicting material properties. The study highlights the importance of initial choices and adaptive interventions in optimizing learning rates and enhancing the efficiency of automated material characterization. This work offers valuable insights into designing more robust and effective AE workflows in microscopy with potential applications across various characterization techniques.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Basic Energy Sciences (BES). Scientific User Facilities (SUF)

Grant/Contract Number:: AC05-00OR22725

OSTI ID:: 2439881

Journal Information:: Journal of Applied Physics, Journal Name: Journal of Applied Physics Journal Issue: 15 Vol. 135; ISSN 0021-8979

Publisher:: American Institute of Physics (AIP)Copyright Statement

Country of Publication:: United States

Language:: English

References (40)

Self‐Driving Platform for Metal Nanoparticle Synthesis: Combining Microfluidics and Machine Learning Tao, Huachen; Wu, Tianyi; Kheiri, Sina Advanced Functional Materials, Vol. 31, Issue 51 https://doi.org/10.1002/adfm.202106725	journal	September 2021
A Processing and Analytics System for Microscopy Data Workflows: The Pycroscopy Ecosystem of Packages Vasudevan, Rama Krishnan; Valleti, Sai Mani; Ziatdinov, Maxim Advanced Theory and Simulations, Vol. 6, Issue 11 https://doi.org/10.1002/adts.202300247	journal	September 2023
Physics Discovery in Nanoplasmonic Systems via Autonomous Experiments in Scanning Transmission Electron Microscopy Roccapriore, Kevin M.; Kalinin, Sergei V.; Ziatdinov, Maxim Advanced Science, Vol. 9, Issue 36 https://doi.org/10.1002/advs.202203422	journal	November 2022
Automated Experiments of Local Non‐Linear Behavior in Ferroelectric Materials Liu, Yongtao; Kelley, Kyle P.; Vasudevan, Rama K. Small, Vol. 18, Issue 48 https://doi.org/10.1002/smll.202204130	journal	October 2022
Recent advances in autonomous synthesis of materials Li, Yucen; Xia, Lingling; Fan, Yiming ChemPhysMater, Vol. 1, Issue 2 https://doi.org/10.1016/j.chphma.2021.10.002	journal	April 2022
Machine learning for high-throughput experimental exploration of metal halide perovskites Ahmadi, Mahshid; Ziatdinov, Maxim; Zhou, Yuanyuan Joule, Vol. 5, Issue 11 https://doi.org/10.1016/j.joule.2021.10.001	journal	November 2021
Autonomous experimentation systems for materials development: A community perspective Stach, Eric; DeCost, Brian; Kusne, A. Gilad Matter, Vol. 4, Issue 9 https://doi.org/10.1016/j.matt.2021.06.036	journal	September 2021
Autonomous scanning probe microscopy with hypothesis learning: Exploring the physics of domain switching in ferroelectric materials Liu, Yongtao; Morozovska, Anna N.; Eliseev, Eugene A. Patterns, Vol. 4, Issue 3 https://doi.org/10.1016/j.patter.2023.100704	journal	March 2023
Explainability and human intervention in autonomous scanning probe microscopy Liu, Yongtao; Ziatdinov, Maxim A.; Vasudevan, Rama K. Patterns, Vol. 4, Issue 11 https://doi.org/10.1016/j.patter.2023.100858	journal	November 2023
Correction of AFM data artifacts using a convolutional neural network trained with synthetically generated data Kocur, Viktor; Hegrová, Veronika; Patočka, Marek Ultramicroscopy, Vol. 246 https://doi.org/10.1016/j.ultramic.2022.113666 113666	journal	April 2023
Autonomous Chemical Experiments: Challenges and Perspectives on Establishing a Self-Driving Lab Seifrid, Martin; Pollice, Robert; Aguilar-Granda, Andrés Accounts of Chemical Research, Vol. 55, Issue 17 https://doi.org/10.1021/acs.accounts.2c00220	journal	August 2022
Exploring the Relationship of Microstructure and Conductivity in Metal Halide Perovskites via Active Learning-Driven Automated Scanning Probe Microscopy Liu, Yongtao; Yang, Jonghee; Vasudevan, Rama K. The Journal of Physical Chemistry Letters, Vol. 14, Issue 13 https://doi.org/10.1021/acs.jpclett.3c00223	journal	March 2023
Automated Searching and Identification of Self-Organized Nanostructures Gordon, Oliver M.; Hodgkinson, Jo E. A.; Farley, Steff M. Nano Letters, Vol. 20, Issue 10 https://doi.org/10.1021/acs.nanolett.0c03213	journal	August 2020
High-Throughput and Autonomous Grazing Incidence X-ray Diffraction Mapping of Organic Combinatorial Thin-Film Library Driven by Machine Learning Maruyama, Shingo; Ouchi, Kana; Koganezawa, Tomoyuki ACS Combinatorial Science, Vol. 22, Issue 7 https://doi.org/10.1021/acscombsci.0c00037	journal	June 2020
Automated Experiment in 4D-STEM: Exploring Emergent Physics and Structural Behaviors Roccapriore, Kevin M.; Dyck, Ondrej; Oxley, Mark P. ACS Nano, Vol. 16, Issue 5 https://doi.org/10.1021/acsnano.1c11118	journal	April 2022
Bayesian Active Learning for Scanning Probe Microscopy: From Gaussian Processes to Hypothesis Learning Ziatdinov, Maxim; Liu, Yongtao; Kelley, Kyle ACS Nano, Vol. 16, Issue 9 https://doi.org/10.1021/acsnano.2c05303	journal	September 2022
Autonomous Scanning Probe Microscopy in Situ Tip Conditioning through Machine Learning Rashidi, Mohammad; Wolkow, Robert A. ACS Nano, Vol. 12, Issue 6 https://doi.org/10.1021/acsnano.8b02208	journal	May 2018
Autonomous platforms for data-driven organic synthesis Gao, Wenhao; Raghavan, Priyanka; Coley, Connor W. Nature Communications, Vol. 13, Issue 1 https://doi.org/10.1038/s41467-022-28736-4	journal	February 2022
Deep neural networks for understanding noisy data applied to physical property extraction in scanning probe microscopy Borodinov, Nikolay; Neumayer, Sabine; Kalinin, Sergei V. npj Computational Materials, Vol. 5, Issue 1 https://doi.org/10.1038/s41524-019-0148-5	journal	February 2019
Recent advances and applications of deep learning methods in materials science Choudhary, Kamal; DeCost, Brian; Chen, Chi npj Computational Materials, Vol. 8, Issue 1 https://doi.org/10.1038/s41524-022-00734-6	journal	April 2022
Autonomous scanning probe microscopy investigations over WS2 and Au{111} Thomas, John C.; Rossi, Antonio; Smalley, Darian npj Computational Materials, Vol. 8, Issue 1 https://doi.org/10.1038/s41524-022-00777-9	journal	May 2022
A dynamic Bayesian optimized active recommender system for curiosity-driven partially Human-in-the-loop automated experiments Biswas, Arpan; Liu, Yongtao; Creange, Nicole npj Computational Materials, Vol. 10, Issue 1 https://doi.org/10.1038/s41524-023-01191-5	journal	February 2024
Artificial-intelligence-driven scanning probe microscopy Krull, A.; Hirsch, P.; Rother, C. Communications Physics, Vol. 3, Issue 1 https://doi.org/10.1038/s42005-020-0317-3	journal	March 2020
Gaussian processes for autonomous data acquisition at large-scale synchrotron and neutron facilities Noack, Marcus M.; Zwart, Petrus H.; Ushizima, Daniela M. Nature Reviews Physics, Vol. 3, Issue 10 https://doi.org/10.1038/s42254-021-00345-y	journal	July 2021
Experimental discovery of structure–property relationships in ferroelectric materials via active learning Liu, Yongtao; Kelley, Kyle P.; Vasudevan, Rama K. Nature Machine Intelligence, Vol. 4, Issue 4 https://doi.org/10.1038/s42256-022-00460-0	journal	April 2022
The rise of self-driving labs in chemical and materials sciences Abolhasani, Milad; Kumacheva, Eugenia Nature Synthesis, Vol. 2, Issue 6 https://doi.org/10.1038/s44160-022-00231-0	journal	January 2023
An artificial intelligence atomic force microscope enabled by machine learning Huang, Boyuan; Li, Zhenghao; Li, Jiangyu Nanoscale, Vol. 10, Issue 45 https://doi.org/10.1039/C8NR06734A	journal	January 2018
Accelerated AI development for autonomous materials synthesis in flow Epps, Robert W.; Volk, Amanda A.; Reyes, Kristofer G. Chemical Science, Vol. 12, Issue 17 https://doi.org/10.1039/D0SC06463G	journal	January 2021
Enabling autonomous scanning probe microscopy imaging of single molecules with deep learning Sotres, Javier; Boyd, Hannah; Gonzalez-Martinez, Juan F. Nanoscale, Vol. 13, Issue 20 https://doi.org/10.1039/D1NR01109J	journal	January 2021
Quantitative mapping of switching behavior in piezoresponse force microscopy Jesse, Stephen; Lee, Ho Nyung; Kalinin, Sergei V. Review of Scientific Instruments, Vol. 77, Issue 7 https://doi.org/10.1063/1.2214699	journal	July 2006
On-the-fly autonomous control of neutron diffraction via physics-informed Bayesian active learning McDannald, Austin; Frontzek, Matthias; Savici, Andrei T. Applied Physics Reviews, Vol. 9, Issue 2 https://doi.org/10.1063/5.0082956	journal	June 2022
Scanning probe microscopy in the age of machine learning Rahman Laskar, Md Ashiqur; Celano, Umberto APL Machine Learning, Vol. 1, Issue 4 https://doi.org/10.1063/5.0160568	journal	November 2023
Disentangling ferroelectric domain wall geometries and pathways in dynamic piezoresponse force microscopy via unsupervised machine learning Kalinin, Sergei V.; Steffes, James J.; Liu, Yongtao Nanotechnology, Vol. 33, Issue 5 https://doi.org/10.1088/1361-6528/ac2f5b	journal	November 2021
Probe microscopy is all you need ^* Kalinin, Sergei V.; Vasudevan, Rama; Liu, Yongtao Machine Learning: Science and Technology, Vol. 4, Issue 2 https://doi.org/10.1088/2632-2153/acccd5	journal	May 2023
Human-in-the-Loop: The Future of Machine Learning in Automated Electron Microscopy Kalinin, Sergei V.; Liu, Yongtao; Biswas, Arpan Microscopy Today, Vol. 32, Issue 1 https://doi.org/10.1093/mictod/qaad096	journal	February 2024
Automated structure discovery in atomic force microscopy Alldritt, Benjamin; Hapala, Prokop; Oinonen, Niko Science Advances, Vol. 6, Issue 9 https://doi.org/10.1126/sciadv.aay6913	journal	February 2020
Exploring order parameters and dynamic processes in disordered systems via variational autoencoders Kalinin, Sergei V.; Dyck, Ondrej; Jesse, Stephen Science Advances, Vol. 7, Issue 17 https://doi.org/10.1126/sciadv.abd5084	journal	April 2021
Data-Driven Design and Autonomous Experimentation in Soft and Biological Materials Engineering Ferguson, Andrew L.; Brown, Keith A. Annual Review of Chemical and Biomolecular Engineering, Vol. 13, Issue 1 https://doi.org/10.1146/annurev-chembioeng-092120-020803	journal	March 2022
Materials science in the artificial intelligence age: high-throughput library generation, machine learning, and a pathway from correlations to the underpinning physics Vasudevan, Rama K.; Choudhary, Kamal; Mehta, Apurva MRS Communications, Vol. 9, Issue 3 https://doi.org/10.1557/mrc.2019.95	journal	July 2019
An Introduction to Variational Autoencoders Kingma, Diederik P.; Welling, Max Foundations and Trends® in Machine Learning, Vol. 12, Issue 4 https://doi.org/10.1561/2200000056	journal	January 2019

Similar Records

Explainability and human intervention in autonomous scanning probe microscopy

Journal Article · 2023 · Patterns · OSTI ID:2059619

Liu, Yongtao; Ziatdinov, Maxim A.; Vasudevan, Rama K.; +1 more

Learning the right channel in multimodal imaging: automated experiment in piezoresponse force microscopy

Journal Article · 2023 · npj Computational Materials · OSTI ID:1960429

Liu, Yongtao; Vasudevan, Rama K.; Kelley, Kyle P.; +3 more

Exploring Causal Physical Mechanisms via Non-Gaussian Linear Models and Deep Kernel Learning: Applications for Ferroelectric Domain Structures

Journal Article · 2021 · ACS Nano · OSTI ID:1869103

Liu, Yongtao; Ziatdinov, Maxim; Kalinin, Sergei V.

Related Subjects

47 OTHER INSTRUMENTATION
Gaussian processes
Machine learning
Material characterization methods
Piezoresponse force spectroscopy
Scanning probe microscopy

Title: Unraveling the impact of initial choices and in-loop interventions on learning dynamics in autonomous scanning probe microscopy

Citation Formats

References (40)

Similar Records

Related Subjects