Machine learning (ML) has become critical for post-acquisition data analysis in (scanning) transmission electron microscopy, (S)TEM, imaging and spectroscopy. An emerging trend is the transition to real-time analysis and closed-loop microscope operation. The effective use of ML in electron microscopy now requires the development of strategies for microscopy-centric experiment workflow design and optimization. Here, we discuss the associated challenges with the transition to active ML, including sequential data analysis and out-of-distribution drift effects, the requirements for edge operation, local and cloud data storage, and theory in the loop operations. Specifically, we discuss the relative contributions of human scientists and ML agents in the ideation, orchestration, and execution of experimental workflows, as well as the need to develop universal hyper languages that can apply across multiple platforms. These considerations will collectively inform the operationalization of ML in next-generation experimentation.
Kalinin, Sergei V., et al. "Machine learning for automated experimentation in scanning transmission electron microscopy." npj Computational Materials, vol. 9, no. 1, Dec. 2023. https://doi.org/10.1038/s41524-023-01142-0
@article{osti_2246971,
author = {Kalinin, Sergei V. and Mukherjee, Debangshu and Roccapriore, Kevin and Blaiszik, Benjamin J. and Ghosh, Ayana and Ziatdinov, Maxim A. and Al-Najjar, Anees and Doty, Christina and Akers, Sarah and Rao, Nageswara S. and others},
title = {Machine learning for automated experimentation in scanning transmission electron microscopy},
annote = {Abstract Machine learning (ML) has become critical for post-acquisition data analysis in (scanning) transmission electron microscopy, (S)TEM, imaging and spectroscopy. An emerging trend is the transition to real-time analysis and closed-loop microscope operation. The effective use of ML in electron microscopy now requires the development of strategies for microscopy-centric experiment workflow design and optimization. Here, we discuss the associated challenges with the transition to active ML, including sequential data analysis and out-of-distribution drift effects, the requirements for edge operation, local and cloud data storage, and theory in the loop operations. Specifically, we discuss the relative contributions of human scientists and ML agents in the ideation, orchestration, and execution of experimental workflows, as well as the need to develop universal hyper languages that can apply across multiple platforms. These considerations will collectively inform the operationalization of ML in next-generation experimentation.},
doi = {10.1038/s41524-023-01142-0},
url = {https://www.osti.gov/biblio/2246971},
journal = {npj Computational Materials},
issn = {ISSN 2057-3960},
number = {1},
volume = {9},
place = {United Kingdom},
publisher = {Nature Publishing Group},
year = {2023},
month = {12}}
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States); Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
Sponsoring Organization:
National Science Foundation; USDOE; USDOE Laboratory Directed Research and Development (LDRD) Program; USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR); USDOE Office of Science (SC), Basic Energy Sciences (BES). Scientific User Facilities (SUF)
Grant/Contract Number:
AC05-00OR22725; AC05-76RL01830; SC0002501
OSTI ID:
2246971
Alternate ID(s):
OSTI ID: 2251636 OSTI ID: 2263339 OSTI ID: 2263340