Studying Transient Phenomena in Thin Films with Reinforcement Learning

Doucet, Mathieu; Candeago, Riccardo; Wang, Hanyu; Browning, James F.; Su, Xiao

doi:10.1021/acs.jpclett.4c00467

Title: Studying Transient Phenomena in Thin Films with Reinforcement Learning

Journal Article · 16 April 2024 · Journal of Physical Chemistry Letters

DOI: https://doi.org/10.1021/acs.jpclett.4c00467 · OSTI ID:2351055

^[1]; Candeago, Riccardo ^[2];

^[3];

^[1];

^[2]

Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
Univ. of Illinois at Urbana-Champaign, IL (United States)
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Center for Nanophase Materials Sciences (CNMS)

Neutron reflectometry has long been a powerful tool to study the interfacial properties of energy materials. Recently, time-resolved neutron reflectometry has been used to better understand transient phenomena in electrochemical systems. Those measurements often comprise a large number of reflectivity curves acquired over a narrow q range, with each individual curve having lower information content compared to a typical steady-state measurement. In this work, we present an approach that leverages existing reinforcement learning tools to model time-resolved data to extract the time evolution of structure parameters. Further, by mapping the reflectivity curves taken at different times as individual states, we use the Soft Actor-Critic algorithm to optimize the time series of structure parameters that best represent the evolution of an electrochemical system. We show that this approach constitutes an elegant solution to the modeling of time-resolved neutron reflectometry data.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Spallation Neutron Source (SNS)

Sponsoring Organization:: USDOE Office of Science (SC), Basic Energy Sciences (BES)

Grant/Contract Number:: AC05-00OR22725; SC0021409

OSTI ID:: 2351055

Journal Information:: Journal of Physical Chemistry Letters, Journal Name: Journal of Physical Chemistry Letters Journal Issue: 16 Vol. 15; ISSN 1948-7185

Publisher:: American Chemical SocietyCopyright Statement

Country of Publication:: United States

Language:: English

References (27)

Structure and Potential‐Dependent Selectivity in Redox‐Metallopolymers: Electrochemically Mediated Multicomponent Metal Separations Chen, Raylin; Feng, Jiangyan; Jeon, Jemin Advanced Functional Materials, Vol. 31, Issue 15 https://doi.org/10.1002/adfm.202009307	journal	February 2021
The Bayesian approach to reflectivity data Sivia, D. S.; Webster, J. R. P. Physica B: Condensed Matter, Vol. 248, Issue 1-4 https://doi.org/10.1016/S0921-4526(98)00259-2	journal	June 1998
Electrochemical deposition of silver and copper from a deep eutectic solvent studied using time-resolved neutron reflectivity Ballantyne, Andrew D.; Barker, Robert; Dalgliesh, Robert M. Journal of Electroanalytical Chemistry, Vol. 819 https://doi.org/10.1016/j.jelechem.2018.01.032	journal	June 2018
Time-resolved total reflection detects mass transfer along thickness direction of bulk Nafion® film Koizumi, Satoshi; Ueda, Satoru; Noda, Yohei Physica B: Condensed Matter, Vol. 551 https://doi.org/10.1016/j.physb.2017.12.029	journal	December 2018
The Study of the Binder Poly(acrylic acid) and Its Role in Concomitant Solid–Electrolyte Interphase Formation on Si Anodes Browning, Katie L.; Sacci, Robert L.; Doucet, Mathieu ACS Applied Materials & Interfaces, Vol. 12, Issue 8 https://doi.org/10.1021/acsami.9b22382	journal	January 2020
Lithium-Mediated Electrochemical Nitrogen Reduction: Tracking Electrode–Electrolyte Interfaces via Time-Resolved Neutron Reflectometry Blair, Sarah J.; Doucet, Mathieu; Browning, James F. ACS Energy Letters, Vol. 7, Issue 6 https://doi.org/10.1021/acsenergylett.1c02833	journal	May 2022
In Situ Measurement of Buried Electrolyte–Electrode Interfaces for Solid State Batteries with Nanometer Level Precision Browning, Katie L.; Westover, Andrew S.; Browning, James F. ACS Energy Letters, Vol. 8, Issue 4 https://doi.org/10.1021/acsenergylett.3c00488	journal	March 2023
Dynamic in Situ Electrochemical Neutron Reflectivity Measurements Cooper, Jonathan M.; Cubitt, Robert; Dalgliesh, Robert M. Journal of the American Chemical Society, Vol. 126, Issue 47 https://doi.org/10.1021/ja044682s	journal	December 2004
Thermo-Electro-Responsive Redox-Copolymers for Amplified Solvation, Morphological Control, and Tunable Ion Interactions Chen, Raylin; Wang, Hanyu; Doucet, Mathieu JACS Au, Vol. 3, Issue 12 https://doi.org/10.1021/jacsau.3c00486	journal	November 2023
Unraveling the Role of Solvation and Ion Valency on Redox-Mediated Electrosorption through In Situ Neutron Reflectometry and Ab Initio Molecular Dynamics Candeago, Riccardo; Wang, Hanyu; Nguyen, Manh-Thuong JACS Au, Vol. 4, Issue 3 https://doi.org/10.1021/jacsau.3c00705	journal	February 2024
Time-Resolved Neutron Reflectometry and Photovoltaic Device Studies on Sequentially Deposited PCDTBT-Fullerene Layers Clulow, Andrew J.; Tao, Chen; Lee, Kwan H. Langmuir, Vol. 30, Issue 38 https://doi.org/10.1021/la5020779	journal	September 2014
Use of Neutron Reflectivity to Measure the Dynamics of Solvation and Structural Changes in Polyvinylferrocene Films During Electrochemically Controlled Redox Cycling Glidle, Andrew; Hillman, A. Robert; Ryder, Karl S. Langmuir, Vol. 25, Issue 7 https://doi.org/10.1021/la803234e	journal	December 2008
Deep learning approach for an interface structure analysis with a large statistical noise in neutron reflectometry Aoki, Hiroyuki; Liu, Yuwei; Yamashita, Takashi Scientific Reports, Vol. 11, Issue 1 https://doi.org/10.1038/s41598-021-02085-6	journal	November 2021
Combined, time-resolved, in situ neutron reflectometry and X-ray diffraction analysis of dynamic SEI formation during electrochemical N₂ reduction Blair, Sarah J.; Doucet, Mathieu; Niemann, Valerie A. Energy & Environmental Science, Vol. 16, Issue 8 https://doi.org/10.1039/D2EE03694K	journal	January 2023
Correlation of diffusion and performance in sequentially processed P3HT/PCBM heterojunction films by time-resolved neutron reflectometry Lee, Kwan H.; Zhang, Yuliang; Burn, Paul L. Journal of Materials Chemistry C, Vol. 1, Issue 14 https://doi.org/10.1039/c3tc00063j	journal	January 2013
Elucidating proximity magnetism through polarized neutron reflectometry and machine learning Andrejevic, Nina; Chen, Zhantao; Nguyen, Thanh Applied Physics Reviews, Vol. 9, Issue 1 https://doi.org/10.1063/5.0078814	journal	March 2022
Cinematic reflectometry using QIKR, the quite intense kinetics reflectometer Ankner, J. F.; Ashkar, R.; Browning, J. F. Review of Scientific Instruments, Vol. 94, Issue 1 https://doi.org/10.1063/5.0122279	journal	January 2023
The SNS Liquids Reflectometer Ankner, J. F.; Tao, X.; Halbert, C. E. Neutron News, Vol. 19, Issue 3 https://doi.org/10.1080/10448630802210545	journal	July 2008
Towards automated analysis for neutron reflectivity Mironov, Daniil; Durant, James H.; Mackenzie, Rebecca Machine Learning: Science and Technology, Vol. 2, Issue 3 https://doi.org/10.1088/2632-2153/abe7b5	journal	April 2021
Machine learning for neutron reflectometry data analysis of two-layer thin films ^* Doucet, Mathieu; Archibald, Richard K.; Heller, William T. Machine Learning: Science and Technology, Vol. 2, Issue 3 https://doi.org/10.1088/2632-2153/abf257	journal	April 2021
Co-refinement of multiple-contrast neutron/X-ray reflectivity data using MOTOFIT Nelson, Andrew Journal of Applied Crystallography, Vol. 39, Issue 2 https://doi.org/10.1107/S0021889806005073	journal	March 2006
GenX : an extensible X-ray reflectivity refinement program utilizing differential evolution Björck, Matts; Andersson, Gabriella Journal of Applied Crystallography, Vol. 40, Issue 6 https://doi.org/10.1107/S0021889807045086	journal	November 2007
refnx : neutron and X-ray reflectometry analysis in Python Nelson, Andrew R. J.; Prescott, Stuart W. Journal of Applied Crystallography, Vol. 52, Issue 1 https://doi.org/10.1107/S1600576718017296	journal	February 2019
Neuronlike adaptive elements that can solve difficult learning control problems Barto, Andrew G.; Sutton, Richard S.; Anderson, Charles W. IEEE Transactions on Systems, Man, and Cybernetics, Vol. SMC-13, Issue 5 https://doi.org/10.1109/TSMC.1983.6313077	journal	September 1983
Accelerating Markov Chain Monte Carlo Simulation by Differential Evolution with Self-Adaptive Randomized Subspace Sampling Vrugt, J. A.; ter Braak, C. J. F.; Diks, C. G. H. International Journal of Nonlinear Sciences and Numerical Simulation, Vol. 10, Issue 3 https://doi.org/10.1515/IJNSNS.2009.10.3.273	journal	January 2009
Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor Haarnoja, Tuomas; Zhou, Aurick; Abbeel, Pieter arXiv https://doi.org/10.48550/arXiv.1801.01290	preprint	January 2018
PyTorch: An Imperative Style, High-Performance Deep Learning Library Paszke, Adam; Gross, Sam; Massa, Francisco arXiv https://doi.org/10.48550/arXiv.1912.01703	preprint	January 2019

Similar Records

Elucidating proximity magnetism through polarized neutron reflectometry and machine learning

Journal Article · 2022 · Applied Physics Reviews · OSTI ID:2564794

Andrejevic, Nina; Chen, Zhantao; Nguyen, Thanh; +7 more

Extreme Risk Mitigation in Reinforcement Learning using Extreme Value Theory

Journal Article · 2024 · Transactions on Machine Learning Research · OSTI ID:2440809

Wang, Yu; NS, Karthik Somayaji; Drgoňa, Ján; +4 more

Operando Freezing Cryogenic Electron Microscopy of Active Battery Materials

Journal Article · 2024 · Microscopy and Microanalysis · OSTI ID:2459368

Dutta, Nikita S.; Carroll, Gerard Michael; Neale, Nathan R.; +3 more

Related Subjects

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CHEMISTRY
Algorithms
Chemical structure
Layers
Optical properties
Scattering

Title: Studying Transient Phenomena in Thin Films with Reinforcement Learning

Citation Formats

References (27)

Similar Records

Related Subjects