Bayesian Optimization of Catalysis with In-Context Learning

Caldas Ramos, Mayk; Michtavy, Shane S.; White, Andrew D.; Porosoff, Marc D.

doi:10.1021/acscentsci.5c02418

Title: Bayesian Optimization of Catalysis with In-Context Learning

Journal Article · Tue Apr 14 00:00:00 UTC 2026 · ACS Central Science

DOI: https://doi.org/10.1021/acscentsci.5c02418 · OSTI ID:3374919

Caldas Ramos, Mayk ^[1]; Michtavy, Shane S. ^[2]; White, Andrew D. ^[3];

^[2]

Edison Scientific Inc., San Francisco, CA (United States)
University of Rochester, NY (United States)
University of Rochester, NY (United States); Edison Scientific Inc., San Francisco, CA (United States)

Large language models (LLMs) can perform accurate classification with zero or few examples through in-context learning (ICL), allowing the model to observe query-relevant examples at inference time and eliminating the need for additional weight updates to generalize beyond its original training data. We extend this capability to regression with uncertainty estimation using frozen LLMs (e.g., GPT-4o, Gemini), enabling Bayesian optimization (BO) in natural language without explicit model training or feature engineering. We apply this to materials discovery by representing materials as synthesis and testing procedures for use in natural language prompts. This Bayesian, design-first approach prioritizes optimization toward target material properties before detailed characterization, in contrast to conventional experimental workflows that often emphasize characterization of suboptimal materials. On benchmarks like aqueous solubility and oxidative coupling of methane (OCM), BO-ICL matches or outperforms Gaussian processes. In live experiments on the reverse water–gas shift (RWGS) reaction, BO-ICL identifies multimetallic catalysts that approach equilibrium CO yield within 6 and 10 iterations from a pool of 3,700 and 360,000 candidates, respectively. Our method redefines materials representation and accelerates discovery, with broad applications across catalysis, materials science, and AI.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: University of Rochester, NY (United States)

Sponsoring Organization:: National Institutes of Health (NIH); National Science Foundation (NSF); USDOE Office of Science (SC), Basic Energy Sciences (BES)

Grant/Contract Number:: SC0023354

OSTI ID:: 3374919

Journal Information:: ACS Central Science, Journal Name: ACS Central Science Journal Issue: 5 Vol. 12; ISSN 2374-7943; ISSN 2374-7951

Publisher:: American Chemical Society (ACS)Copyright Statement

Country of Publication:: United States

Language:: English

References (82)

Machine intelligence for chemical reaction space Schwaller, Philippe; Vaucher, Alain C.; Laplaza, Ruben WIREs Computational Molecular Science, Vol. 12, Issue 5 https://doi.org/10.1002/wcms.1604	journal	March 2022
Dual function materials for CO 2 capture and conversion using renewable H 2 Duyar, Melis S.; Treviño, Martha A. Arellano; Farrauto, Robert J. Applied Catalysis B: Environmental, Vol. 168-169 https://doi.org/10.1016/j.apcatb.2014.12.025	journal	June 2015
Molecular structure and catalytic promotional effect of Mn on supported Na2WO4/SiO2 catalysts for oxidative coupling of methane (OCM) reaction Sourav, Sagar; Kiani, Daniyal; Wang, Yixiao Catalysis Today https://doi.org/10.1016/j.cattod.2022.07.005	journal	July 2022
Recent advances in carbon dioxide hydrogenation to produce olefins and aromatics Wang, Dong; Xie, Zhenhua; Porosoff, Marc D. Chem, Vol. 7, Issue 9 https://doi.org/10.1016/j.chempr.2021.02.024	journal	September 2021
Statistical inference and adaptive design for materials discovery Lookman, Turab; Balachandran, Prasanna V.; Xue, Dezhen Current Opinion in Solid State and Materials Science, Vol. 21, Issue 3 https://doi.org/10.1016/j.cossms.2016.10.002	journal	June 2017
Exploring chemical space using natural language processing methodologies for drug discovery Öztürk, Hakime; Özgür, Arzucan; Schwaller, Philippe Drug Discovery Today, Vol. 25, Issue 4 https://doi.org/10.1016/j.drudis.2020.01.020	journal	April 2020
AI-based language models powering drug discovery and development Liu, Zhichao; Roberts, Ruth A.; Lal-Nag, Madhu Drug Discovery Today, Vol. 26, Issue 11 https://doi.org/10.1016/j.drudis.2021.06.009	journal	November 2021
Dual Functional Materials: At the Interface of Catalysis and Separations Ahmadov, Rashad; Michtavy, Shane S.; Porosoff, Marc D. Langmuir, Vol. 40, Issue 19 https://doi.org/10.1021/acs.langmuir.3c03888	journal	March 2024
High-Throughput Experimentation and Catalyst Informatics for Oxidative Coupling of Methane Nguyen, Thanh Nhat; Nhat, Thuy Tran Phuong; Takimoto, Ken ACS Catalysis, Vol. 10, Issue 2 https://doi.org/10.1021/acscatal.9b04293	journal	December 2019
Initial Sample Selection in Bayesian Optimization for Combinatorial Optimization of Chemical Compounds Morishita, Toshiharu; Kaneko, Hiromasa ACS Omega, Vol. 8, Issue 2 https://doi.org/10.1021/acsomega.2c05145	journal	December 2022
Oxidative Coupling of Methane Using Mg/Ti-Doped SiO₂-Supported Na₂WO₄/Mn Catalysts Yunarti, Rika Tri; Gu, Sangseo; Choi, Jae-Wook ACS Sustainable Chemistry & Engineering, Vol. 5, Issue 5 https://doi.org/10.1021/acssuschemeng.6b02914	journal	March 2017
ESOL: Estimating Aqueous Solubility Directly from Molecular Structure Delaney, John S. Journal of Chemical Information and Computer Sciences, Vol. 44, Issue 3 https://doi.org/10.1021/ci034243x	journal	May 2004
Bayesian-optimization-assisted discovery of stereoselective aluminum complexes for ring-opening polymerization of racemic lactide Wang, Xiaoqian; Huang, Yang; Xie, Xiaoyu Nature Communications, Vol. 14, Issue 1 https://doi.org/10.1038/s41467-023-39405-5	journal	June 2023
High-throughput ab initio design of atomic interfaces using InterMatch Gerber, Eli; Torrisi, Steven B.; Shabani, Sara Nature Communications, Vol. 14, Issue 1 https://doi.org/10.1038/s41467-023-43496-5	journal	December 2023
Language models and protocol standardization guidelines for accelerating synthesis planning in heterogeneous catalysis Suvarna, Manu; Vaucher, Alain Claude; Mitchell, Sharon Nature Communications, Vol. 14, Issue 1 https://doi.org/10.1038/s41467-023-43836-5	journal	December 2023
Benchmarking the performance of Bayesian optimization across multiple experimental materials science domains Liang, Qiaohao; Gongora, Aldair E.; Ren, Zekun npj Computational Materials, Vol. 7, Issue 1 https://doi.org/10.1038/s41524-021-00656-9	journal	November 2021
Bayesian optimization with adaptive surrogate models for automated experimental design Lei, Bowen; Kirk, Tanner Quinn; Bhattacharya, Anirban npj Computational Materials, Vol. 7, Issue 1 https://doi.org/10.1038/s41524-021-00662-x	journal	December 2021
TransPolymer: a Transformer-based language model for polymer property predictions Xu, Changwen; Wang, Yuyang; Barati Farimani, Amir npj Computational Materials, Vol. 9, Issue 1 https://doi.org/10.1038/s41524-023-01016-5	journal	April 2023
A deep learning framework to emulate density functional theory del Rio, Beatriz G.; Phan, Brandon; Ramprasad, Rampi npj Computational Materials, Vol. 9, Issue 1 https://doi.org/10.1038/s41524-023-01115-3	journal	August 2023
Machine learning interatomic potential with DFT accuracy for general grain boundaries in α-Fe Ito, Kazuma; Yokoi, Tatsuya; Hyodo, Katsutoshi npj Computational Materials, Vol. 10, Issue 1 https://doi.org/10.1038/s41524-024-01451-y	journal	November 2024
Machine learning Hubbard parameters with equivariant neural networks Uhrin, Martin; Zadoks, Austin; Binci, Luca npj Computational Materials, Vol. 11, Issue 1 https://doi.org/10.1038/s41524-024-01501-5	journal	January 2025
Sequential closed-loop Bayesian optimization as a guide for organic molecular metallophotocatalyst formulation discovery Li, Xiaobo; Che, Yu; Chen, Linjiang Nature Chemistry, Vol. 16, Issue 8 https://doi.org/10.1038/s41557-024-01546-5	journal	June 2024
Detecting hallucinations in large language models using semantic entropy Farquhar, Sebastian; Kossen, Jannik; Kuhn, Lorenz Nature, Vol. 630, Issue 8017 https://doi.org/10.1038/s41586-024-07421-0	journal	June 2024
A Multimodal Generative AI Copilot for Human Pathology Lu, Ming Y.; Chen, Bowen; Williamson, Drew F. K. Nature https://doi.org/10.1038/s41586-024-07618-3	journal	June 2024
Medical transformer for multimodal survival prediction in intensive care: integration of imaging and non-imaging data Khader, Firas; Kather, Jakob Nikolas; Müller-Franzes, Gustav Scientific Reports, Vol. 13, Issue 1 https://doi.org/10.1038/s41598-023-37835-1	journal	July 2023
Optimizing classification of diseases through language model analysis of symptoms Hassan, Esraa; Abd El-Hafeez, Tarek; Shams, Mahmoud Y. Scientific Reports, Vol. 14, Issue 1 https://doi.org/10.1038/s41598-024-51615-5	journal	January 2024
Medical language model specialized in extracting cardiac knowledge Gwon, Hansle; Seo, Jiahn; Park, Seohyun Scientific Reports, Vol. 14, Issue 1 https://doi.org/10.1038/s41598-024-80165-z	journal	November 2024
Human interpretable structure-property relationships in chemistry using explainable machine learning and large language models Wellawatte, Geemi P.; Schwaller, Philippe Communications Chemistry, Vol. 8, Issue 1 https://doi.org/10.1038/s42004-024-01393-y	journal	January 2025
Mapping the space of chemical reactions using attention-based neural networks Schwaller, Philippe; Probst, Daniel; Vaucher, Alain C. Nature Machine Intelligence, Vol. 3, Issue 2 https://doi.org/10.1038/s42256-020-00284-w	journal	January 2021
Large-scale chemical language representations capture molecular structure and properties Ross, Jerret; Belgodere, Brian; Chenthamarakshan, Vijil Nature Machine Intelligence, Vol. 4, Issue 12 https://doi.org/10.1038/s42256-022-00580-7	journal	December 2022
Leveraging large language models for predictive chemistry Jablonka, Kevin Maik; Schwaller, Philippe; Ortega-Guerrero, Andres Nature Machine Intelligence, Vol. 6, Issue 2 https://doi.org/10.1038/s42256-023-00788-1	journal	February 2024
Machine-learned approximations to Density Functional Theory Hamiltonians Hegde, Ganesh; Bowen, R. Chris Scientific Reports, Vol. 7, Issue 1 https://doi.org/10.1038/srep42669	journal	February 2017
Tuning the reactivity of molybdenum (oxy)carbide catalysts by the carburization degree: CO₂ reduction and anisole hydrodeoxygenation Zhu, Jiadong; Uslamin, Evgeny A.; Kosinov, Nikolay Catalysis Science & Technology, Vol. 10, Issue 11 https://doi.org/10.1039/D0CY00484G	journal	January 2020
Assessing the viability of K-Mo ₂ C for reverse water–gas shift scale-up: molecular to laboratory to pilot scale Juneau, Mitchell; Vonglis, Madeline; Hartvigsen, Joseph Energy & Environmental Science, Vol. 13, Issue 8 https://doi.org/10.1039/D0EE01457E	journal	January 2020
Iterative experimental design based on active machine learning reduces the experimental burden associated with reaction screening Eyke, Natalie S.; Green, William H.; Jensen, Klavs F. Reaction Chemistry & Engineering, Vol. 5, Issue 10 https://doi.org/10.1039/D0RE00232A	journal	January 2020
Assessment of chemistry knowledge in large language models that generate code White, Andrew D.; Hocky, Glen M.; Gandhi, Heta A. Digital Discovery, Vol. 2, Issue 2 https://doi.org/10.1039/D2DD00087C	journal	January 2023
A review of large language models and autonomous agents in chemistry Ramos, Mayk Caldas; Collison, Christopher J.; White, Andrew D. Chemical Science, Vol. 16, Issue 6 https://doi.org/10.1039/D4SC03921A	journal	January 2025
Commentary: The Materials Project: A materials genome approach to accelerating materials innovation Jain, Anubhav; Ong, Shyue Ping; Hautier, Geoffroy APL Materials, Vol. 1, Issue 1 https://doi.org/10.1063/1.4812323	journal	July 2013
An Introduction to Kernel and Nearest-Neighbor Nonparametric Regression Altman, N. S. The American Statistician, Vol. 46, Issue 3 https://doi.org/10.1080/00031305.1992.10475879	journal	August 1992
Billion-Scale Similarity Search with GPUs Johnson, Jeff; Douze, Matthijs; Jegou, Herve IEEE Transactions on Big Data, Vol. 7, Issue 3 https://doi.org/10.1109/TBDATA.2019.2921572	journal	July 2021
A Global Geometric Framework for Nonlinear Dimensionality Reduction Tenenbaum, J. B. Science, Vol. 290, Issue 5500 https://doi.org/10.1126/science.290.5500.2319	journal	December 2000
Probabilistic latent maximal marginal relevance Guo, Shengbo; Sanner, Scott Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval https://doi.org/10.1145/1835449.1835639	conference	July 2010
Recent Advances in Bayesian Optimization Wang, Xilu; Jin, Yaochu; Schmitt, Sebastian ACM Computing Surveys, Vol. 55, Issue 13s https://doi.org/10.1145/3582078	journal	July 2023
Language models for the prediction of SARS-CoV-2 inhibitors Blanchard, Andrew E.; Gounley, John; Bhowmik, Debsindhu The International Journal of High Performance Computing Applications, Vol. 36, Issue 5-6 https://doi.org/10.1177/10943420221121804	journal	October 2022
Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models Kung, Tiffany H.; Cheatham, Morgan; Medenilla, Arielle PLOS Digital Health, Vol. 2, Issue 2 https://doi.org/10.1371/journal.pdig.0000198	journal	February 2023
Extractive summarization of meeting recordings Murray, Gabriel; Renals, Steve; Carletta, Jean Interspeech 2005 https://doi.org/10.21437/Interspeech.2005-59	conference	September 2005
Quantifying Mental Health Signals in Twitter Coppersmith, Glen; Dredze, Mark; Harman, Craig Proceedings of the Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality https://doi.org/10.3115/v1/w14-3207	conference	January 2014
Understanding Kernel Ridge Regression: Common behaviors from simple functions to density functionals Vu, Kevin; Snyder, John; Li, Li arXiv https://doi.org/10.48550/arXiv.1501.03854	preprint	January 2015
Attention Is All You Need Vaswani, Ashish; Shazeer, Noam; Parmar, Niki arXiv https://doi.org/10.48550/arXiv.1706.03762	preprint	January 2017
Know What You Don't Know: Unanswerable Questions for SQuAD Rajpurkar, Pranav; Jia, Robin; Liang, Percy arXiv https://doi.org/10.48550/arXiv.1806.03822	preprint	January 2018
A Tutorial on Bayesian Optimization Frazier, Peter I. arXiv https://doi.org/10.48550/arXiv.1807.02811	preprint	January 2018
Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentence Sun, Chi; Huang, Luyao; Qiu, Xipeng arXiv https://doi.org/10.48550/arXiv.1903.09588	preprint	January 2019
SMILES Transformer: Pre-trained Molecular Fingerprint for Low Data Drug Discovery Honda, Shion; Shi, Shoi; Ueda, Hiroki R. arXiv https://doi.org/10.48550/arXiv.1911.04738	preprint	January 2019
Language Models are Few-Shot Learners Brown, Tom B.; Mann, Benjamin; Ryder, Nick arXiv https://doi.org/10.48550/arXiv.2005.14165	preprint	January 2020
Evaluating Large Language Models Trained on Code Chen, Mark; Tworek, Jerry; Jun, Heewoo arXiv https://doi.org/10.48550/arXiv.2107.03374	preprint	January 2021
Uncertainty Toolbox: an Open-Source Library for Assessing, Visualizing, and Improving Uncertainty Quantification Chung, Youngseog; Char, Ian; Guo, Han arXiv https://doi.org/10.48550/arXiv.2109.10254	preprint	January 2021
Text and Code Embeddings by Contrastive Pre-Training Neelakantan, Arvind; Xu, Tao; Puri, Raul arXiv https://doi.org/10.48550/arXiv.2201.10005	preprint	January 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Wei, Jason; Wang, Xuezhi; Schuurmans, Dale arXiv https://doi.org/10.48550/arXiv.2201.11903	preprint	January 2022
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances Ahn, Michael; Brohan, Anthony; Brown, Noah arXiv https://doi.org/10.48550/arXiv.2204.01691	preprint	January 2022
Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations Jung, Jaehun; Qin, Lianhui; Welleck, Sean arXiv https://doi.org/10.48550/arXiv.2205.11822	preprint	January 2022
Large Language Models are Zero-Shot Reasoners Kojima, Takeshi; Gu, Shixiang Shane; Reid, Machel arXiv https://doi.org/10.48550/arXiv.2205.11916	preprint	January 2022
LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Tasks Dinh, Tuan; Zeng, Yuchen; Zhang, Ruisu arXiv https://doi.org/10.48550/arXiv.2206.06565	preprint	January 2022
Selective Annotation Makes Language Models Better Few-Shot Learners Su, Hongjin; Kasai, Jungo; Wu, Chen Henry arXiv https://doi.org/10.48550/arXiv.2209.01975	preprint	January 2022
Teaching Algorithmic Reasoning via In-context Learning Zhou, Hattie; Nova, Azade; Larochelle, Hugo arXiv https://doi.org/10.48550/arXiv.2211.09066	preprint	January 2022
Faithful Chain-of-Thought Reasoning Lyu, Qing; Havaldar, Shreya; Stein, Adam arXiv https://doi.org/10.48550/arXiv.2301.13379	preprint	January 2023
Large Language Models for Code: Security Hardening and Adversarial Testing He, Jingxuan; Vechev, Martin arXiv https://doi.org/10.48550/arXiv.2302.05319	text	January 2023
GPT-4 Technical Report Achiam, Josh; Adler, Steven; Agarwal, Sandhini https://doi.org/10.48550/arXiv.2303.08774	preprint	March 2024
Bayesian Optimization of Catalysts With In-context Learning Ramos, Mayk Caldas; Michtavy, Shane S.; Porosoff, Marc D. arXiv https://doi.org/10.48550/arXiv.2304.05341	preprint	January 2023
Positional Information Matters for Invariant In-Context Learning: A Case Study of Simple Function Classes Chen, Yongqiang; Xie, Binghui; Zhou, Kaiwen arXiv https://doi.org/10.48550/arXiv.2311.18194	preprint	January 2023
Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment Xu, Lingling; Xie, Haoran; Qin, Si-Zhao Joe arXiv https://doi.org/10.48550/arXiv.2312.12148	preprint	January 2023
Large Language Models to Enhance Bayesian Optimization Liu, Tennison; Astorga, Nicolás; Seedat, Nabeel arXiv https://doi.org/10.48550/arXiv.2402.03921	preprint	January 2024
A Sober Look at LLMs for Material Discovery: Are They Actually Good for Bayesian Optimization Over Molecules? Kristiadi, Agustinus; Strieth-Kalthoff, Felix; Skreta, Marta arXiv https://doi.org/10.48550/arXiv.2402.05015	preprint	January 2024
LlaSMol: Advancing Large Language Models for Chemistry with a Large-Scale, Comprehensive, High-Quality Instruction Tuning Dataset Yu, Botao; Baker, Frazier N.; Chen, Ziqi arXiv https://doi.org/10.48550/arXiv.2402.09391	preprint	January 2024
Calibrating Large Language Models with Sample Consistency Lyu, Qing; Shridhar, Kumar; Malaviya, Chaitanya arXiv https://doi.org/10.48550/arXiv.2402.13904	preprint	January 2024
Context versus Prior Knowledge in Language Models Du, Kevin; Snæbjarnarson, Vésteinn; Stoehr, Niklas arXiv https://doi.org/10.48550/arXiv.2404.04633	preprint	January 2024
Many-Shot In-Context Learning Agarwal, Rishabh; Singh, Avi; Zhang, Lei M. arXiv https://doi.org/10.48550/arXiv.2404.11018	preprint	January 2024
Large Language Models Must Be Taught to Know What They Don't Know Kapoor, Sanyam; Gruver, Nate; Roberts, Manley arXiv https://doi.org/10.48550/arXiv.2406.08391	preprint	January 2024
Many-Shot In-Context Learning for Molecular Inverse Design Moayedpour, Saeed; Corrochano-Navarro, Alejandro; Sahneh, Faryad arXiv https://doi.org/10.48550/arXiv.2407.19089	preprint	January 2024
LLMs Are In-Context Bandit Reinforcement Learners Monea, Giovanni; Bosselut, Antoine; Brantley, Kianté arXiv https://doi.org/10.48550/arXiv.2410.05362	preprint	January 2024
Revisiting In-Context Learning with Long Context Language Models Baek, Jinheon; Lee, Sun Jae; Gupta, Prakhar arXiv https://doi.org/10.48550/arXiv.2412.16926	preprint	January 2024
Why Language Models Hallucinate Kalai, Adam Tauman; Nachum, Ofir; Vempala, Santosh S. arXiv https://doi.org/10.48550/arXiv.2509.04664	preprint	January 2025
Gaussian Processes for Machine Learning Rasmussen, Carl Edward; Williams, Christopher K. I. The MIT Press https://doi.org/10.7551/mitpress/3206.001.0001	book	January 2005

Similar Records

ChemGraph as an agentic framework for computational chemistry workflows

Journal Article · Thu Jan 08 00:00:00 UTC 2026 · Communications Chemistry · OSTI ID:3374567

Pham, Thang Duc; Tanikanti, Aditya; Keceli, Murat

Text Mining for Process–Structure–Properties Relationships in Metals

Journal Article · Fri Sep 26 00:00:00 UTC 2025 · Integrating Materials and Manufacturing Innovation · OSTI ID:3014104

Verma, Amit K.; Zhang, Zhisong; Seo, Junwon; +4 more

Consistent performance of large language models in rare disease diagnosis across ten languages and 4917 cases

Journal Article · Tue Oct 14 00:00:00 UTC 2025 · EBioMedicine · OSTI ID:3014511

Chimirri, Leonardo; Caufield, J. Harry; Bridges, Yasemin; +27 more

Related Subjects

Calibration
Catalysts
Hydrocarbons
Materials
Optimization

Title: Bayesian Optimization of Catalysis with In-Context Learning

Citation Formats

References (82)

Similar Records

Related Subjects