Language Models for the Prediction of SARS-CoV-2 Inhibitors

Conference · Sat Oct 01 04:00:00 EDT 2022 · International Journal of High Performance Computing Applications

DOI:https://doi.org/10.1177/10943420221121804· OSTI ID:1892426

Blanchard, Andrew ^[1]; Gounley, John ^[1]; Bhowmik, Debsindhu ^[1]; Chandra Shekar, Mayanka ^[1]; Lyngaas, Isaac ^[1]; Gao, Shang ^[1]; Yin, Junqi ^[1]; Tsaris, Aristeidis (aris) ^[1]; Wang, Feiyi ^[1]; Glaser, Jens ^[1]

---

1. ORNL

The COVID-19 pandemic highlights the need for computational tools to automate and accelerate drug design for novel protein targets. We leverage deep learning language models to generate and score drug candidates based on predicted protein binding affinity. We pre-trained a deep learning language model (BERT) on ~9.6 billion molecules and achieved peak performance of 603 petaflops in mixed precision. Our work reduces pre-training time from days to hours, compared to previous efforts with this architecture, while also increasing the dataset size by nearly an order of magnitude. For scoring, we fine-tuned the language model using an assembled set of thousands of protein targets with binding affinity data and searched for inhibitors of specific protein targets, SARS-CoV-2 Mpro and PLpro. We utilized a genetic algorithm approach for finding optimal candidates using the generation and scoring capabilities of the language model. Our generalizable models accelerate the identification of inhibitors for emerging therapeutic targets.

• Research Organization: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
• Sponsoring Organization: USDOE; USDOE Office of Science (SC); USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)
• DOE Contract Number: AC05-00OR22725
• OSTI ID: 1892426
• Conference Information: Journal Name: International Journal of High Performance Computing Applications Journal Volume: 2021
• Country of Publication: United States
• Language: English