Structured information extraction from scientific text with large language models

Dagdelen, John; Dunn, Alexander; Lee, Sanghoon; Walker, Nicholas; Rosen, Andrew S.; Ceder, Gerbrand; Persson, Kristin A.; Jain, Anubhav

doi:10.1038/s41467-024-45563-x

Title: Structured information extraction from scientific text with large language models

Journal Article · 14 February 2024 · Nature Communications

DOI: https://doi.org/10.1038/s41467-024-45563-x · OSTI ID:2433806

^[1];

^[1]; Lee, Sanghoon ^[1]; Walker, Nicholas ^[2];

^[1]; Ceder, Gerbrand ^[1];

^[1];

^[2]

Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States); University of California, Berkeley, CA (United States)
Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)

Extracting structured knowledge from scientific text remains a challenging task for machine learning models. Here, we present a simple approach to joint named entity recognition and relation extraction and demonstrate how pretrained large language models (GPT-3, Llama-2) can be fine-tuned to extract useful records of complex scientific knowledge. We test three representative tasks in materials chemistry: linking dopants and host materials, cataloging metal-organic frameworks, and general composition/phase/morphology/application information extraction. Records are extracted from single sentences or entire paragraphs, and the output can be returned as simple English sentences or a more structured format such as a list of JSON objects. This approach represents a simple, accessible, and highly flexible route to obtaining large databases of structured specialized scientific knowledge extracted from research papers.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). National Energy Research Scientific Computing Center (NERSC)

Sponsoring Organization:: USDOE Office of Science (SC), Basic Energy Sciences (BES). Materials Sciences & Engineering Division (MSE)

Grant/Contract Number:: AC02-05CH11231

OSTI ID:: 2433806

Journal Information:: Nature Communications, Journal Name: Nature Communications Journal Issue: 1 Vol. 15; ISSN 2041-1723

Publisher:: Nature Publishing GroupCopyright Statement

Country of Publication:: United States

Language:: English

References (44)

LoRA weights for Llama-2 NERRE Materials, Hacking figshare https://doi.org/10.6084/m9.figshare.24501331	dataset	January 2023
Metal–Organic Framework‐Based Ion‐Selective Membranes Li, Xingya; Hill, Matthew R.; Wang, Huanting Advanced Materials Technologies, Vol. 6, Issue 10 https://doi.org/10.1002/admt.202000790	journal	February 2021
Joint entity recognition and relation extraction as a multi-head selection problem Bekoulis, Giannis; Deleu, Johannes; Demeester, Thomas Expert Systems with Applications, Vol. 114 https://doi.org/10.1016/j.eswa.2018.07.032	journal	December 2018
Quantifying the advantage of domain-specific pre-training on named entity recognition tasks in materials science Trewartha, Amalie; Walker, Nicholas; Huo, Haoyan Patterns, Vol. 3, Issue 4 https://doi.org/10.1016/j.patter.2022.100488	journal	April 2022
Identification Schemes for Metal–Organic Frameworks To Enable Rapid Search and Cheminformatics Analysis Bucior, Benjamin J.; Rosen, Andrew S.; Haranczyk, Maciej Crystal Growth & Design, Vol. 19, Issue 11 https://doi.org/10.1021/acs.cgd.9b01050	journal	September 2019
Similarity of Precursors in Solid-State Synthesis as Text-Mined from Scientific Literature He, Tanjin; Sun, Wenhao; Huo, Haoyan Chemistry of Materials, Vol. 32, Issue 18 https://doi.org/10.1021/acs.chemmater.0c02553	journal	August 2020
Machine-Learning Rationalization and Prediction of Solid-State Synthesis Conditions Huo, Haoyan; Bartel, Christopher J.; He, Tanjin Chemistry of Materials, Vol. 34, Issue 16 https://doi.org/10.1021/acs.chemmater.2c01293	journal	August 2022
MOF-Based Membranes for Gas Separations Qian, Qihui; Asinger, Patrick A.; Lee, Moon Joo Chemical Reviews, Vol. 120, Issue 16 https://doi.org/10.1021/acs.chemrev.0c00119	journal	July 2020
Single Model for Organic and Inorganic Chemical Named Entity Recognition in ChemDataExtractor Isazawa, Taketomo; Cole, Jacqueline M. Journal of Chemical Information and Modeling, Vol. 62, Issue 5 https://doi.org/10.1021/acs.jcim.1c01199	journal	February 2022
BatteryBERT: A Pretrained Language Model for Battery Database Enhancement Huang, Shu; Cole, Jacqueline M. Journal of Chemical Information and Modeling https://doi.org/10.1021/acs.jcim.2c00035	journal	May 2022
Do Large Language Models Understand Chemistry? A Conversation with ChatGPT Castro Nascimento, Cayque Monteiro; Pimentel, André Silva Journal of Chemical Information and Modeling, Vol. 63, Issue 6, p. 1649-1655 https://doi.org/10.1021/acs.jcim.3c00285	journal	March 2023
Named Entity Recognition and Normalization Applied to Large-Scale Information Extraction from the Materials Science Literature Weston, L.; Tshitoyan, V.; Dagdelen, J. Journal of Chemical Information and Modeling, Vol. 59, Issue 9 https://doi.org/10.1021/acs.jcim.9b00470	journal	July 2019
ChatGPT Chemistry Assistant for Text Mining and the Prediction of MOF Synthesis Zheng, Zhiling; Zhang, Oufan; Borgs, Christian Journal of the American Chemical Society, Vol. 145, Issue 32 https://doi.org/10.1021/jacs.3c05819	journal	August 2023
Recent advances and applications of deep learning methods in materials science Choudhary, Kamal; DeCost, Brian; Chen, Chi npj Computational Materials, Vol. 8, Issue 1 https://doi.org/10.1038/s41524-022-00734-6	journal	April 2022
Unsupervised word embeddings capture latent knowledge from materials science literature Tshitoyan, Vahe; Dagdelen, John; Weston, Leigh Nature, Vol. 571, Issue 7763 https://doi.org/10.1038/s41586-019-1335-8	journal	July 2019
Text-mined dataset of inorganic materials synthesis recipes Kononova, Olga; Huo, Haoyan; He, Tanjin Scientific Data, Vol. 6, Issue 1 https://doi.org/10.1038/s41597-019-0224-1	journal	October 2019
Comparative dataset of experimental and computational attributes of UV/vis absorption spectra Beard, Edward J.; Sivaraman, Ganesh; Vázquez-Mayagoitia, Álvaro Scientific Data, Vol. 6, Issue 1 https://doi.org/10.1038/s41597-019-0306-0	journal	December 2019
A database of battery materials auto-generated using ChemDataExtractor Huang, Shu; Cole, Jacqueline M. Scientific Data, Vol. 7, Issue 1 https://doi.org/10.1038/s41597-020-00602-2	journal	August 2020
Auto-generated database of semiconductor band gaps using ChemDataExtractor Dong, Qingyang; Cole, Jacqueline M. Scientific Data, Vol. 9, Issue 1 https://doi.org/10.1038/s41597-022-01294-6	journal	May 2022
A database of refractive indices and dielectric constants auto-generated using ChemDataExtractor Zhao, Jiuyang; Cole, Jacqueline M. Scientific Data, Vol. 9, Issue 1 https://doi.org/10.1038/s41597-022-01295-5	journal	May 2022
Auto-generating databases of Yield Strength and Grain Size using ChemDataExtractor Kumar, Pankaj; Kabra, Saurabh; Cole, Jacqueline M. Scientific Data, Vol. 9, Issue 1 https://doi.org/10.1038/s41597-022-01301-w	journal	June 2022
Dataset of solution-based inorganic materials synthesis procedures extracted from the scientific literature Wang, Zheren; Kononova, Olga; Cruse, Kevin Scientific Data, Vol. 9, Issue 1 https://doi.org/10.1038/s41597-022-01317-2	journal	May 2022
Perovskite- and Dye-Sensitized Solar-Cell Device Databases Auto-generated Using ChemDataExtractor Beard, Edward J.; Cole, Jacqueline M. Scientific Data, Vol. 9, Issue 1 https://doi.org/10.1038/s41597-022-01355-w	journal	June 2022
A thermoelectric materials database auto-generated from the scientific literature using ChemDataExtractor Sierepeklis, Odysseas; Cole, Jacqueline M. Scientific Data, Vol. 9, Issue 1 https://doi.org/10.1038/s41597-022-01752-1	journal	October 2022
Assessment of chemistry knowledge in large language models that generate code White, Andrew D.; Hocky, Glen M.; Gandhi, Heta A. Digital Discovery, Vol. 2, Issue 2 https://doi.org/10.1039/D2DD00087C	journal	January 2023
14 examples of how LLMs can transform materials science and chemistry: a reflection on a large language model hackathon Jablonka, Kevin Maik; Ai, Qianxiang; Al-Feghali, Alexander Digital Discovery, Vol. 2, Issue 5 https://doi.org/10.1039/D3DD00113J	journal	January 2023
BioCreative V CDR task corpus: a resource for chemical disease relation extraction Li, Jiao; Sun, Yueping; Johnson, Robin J. Database, Vol. 2016 https://doi.org/10.1093/database/baw068	journal	January 2016
Fine-Tuning BERT Model for Materials Named Entity Recognition Zhao, Xintong; Greenberg, Jane; An, Yuan 2021 IEEE International Conference on Big Data (Big Data) https://doi.org/10.1109/bigdata52589.2021.9671697	conference	December 2021
Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing Gu, Yu; Tinn, Robert; Cheng, Hao ACM Transactions on Computing for Healthcare, Vol. 3, Issue 1 https://doi.org/10.1145/3458754	journal	October 2021
Survey of Hallucination in Natural Language Generation Ji, Ziwei; Lee, Nayeon; Frieske, Rita ACM Computing Surveys, Vol. 55, Issue 12 https://doi.org/10.1145/3571730	journal	March 2023
Machine Learning in Materials Discovery: Confirmed Predictions and Their Underlying Approaches Saal, James E.; Oliynyk, Anton O.; Meredig, Bryce Annual Review of Materials Research, Vol. 50, Issue 1 https://doi.org/10.1146/annurev-matsci-090319-010954	journal	July 2020
Extraction of relations between genes and diseases from text and large-scale data analysis: implications for translational research Bravo, Àlex; Piñero, Janet; Queralt-Rosinach, Núria BMC Bioinformatics, Vol. 16, Issue 1 https://doi.org/10.1186/s12859-015-0472-9	journal	February 2015
REBEL: Relation Extraction By End-to-end Language generation Huguet Cabot, Pere-Lluís; Navigli, Roberto Findings of the Association for Computational Linguistics: EMNLP 2021 https://doi.org/10.18653/v1/2021.findings-emnlp.204	conference	January 2021
GPT-NeoX-20B: An Open-Source Autoregressive Language Model Black, Sidney; Biderman, Stella; Hallahan, Eric Proceedings of BigScience Episode #5 -- Workshop on Challenges & Perspectives in Creating Large Language Models https://doi.org/10.18653/v1/2022.bigscience-1.9	conference	January 2022
A sequence-to-sequence approach for document-level relation extraction Giorgi, John; Bader, Gary; Wang, Bo Proceedings of the 21st Workshop on Biomedical Language Processing https://doi.org/10.18653/v1/2022.bionlp-1.2	conference	January 2022
Multi-Stage Prompting for Knowledgeable Dialogue Generation Liu, Zihan; Patwary, Mostofa; Prenger, Ryan Findings of the Association for Computational Linguistics: ACL 2022 https://doi.org/10.18653/v1/2022.findings-acl.104	conference	January 2022
PcMSP: A Dataset for Scientific Action Graphs Extraction from Polycrystalline Materials Synthesis Procedure Text Yang, Xianjun; Zhuo, Ya; Zuo, Julia Findings of the Association for Computational Linguistics: EMNLP 2022 https://doi.org/10.18653/v1/2022.findings-emnlp.446	conference	January 2022
Language Models as Knowledge Bases? Petroni, Fabio; Rocktäschel, Tim; Riedel, Sebastian Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) https://doi.org/10.18653/v1/D19-1250	conference	January 2019
Pretraining-Based Natural Language Generation for Text Summarization Zhang, Haoyu; Cai, Jingjing; Xu, Jianjun Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL) https://doi.org/10.18653/v1/K19-1074	conference	January 2019
DocRED: A Large-Scale Document-Level Relation Extraction Dataset Yao, Yuan; Ye, Deming; Li, Peng Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics https://doi.org/10.18653/v1/P19-1074	conference	January 2019
Materials Discovery With Machine Learning and Knowledge Discovery Oliveira, Osvaldo N.; Oliveira, Maria Cristina F. Frontiers in Chemistry, Vol. 10 https://doi.org/10.3389/fchem.2022.930369	journal	July 2022
LoRA weights for Llama-2 NERRE Materials, Hacking figshare https://doi.org/10.6084/m9.figshare.24501331	dataset	January 2023
LoRA weights for Llama-2 NERRE Materials, Hacking figshare https://doi.org/10.6084/m9.figshare.24501331.v1	dataset	January 2023
Comparative dataset of experimental and computational attributes of UV/vis absorption spectra Beard, Edward; Sivaraman, Ganesh; Vazquez-Mayagoitia, Alvaro figshare https://doi.org/10.6084/m9.figshare.7619672.v1	dataset	January 2019

Similar Records

An Introduction to Word Embeddings and Language Models

Technical Report · 2021 · OSTI ID:1773690

Borders, Tammie L; Volkova, Svitlana

KEBLM: Knowledge-Enhanced Biomedical Language Models

Journal Article · 2023 · Journal of Biomedical Informatics · OSTI ID:2420838

Lai, Tuan Manh; Zhai, ChengXiang; Ji, Heng

Extracting Material Property Measurements from Scientific Literature with Limited Annotations

Journal Article · 2025 · Journal of Chemical Information and Modeling · OSTI ID:2569760

Kong, Jessica; Panapitiya, Gihan; Saldanha, Emily

Related Subjects

96 KNOWLEDGE MANAGEMENT AND PRESERVATION

Title: Structured information extraction from scientific text with large language models

Citation Formats

References (44)

Similar Records

Related Subjects