Quantifying uncertainty in high-throughput density functional theory: A comparison of AFLOW, Materials Project, and OQMD

Hegde, Vinay I.; Borg, Christopher K. H.; del Rosario, Zachary; Kim, Yoolhee; Hutchinson, Maxwell; Antono, Erin; Ling, Julia; Saxe, Paul; Saal, James E.; Meredig, Bryce

doi:10.1103/PhysRevMaterials.7.053805

Title: Quantifying uncertainty in high-throughput density functional theory: A comparison of AFLOW, Materials Project, and OQMD

Journal Article · Tue May 30 00:00:00 EDT 2023 · Physical Review Materials

DOI:https://doi.org/10.1103/PhysRevMaterials.7.053805· OSTI ID:1975497

^[1]; Borg, Christopher K. H. ^[2]; del Rosario, Zachary ^[3]; Kim, Yoolhee ^[1]; Hutchinson, Maxwell ^[1]; Antono, Erin ^[1]; Ling, Julia ^[1]; Saxe, Paul ^[4]; Saal, James E. ^[2]; Meredig, Bryce ^[2]

Citrine Informatics, Redwood City, CA (United States)
Citrine Informatics, Inc., Redwood City, CA (United States)
Citrine Informatics, Redwood City, CA (United States); Olin College of Engineering, Needham, MA (United States)
Virginia Polytechnic Inst. and State Univ. (Virginia Tech), Blacksburg, VA (United States)

A central challenge in high-throughput density functional theory (HT-DFT) calculations is selecting a combination of input parameters and postprocessing techniques that can be used across all materials classes, while also managing accuracy-cost tradeoffs. To investigate the effects of these parameter choices, we consolidate three large HT-DFT databases: Automatic-FLOW (AFLOW), the Materials Project (MP), and the Open Quantum Materials Database (OQMD), and compare reported properties across each pair of databases for materials calculated using the same initial crystal structure. We find that HT-DFT formation energies and volumes are generally more reproducible than band gaps and total magnetizations; for instance, a notable fraction of records disagree on whether a material is metallic (up to 7%) or magnetic (up to 15%). The variance between calculated properties is as high as 0.105 eV/atom (median relative absolute difference, or MRAD, of 6%) for formation energy, 0.65 Å³/ atom (MRAD of 4%) for volume, 0.21 eV (MRAD of 9%) for band gap, and 0.15 μB/ formula unit (MRAD of 8%) for total magnetization, comparable to the differences between DFT and experiment. Here, we trace some of the larger discrepancies to choices involving pseudopotentials, the DFT + U formalism, and elemental reference states, and argue that further standardization of HT-DFT would be beneficial to reproducibility.

This content will become publicly available on Thu May 30 00:00:00 EDT 2024

Cite

Export

Save

Research Organization:: Citrine Informatics, Redwood City, CA (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Office of SBIR/STTR Programs (SBIR/STTR)

Grant/Contract Number:: SC0015106

OSTI ID:: 1975497

Journal Information:: Physical Review Materials, Vol. 7; ISSN 2475-9953

Publisher:: American Physical Society (APS)Copyright Statement

Country of Publication:: United States

Language:: English

References (74)

The high-throughput highway to computational materials design Curtarolo, Stefano; Hart, Gus L. W.; Nardelli, Marco Buongiorno Nature Materials, Vol. 12, Issue 3 https://doi.org/10.1038/nmat3568	journal	February 2013
Computational predictions of energy materials using density functional theory Jain, Anubhav; Shin, Yongwoo; Persson, Kristin A. Nature Reviews Materials, Vol. 1, Issue 1 https://doi.org/10.1038/natrevmats.2015.4	journal	January 2016
Materials Design and Discovery with High-Throughput Density Functional Theory: The Open Quantum Materials Database (OQMD) Saal, James E.; Kirklin, Scott; Aykol, Muratahan JOM, Vol. 65, Issue 11 https://doi.org/10.1007/s11837-013-0755-4	journal	September 2013
Efficiency of ab-initio total energy calculations for metals and semiconductors using a plane-wave basis set Kresse, G.; Furthmüller, J. Computational Materials Science, Vol. 6, Issue 1, p. 15-50 https://doi.org/10.1016/0927-0256(96)00008-0	journal	July 1996
Efficient iterative schemes for ab initio total-energy calculations using a plane-wave basis set Kresse, G.; Furthmüller, J. Physical Review B, Vol. 54, Issue 16, p. 11169-11186 https://doi.org/10.1103/PhysRevB.54.11169	journal	October 1996
AFLOWLIB.ORG: A distributed materials properties repository from high-throughput ab initio calculations Curtarolo, Stefano; Setyawan, Wahyu; Wang, Shidong Computational Materials Science, Vol. 58 https://doi.org/10.1016/j.commatsci.2012.02.002	journal	June 2012
Stability of Fe-Based Alloys With Structure Type C ₆ Cr ₂₃ Widom, M.; Mihalkovic, M. Journal of Materials Research, Vol. 20, Issue 1 https://doi.org/10.1557/JMR.2005.0028	journal	January 2005
CatApp: A Web Application for Surface Chemistry and Heterogeneous Catalysis Hummelshøj, Jens S.; Abild-Pedersen, Frank; Studt, Felix Angewandte Chemie, Vol. 124, Issue 1 https://doi.org/10.1002/ange.201107947	journal	December 2011
The Computational Materials Repository Landis, David D.; Hummelshoj, Jens S.; Nestorov, Svetlozar Computing in Science & Engineering, Vol. 14, Issue 6 https://doi.org/10.1109/MCSE.2012.16	journal	November 2012
Surface energies of elemental crystals Tran, Richard; Xu, Zihan; Radhakrishnan, Balachandran Scientific Data, Vol. 3, Issue 1 https://doi.org/10.1038/sdata.2016.80	journal	September 2016
The Harvard Clean Energy Project: Large-Scale Computational Screening and Design of Organic Photovoltaics on the World Community Grid Hachmann, Johannes; Olivares-Amaya, Roberto; Atahan-Evrenk, Sule The Journal of Physical Chemistry Letters, Vol. 2, Issue 17 https://doi.org/10.1021/jz200866s	journal	August 2011
The joint automated repository for various integrated simulations (JARVIS) for data-driven materials design Choudhary, Kamal; Garrity, Kevin F.; Reid, Andrew C. E. npj Computational Materials, Vol. 6, Issue 1 https://doi.org/10.1038/s41524-020-00440-1	journal	November 2020
Materials Cloud, a platform for open computational science Talirz, Leopold; Kumbhar, Snehal; Passaro, Elsa Scientific Data, Vol. 7, Issue 1 https://doi.org/10.1038/s41597-020-00637-5	journal	September 2020
Commentary: The Materials Project: A materials genome approach to accelerating materials innovation Jain, Anubhav; Ong, Shyue Ping; Hautier, Geoffroy APL Materials, Vol. 1, Issue 1 https://doi.org/10.1063/1.4812323	journal	July 2013
Correcting density functional theory for accurate predictions of compound enthalpies of formation: Fitted elemental-phase reference energies Stevanović, Vladan; Lany, Stephan; Zhang, Xiuwen Physical Review B, Vol. 85, Issue 11 https://doi.org/10.1103/PhysRevB.85.115104	journal	March 2012
TE Design Lab: A virtual laboratory for thermoelectric material design Gorai, Prashun; Gao, Duanfeng; Ortiz, Brenden Computational Materials Science, Vol. 112 https://doi.org/10.1016/j.commatsci.2015.11.006	journal	February 2016
High-throughput electronic band structure calculations: Challenges and tools Setyawan, Wahyu; Curtarolo, Stefano Computational Materials Science, Vol. 49, Issue 2 https://doi.org/10.1016/j.commatsci.2010.05.010	journal	August 2010
AiiDA: automated interactive infrastructure and database for computational science Pizzi, Giovanni; Cepellotti, Andrea; Sabatini, Riccardo Computational Materials Science, Vol. 111 https://doi.org/10.1016/j.commatsci.2015.09.013	journal	January 2016
The atomic simulation environment—a Python library for working with atoms Hjorth Larsen, Ask; Jørgen Mortensen, Jens; Blomqvist, Jakob Journal of Physics: Condensed Matter, Vol. 29, Issue 27 https://doi.org/10.1088/1361-648X/aa680e	journal	June 2017
MPInterfaces: A Materials Project based Python tool for high-throughput computational screening of interfacial systems Mathew, Kiran; Singh, Arunima K.; Gabriel, Joshua J. Computational Materials Science, Vol. 122 https://doi.org/10.1016/j.commatsci.2016.05.020	journal	September 2016
Python Materials Genomics (pymatgen): A robust, open-source python library for materials analysis Ong, Shyue Ping; Richards, William Davidson; Jain, Anubhav Computational Materials Science, Vol. 68 https://doi.org/10.1016/j.commatsci.2012.10.028	journal	February 2013
The Open Quantum Materials Database (OQMD): assessing the accuracy of DFT formation energies Kirklin, Scott; Saal, James E.; Meredig, Bryce npj Computational Materials, Vol. 1, Issue 1 https://doi.org/10.1038/npjcompumats.2015.10	journal	December 2015
Amp: A modular approach to machine learning in atomistic simulations Khorshidi, Alireza; Peterson, Andrew A. Computer Physics Communications, Vol. 207 https://doi.org/10.1016/j.cpc.2016.05.010	journal	October 2016
The alloy theoretic automated toolkit: A user guide van de Walle, A.; Asta, M.; Ceder, G. Calphad, Vol. 26, Issue 4 https://doi.org/10.1016/S0364-5916(02)80006-2	journal	December 2002
Atomate: A high-level interface to generate, execute, and analyze computational materials science workflows Mathew, Kiran; Montoya, Joseph H.; Faghaninia, Alireza Computational Materials Science, Vol. 139 https://doi.org/10.1016/j.commatsci.2017.07.030	journal	November 2017
CALYPSO: A method for crystal structure prediction Wang, Yanchao; Lv, Jian; Zhu, Li Computer Physics Communications, Vol. 183, Issue 10 https://doi.org/10.1016/j.cpc.2012.05.008	journal	October 2012
The MAterials Simulation Toolkit (MAST) for atomistic modeling of defects and diffusion Mayeshiba, Tam; Wu, Henry; Angsten, Thomas Computational Materials Science, Vol. 126 https://doi.org/10.1016/j.commatsci.2016.09.018	journal	January 2017
First principles phonon calculations in materials science Togo, Atsushi; Tanaka, Isao Scripta Materialia, Vol. 108 https://doi.org/10.1016/j.scriptamat.2015.07.021	journal	November 2015
Band structure diagram paths based on crystallography Hinuma, Yoyo; Pizzi, Giovanni; Kumagai, Yu Computational Materials Science, Vol. 128 https://doi.org/10.1016/j.commatsci.2016.10.015	journal	February 2017
A user guide for SLUSCHI: Solid and Liquid in Ultra Small Coexistence with Hovering Interfaces Hong, Qi-Jun; van de Walle, Axel Calphad, Vol. 52 https://doi.org/10.1016/j.calphad.2015.12.003	journal	March 2016
USPEX—Evolutionary crystal structure prediction Glass, Colin W.; Oganov, Artem R.; Hansen, Nikolaus Computer Physics Communications, Vol. 175, Issue 11-12 https://doi.org/10.1016/j.cpc.2006.07.020	journal	December 2006
XtalOpt: An open-source evolutionary algorithm for crystal structure prediction Lonie, David C.; Zurek, Eva Computer Physics Communications, Vol. 182, Issue 2 https://doi.org/10.1016/j.cpc.2010.07.048	journal	February 2011
Designing meaningful density functional theory calculations in materials science—a primer Mattsson, Ann E.; Schultz, Peter A.; Desjarlais, Michael P. Modelling and Simulation in Materials Science and Engineering, Vol. 13, Issue 1 https://doi.org/10.1088/0965-0393/13/1/R01	journal	November 2004
AiiDA 1.0, a scalable computational infrastructure for automated reproducible workflows and data provenance Huber, Sebastiaan P.; Zoupanos, Spyros; Uhrin, Martin Scientific Data, Vol. 7, Issue 1 https://doi.org/10.1038/s41597-020-00638-4	journal	September 2020
Identifying an efficient, thermally robust inorganic phosphor host via machine learning Zhuo, Ya; Mansouri Tehrani, Aria; Oliynyk, Anton O. Nature Communications, Vol. 9, Issue 1 https://doi.org/10.1038/s41467-018-06625-z	journal	October 2018
Combinatorial screening for new materials in unconstrained composition space with machine learning Meredig, B.; Agrawal, A.; Kirklin, S. Physical Review B, Vol. 89, Issue 9 https://doi.org/10.1103/PhysRevB.89.094104	journal	March 2014
Reproducibility in density functional theory calculations of solids Lejaeghere, K.; Bihlmayer, G.; Bjorkman, T. Science, Vol. 351, Issue 6280 https://doi.org/10.1126/science.aad3000	journal	March 2016
New developments in the Inorganic Crystal Structure Database (ICSD): accessibility in support of materials research and design Belsky, Alec; Hellenbrandt, Mariette; Karen, Vicky Lynn Acta Crystallographica Section B Structural Science, Vol. 58, Issue 3 https://doi.org/10.1107/S0108768102006948	journal	May 2002
Projector augmented-wave method Blöchl, P. E. Physical Review B, Vol. 50, Issue 24, p. 17953-17979 https://doi.org/10.1103/PhysRevB.50.17953	journal	December 1994
From ultrasoft pseudopotentials to the projector augmented-wave method Kresse, G.; Joubert, D. Physical Review B, Vol. 59, Issue 3, p. 1758-1775 https://doi.org/10.1103/PhysRevB.59.1758	journal	January 1999
Generalized Gradient Approximation Made Simple Perdew, John P.; Burke, Kieron; Ernzerhof, Matthias Physical Review Letters, Vol. 77, Issue 18, p. 3865-3868 https://doi.org/10.1103/PhysRevLett.77.3865	journal	October 1996
High-Throughput Combinatorial Database of Electronic Band Structures for Inorganic Scintillator Materials Setyawan, Wahyu; Gaume, Romain M.; Lam, Stephanie ACS Combinatorial Science, Vol. 13, Issue 4 https://doi.org/10.1021/co200012w	journal	June 2011
Uncovering Compounds by Synergy of Cluster Expansion and High-Throughput Methods Levy, Ohad; Hart, Gus L. W.; Curtarolo, Stefano Journal of the American Chemical Society, Vol. 132, Issue 13 https://doi.org/10.1021/ja9105623	journal	April 2010
High-throughput computational screening of thermal conductivity, Debye temperature, and Grüneisen parameter using a quasiharmonic Debye model Toher, Cormac; Plata, Jose J.; Levy, Ohad Physical Review B, Vol. 90, Issue 17 https://doi.org/10.1103/PhysRevB.90.174107	journal	November 2014
A RESTful API for exchanging materials data in the AFLOWLIB.org consortium Taylor, Richard H.; Rose, Frisco; Toher, Cormac Computational Materials Science, Vol. 93 https://doi.org/10.1016/j.commatsci.2014.05.014	journal	October 2014
Charting the complete elastic properties of inorganic crystalline compounds de Jong, Maarten; Chen, Wei; Angsten, Thomas Scientific Data, Vol. 2, Issue 1 https://doi.org/10.1038/sdata.2015.9	journal	March 2015
Understanding thermoelectric properties from high-throughput calculations: trends, insights, and comparisons with experiment Chen, Wei; Pöhls, Jan-Hendrik; Hautier, Geoffroy Journal of Materials Chemistry C, Vol. 4, Issue 20 https://doi.org/10.1039/C5TC04339E	journal	January 2016
A database to enable discovery and design of piezoelectric materials de Jong, Maarten; Chen, Wei; Geerlings, Henry Scientific Data, Vol. 2, Issue 1 https://doi.org/10.1038/sdata.2015.53	journal	September 2015
High-throughput screening of inorganic compounds for the discovery of novel dielectric and optical materials Petousis, Ioannis; Mrdjenovich, David; Ballouz, Eric Scientific Data, Vol. 4, Issue 1 https://doi.org/10.1038/sdata.2016.134	journal	January 2017
High-throughput density-functional perturbation theory phonons for inorganic materials Petretto, Guido; Dwaraknath, Shyam; P. C. Miranda, Henrique Scientific Data, Vol. 5, Issue 1 https://doi.org/10.1038/sdata.2018.65	journal	May 2018
Author Correction: Automated generation and ensemble-learned matching of X-ray absorption spectra Zheng, Chen; Mathew, Kiran; Chen, Chi npj Computational Materials, Vol. 4, Issue 1 https://doi.org/10.1038/s41524-018-0080-0	journal	April 2018
Prediction of solid-aqueous equilibria: Scheme to combine first-principles calculations of solids with experimental aqueous states Persson, Kristin A.; Waldwick, Bryn; Lazic, Predrag Physical Review B, Vol. 85, Issue 23 https://doi.org/10.1103/PhysRevB.85.235438	journal	June 2012
The Materials Application Programming Interface (API): A simple, flexible and efficient API for materials data based on REpresentational State Transfer (REST) principles Ong, Shyue Ping; Cholia, Shreyas; Jain, Anubhav Computational Materials Science, Vol. 97 https://doi.org/10.1016/j.commatsci.2014.10.037	journal	February 2015
High-throughput computational search for strengthening precipitates in alloys Kirklin, S.; Saal, James E.; Hegde, Vinay I. Acta Materialia, Vol. 102 https://doi.org/10.1016/j.actamat.2015.09.016	journal	January 2016
High-Throughput Computational Screening of Perovskites for Thermochemical Water Splitting Applications Emery, Antoine A.; Saal, James E.; Kirklin, Scott Chemistry of Materials, Vol. 28, Issue 16 https://doi.org/10.1021/acs.chemmater.6b01182	journal	July 2016
Crystal structure, energetics, and phase stability of strengthening precipitates in Mg alloys: A first-principles study Wang, Dongshu; Amsler, Maximilian; Hegde, Vinay I. Acta Materialia, Vol. 158 https://doi.org/10.1016/j.actamat.2018.07.041	journal	October 2018
First-Principles Determination of Multicomponent Hydride Phase Diagrams: Application to the Li-Mg-N-H System R. Akbarzadeh, A.; Ozoliņš, V.; Wolverton, C. Advanced Materials, Vol. 19, Issue 20 https://doi.org/10.1002/adma.200700843	journal	September 2007
The phase stability network of all inorganic materials Hegde, Vinay I.; Aykol, Muratahan; Kirklin, Scott Science Advances, Vol. 6, Issue 9 https://doi.org/10.1126/sciadv.aay5606	journal	February 2020
Exploring the High-Pressure Materials Genome Amsler, Maximilian; Hegde, Vinay I.; Jacobsen, Steven D. Physical Review X, Vol. 8, Issue 4 https://doi.org/10.1103/PhysRevX.8.041021	journal	November 2018
Beyond bulk single crystals: A data format for all materials structure–property–processing relationships Michel, Kyle; Meredig, Bryce MRS Bulletin, Vol. 41, Issue 08 https://doi.org/10.1557/mrs.2016.166	journal	August 2016
Calculation of the lattice constant of solids with semilocal functionals Haas, Philipp; Tran, Fabien; Blaha, Peter Physical Review B, Vol. 79, Issue 8 https://doi.org/10.1103/PhysRevB.79.085104	journal	February 2009
The effect of elastic strain on energy band gap and lattice parameter in III-V compounds Olsen, G. H.; Nuese, C. J.; Smith, R. T. Journal of Applied Physics, Vol. 49, Issue 11 https://doi.org/10.1063/1.324472	journal	November 1978
Effect of mismatch strain on band gap in III‐V semiconductors Kuo, C. P.; Vong, S. K.; Cohen, R. M. Journal of Applied Physics, Vol. 57, Issue 12, p. 5428-5432 https://doi.org/10.1063/1.334817	journal	June 1985
Predicted band-gap pressure coefficients of all diamond and zinc-blende semiconductors: Chemical trends Wei, Su-Huai; Zunger, Alex Physical Review B, Vol. 60, Issue 8 https://doi.org/10.1103/PhysRevB.60.5404	journal	August 1999
An improved symmetry-based approach to reciprocal space path selection in band structure calculations Munro, Jason M.; Latimer, Katherine; Horton, Matthew K. npj Computational Materials, Vol. 6, Issue 1 https://doi.org/10.1038/s41524-020-00383-7	journal	July 2020
Accelerated discovery of new magnets in the Heusler alloy family Sanvito, Stefano; Oses, Corey; Xue, Junkai Science Advances, Vol. 3, Issue 4 https://doi.org/10.1126/sciadv.1602241	journal	April 2017
High-throughput prediction of the ground-state collinear magnetic order of inorganic materials using Density Functional Theory Horton, Matthew Kristofer; Montoya, Joseph Harold; Liu, Miao npj Computational Materials, Vol. 5, Issue 1 https://doi.org/10.1038/s41524-019-0199-7	journal	June 2019
MATCOR, a program for the cross-validation of material properties between databases Marquez Chavez, Jorge; Kiefer, Boris Computational Materials Science, Vol. 187 https://doi.org/10.1016/j.commatsci.2020.110103	journal	February 2021
A high-throughput infrastructure for density functional theory calculations Jain, Anubhav; Hautier, Geoffroy; Moore, Charles J. Computational Materials Science, Vol. 50, Issue 8 https://doi.org/10.1016/j.commatsci.2011.02.023	journal	June 2011
AFLOW: An automatic framework for high-throughput materials discovery Curtarolo, Stefano; Setyawan, Wahyu; Hart, Gus L. W. Computational Materials Science, Vol. 58 https://doi.org/10.1016/j.commatsci.2012.02.005	journal	June 2012
Band theory and Mott insulators: Hubbard U instead of Stoner I Anisimov, Vladimir I.; Zaanen, Jan; Andersen, Ole K. Physical Review B, Vol. 44, Issue 3, p. 943-954 https://doi.org/10.1103/PhysRevB.44.943	journal	July 1991
Perspective: Treating electron over-delocalization with the DFT+U method Kulik, Heather J. The Journal of Chemical Physics, Vol. 142, Issue 24 https://doi.org/10.1063/1.4922693	journal	June 2015
The AFLOW standard for high-throughput materials science calculations Calderon, Camilo E.; Plata, Jose J.; Toher, Cormac Computational Materials Science, Vol. 108 https://doi.org/10.1016/j.commatsci.2015.07.019	journal	October 2015
Formation enthalpies by mixing GGA and GGA $+$ $U$ calculations Jain, Anubhav; Hautier, Geoffroy; Ong, Shyue Ping Physical Review B, Vol. 84, Issue 4 https://doi.org/10.1103/PhysRevB.84.045115	journal	July 2011

Similar Records

Reflections on one million compounds in the open quantum materials database (OQMD)

Journal Article · Wed Jul 13 00:00:00 EDT 2022 · JPhys Materials · OSTI ID:1975497

Shen, Jiahong; Griesemer, Sean D.; Gopakumar, Abhijith; +5 more

High-throughput search for magnetic topological materials using spin-orbit spillage, machine learning, and experiments

Journal Article · Fri Apr 16 00:00:00 EDT 2021 · Physical Review. B · OSTI ID:1975497

Choudhary, Kamal; Garrity, Kevin F.; Ghimire, Nirmal J.; +2 more

Discovery of Stable Surfaces with Extreme Work Functions by High‐Throughput Density Functional Theory and Machine Learning

Journal Article · Wed Mar 20 00:00:00 EDT 2024 · Advanced Functional Materials · OSTI ID:1975497

Schindler, Peter; Antoniuk, Evan R.; Cheon, Gowoon; +2 more

Related Subjects

36 MATERIALS SCIENCE
Density functional theory
high-throughput databases
uncertainty

Title: Quantifying uncertainty in high-throughput density functional theory: A comparison of AFLOW, Materials Project, and OQMD

Citation Formats

References (74)

Similar Records

Related Subjects