RFQAmodel: Random Forest Quality Assessment to identify a predicted protein structure in the correct fold
Abstract
While template-free protein structure prediction protocols now produce good quality models for many targets, modelling failure remains common. For these methods to be useful it is important that users can both choose the best model from the hundreds to thousands of models that are commonly generated for a target, and determine whether this model is likely to be correct. We have developed Random Forest Quality Assessment (RFQAmodel), which assesses whether models produced by a protein structure prediction pipeline have the correct fold. RFQAmodel uses a combination of existing quality assessment scores with two predicted contact map alignment scores. These alignment scores are able to identify correct models for targets that are not otherwise captured. Our classifier was trained on a large set of protein domains that are structurally diverse and evenly balanced in terms of protein features known to have an effect on modelling success, and then tested on a second set of 244 protein domains with a similar spread of properties. When models for each target in this second set were ranked according to the RFQAmodel score, the highest-ranking model had a high-confidence RFQAmodel score for 67 modelling targets, of which 52 had the correct fold. At the othermore »
- Authors:
-
- Oxford Univ. (United Kingdom)
- Stanford Univ., CA (United States)
- Publication Date:
- Research Org.:
- SLAC National Accelerator Lab., Menlo Park, CA (United States)
- Sponsoring Org.:
- USDOE
- OSTI Identifier:
- 1633852
- Grant/Contract Number:
- AC02-76SF00515
- Resource Type:
- Accepted Manuscript
- Journal Name:
- PLoS ONE
- Additional Journal Information:
- Journal Volume: 14; Journal Issue: 10; Journal ID: ISSN 1932-6203
- Publisher:
- Public Library of Science
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 59 BASIC BIOLOGICAL SCIENCES
Citation Formats
West, Clare E., de Oliveira, Saulo H. P., and Deane, Charlotte M. RFQAmodel: Random Forest Quality Assessment to identify a predicted protein structure in the correct fold. United States: N. p., 2019.
Web. doi:10.1371/journal.pone.0218149.
West, Clare E., de Oliveira, Saulo H. P., & Deane, Charlotte M. RFQAmodel: Random Forest Quality Assessment to identify a predicted protein structure in the correct fold. United States. https://doi.org/10.1371/journal.pone.0218149
West, Clare E., de Oliveira, Saulo H. P., and Deane, Charlotte M. Mon .
"RFQAmodel: Random Forest Quality Assessment to identify a predicted protein structure in the correct fold". United States. https://doi.org/10.1371/journal.pone.0218149. https://www.osti.gov/servlets/purl/1633852.
@article{osti_1633852,
title = {RFQAmodel: Random Forest Quality Assessment to identify a predicted protein structure in the correct fold},
author = {West, Clare E. and de Oliveira, Saulo H. P. and Deane, Charlotte M.},
abstractNote = {While template-free protein structure prediction protocols now produce good quality models for many targets, modelling failure remains common. For these methods to be useful it is important that users can both choose the best model from the hundreds to thousands of models that are commonly generated for a target, and determine whether this model is likely to be correct. We have developed Random Forest Quality Assessment (RFQAmodel), which assesses whether models produced by a protein structure prediction pipeline have the correct fold. RFQAmodel uses a combination of existing quality assessment scores with two predicted contact map alignment scores. These alignment scores are able to identify correct models for targets that are not otherwise captured. Our classifier was trained on a large set of protein domains that are structurally diverse and evenly balanced in terms of protein features known to have an effect on modelling success, and then tested on a second set of 244 protein domains with a similar spread of properties. When models for each target in this second set were ranked according to the RFQAmodel score, the highest-ranking model had a high-confidence RFQAmodel score for 67 modelling targets, of which 52 had the correct fold. At the other end of the scale RFQAmodel correctly predicted that for 59 targets the highest-ranked model was incorrect. In comparisons to other methods we found that RFQAmodel is better able to identify correct models for targets where only a few of the models are correct. We found that RFQAmodel achieved a similar performance on the model sets for CASP12 and CASP13 free-modelling targets. Finally, by iteratively generating models and running RFQAmodel until a model is produced that is predicted to be correct with high confidence, we demonstrate how such a protocol can be used to focus computational efforts on difficult modelling targets. RFQAmodel and the accompanying data can be downloaded from http://opig.stats.ox.ac.uk/resources.},
doi = {10.1371/journal.pone.0218149},
journal = {PLoS ONE},
number = 10,
volume = 14,
place = {United States},
year = {2019},
month = {10}
}
Figures / Tables:

Works referenced in this record:
Assessment of model accuracy estimations in CASP12
journal, September 2017
- Kryshtafovych, Andriy; Monastyrskyy, Bohdan; Fidelis, Krzysztof
- Proteins: Structure, Function, and Bioinformatics, Vol. 86
Protein Secondary Structure Prediction Using Deep Convolutional Neural Fields
journal, January 2016
- Wang, Sheng; Peng, Jian; Ma, Jianzhu
- Scientific Reports, Vol. 6, Issue 1
MetaPSICOV: combining coevolution methods for accurate prediction of contacts and long range hydrogen bonding in proteins
journal, November 2014
- Jones, David T.; Singh, Tanya; Kosciolek, Tomasz
- Bioinformatics, Vol. 31, Issue 7
Sequential search leads to faster, more efficient fragment-based de novo protein structure prediction
journal, November 2017
- de Oliveira, Saulo H. P.; Law, Eleanor C.; Shi, Jiye
- Bioinformatics, Vol. 34, Issue 7
Scoring function for automated assessment of protein structure template quality
journal, January 2004
- Zhang, Yang; Skolnick, Jeffrey
- Proteins: Structure, Function, and Bioinformatics, Vol. 57, Issue 4
MQAPsingle: A quasi single-model approach for estimation of the quality of individual protein structure models: MQAPsingle
journal, May 2016
- Pawlowski, Marcin; Kozlowski, Lukasz; Kloczkowski, Andrzej
- Proteins: Structure, Function, and Bioinformatics, Vol. 84, Issue 8
Comparing co-evolution methods and their application to template-free protein structure prediction
journal, September 2016
- de Oliveira, Saulo Henrique Pires; Shi, Jiye; Deane, Charlotte M.
- Bioinformatics
The Protein Data Bank
journal, May 2002
- Berman, Helen M.; Battistuz, Tammy; Bhat, T. N.
- Acta Crystallographica Section D Biological Crystallography, Vol. 58, Issue 6
Scoring function for automated assessment of protein structure template quality
journal, June 2007
- Zhang, Yang; Skolnick, Jeffrey
- Proteins: Structure, Function, and Bioinformatics, Vol. 68, Issue 4
Biopython: freely available Python tools for computational molecular biology and bioinformatics
journal, March 2009
- Cock, P. J. A.; Antao, T.; Chang, J. T.
- Bioinformatics, Vol. 25, Issue 11
Predicting accurate contacts in thousands of Pfam domain families using PconsC3
journal, May 2017
- Michel, Mirco; Skwark, Marcin J.; Menéndez Hurtado, David
- Bioinformatics, Vol. 33, Issue 18
Asymmetric electron and hole transport in a high-mobility -type conjugated polymer
journal, October 2012
- Wetzelaer, Gert-Jan A. H.; Kuik, Martijn; Olivier, Yoann
- Physical Review B, Vol. 86, Issue 16
ModFOLD6: an accurate web server for the global and local quality estimation of 3D protein models
journal, April 2017
- Maghrabi, Ali H. A.; McGuffin, Liam J.
- Nucleic Acids Research, Vol. 45, Issue W1
The Pfam protein families database
journal, November 2011
- Punta, M.; Coggill, P. C.; Eberhardt, R. Y.
- Nucleic Acids Research, Vol. 40, Issue D1
EigenTHREADER: analogous protein fold recognition by efficient contact map threading
journal, April 2017
- Buchan, Daniel W. A.; Jones, David T.
- Bioinformatics, Vol. 33, Issue 17
Large-scale structure prediction by improved contact predictions and model quality assessment
journal, July 2017
- Michel, Mirco; Menéndez Hurtado, David; Uziela, Karolis
- Bioinformatics, Vol. 33, Issue 14
Identification of correct regions in protein models using structural, alignment, and consensus information
journal, March 2006
- Wallner, B.
- Protein Science, Vol. 15, Issue 4
Protein structure determination using metagenome sequence data
journal, January 2017
- Ovchinnikov, Sergey; Park, Hahnbeom; Varghese, Neha
- Science, Vol. 355, Issue 6322
A Selected Core Microbiome Drives the Early Stages of Three Popular Italian Cheese Manufactures
journal, February 2014
- De Filippis, Francesca; La Storia, Antonietta; Stellato, Giuseppina
- PLoS ONE, Vol. 9, Issue 2
Random Forest-Based Protein Model Quality Assessment (RFMQA) Using Structural Features and Potential Energy Terms
journal, September 2014
- Manavalan, Balachandran; Lee, Juyong; Lee, Jooyoung
- PLoS ONE, Vol. 9, Issue 9
ProQ3D: improved model quality assessments using deep learning
journal, January 2017
- Uziela, Karolis; Menéndez Hurtado, David; Shu, Nanjiang
- Bioinformatics
SCOPe: Structural Classification of Proteins—extended, integrating SCOP and ASTRAL data and classification of new structures
journal, December 2013
- Fox, Naomi K.; Brenner, Steven E.; Chandonia, John-Marc
- Nucleic Acids Research, Vol. 42, Issue D1
Critical assessment of methods of protein structure prediction (CASP)-Round XII
journal, December 2017
- Moult, John; Fidelis, Krzysztof; Kryshtafovych, Andriy
- Proteins: Structure, Function, and Bioinformatics, Vol. 86
How significant is a protein structure similarity with TM-score = 0.5?
journal, February 2010
- Xu, Jinrui; Zhang, Yang
- Bioinformatics, Vol. 26, Issue 7
Protein secondary structure prediction using deep convolutional neural fields
preprint, January 2015
- Wang, Sheng; Peng, Jian; Ma, Jianzhu
- arXiv
Figures / Tables found in this record: