Target–decoy false discovery rate estimation using Crema

Lin, Andy; See, Donavan; Fondrie, William E.; Keich, Uri; Noble, William Stafford

doi:10.1002/pmic.202300084

Target–decoy false discovery rate estimation using Crema

Journal Article · Wed Feb 21 00:00:00 EST 2024 · Proteomics

DOI:https://doi.org/10.1002/pmic.202300084· OSTI ID:2370132

^[1]; See, Donavan ^[2]; Fondrie, William E. ^[3]; Keich, Uri ^[4]; ^[2]

Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
Univ. of Washington, Seattle, WA (United States)
Talus Bioscience, Seattle, WA (United States)
Univ. of Sydney, NSW (Australia)

Assigning statistical confidence estimates to discoveries produced by a tandem mass spectrometry proteomics experiment is critical to enabling principled interpretation of the results and assessing the cost/benefit ratio of experimental follow-up. The most common technique for computing such estimates is to use target-decoy competition (TDC), in which observed spectra are searched against a database of real (target) peptides and a database of shuffled or reversed (decoy) peptides. TDC procedures for estimating the false discovery rate (FDR) at a given score threshold have been developed for application at the level of spectra, peptides, or proteins. Although these techniques are relatively straightforward to implement, it is common in the literature to skip over the implementation details or even to make mistakes in how the TDC procedures are applied in practice. Here we present Crema, an open-source Python tool that implements several TDC methods of spectrum-, peptide- and protein-level FDR estimation. Crema is compatible with a variety of existing database search tools and provides a straightforward way to obtain robust FDR estimates.

View Accepted Manuscript (DOE)

Research Organization:: Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)

Sponsoring Organization:: USDOE Laboratory Directed Research and Development (LDRD) Program

Grant/Contract Number:: AC05-76RL01830

OSTI ID:: 2370132

Report Number(s):: PNNL-SA--186220

Journal Information:: Proteomics, Journal Name: Proteomics Journal Issue: 8 Vol. 24; ISSN 1615-9853

Publisher:: WileyCopyright Statement

Country of Publication:: United States

Language:: English

References (45)

Comet: An open-source MS/MS sequence database search tool Eng, Jimmy K.; Jahan, Tahmina A.; Hoopmann, Michael R. PROTEOMICS, Vol. 13, Issue 1 https://doi.org/10.1002/pmic.201200439	journal	December 2012
Target-Decoy Search Strategy for Mass Spectrometry-Based Proteomics Elias, Joshua E.; Gygi, Steven P. Methods in Molecular Biology https://doi.org/10.1007/978-1-60761-444-9_5	book	December 2009
Pyteomics—a Python Framework for Exploratory Data Analysis and Rapid Software Prototyping in Proteomics Goloborodko, Anton A.; Levitsky, Lev I.; Ivanov, Mark V. Journal of the American Society for Mass Spectrometry, Vol. 24, Issue 2 https://doi.org/10.1007/s13361-012-0516-6	journal	January 2013
A Deeper Look into Comet—Implementation and Features Eng, Jimmy K.; Hoopmann, Michael R.; Jahan, Tahmina A. Journal of the American Society for Mass Spectrometry, Vol. 26, Issue 11 https://doi.org/10.1007/s13361-015-1179-x	journal	June 2015
Determining the calibration of confidence estimation procedures for unique peptides in shotgun proteomics Granholm, Viktor; Navarro, José Fernández; Noble, William Stafford Journal of Proteomics, Vol. 80 https://doi.org/10.1016/j.jprot.2012.12.007	journal	March 2013
Reanalysis of ProteomicsDB Using an Accurate, Sensitive, and Scalable False Discovery Rate Estimation Approach for Protein Groups The, Matthew; Samaras, Patroklos; Kuster, Bernhard Molecular & Cellular Proteomics, Vol. 21, Issue 12 https://doi.org/10.1016/j.mcpro.2022.100437	journal	December 2022
Effect of cleavage enzyme, search algorithm and decoy database on mass spectrometric identification of wheat gluten proteins Vensel, William H.; Dupont, Frances M.; Sloane, Stacia Phytochemistry, Vol. 72, Issue 10 https://doi.org/10.1016/j.phytochem.2011.01.002	journal	July 2011
Controlling the false discovery rate via competition: Is the +1 needed? Rajchert, Andrew; Keich, Uri Statistics & Probability Letters, Vol. 197 https://doi.org/10.1016/j.spl.2023.109819	journal	June 2023
Empirical Statistical Model To Estimate the Accuracy of Peptide Identifications Made by MS/MS and Database Search Keller, Andrew; Nesvizhskii, Alexey I.; Kolker, Eugene Analytical Chemistry, Vol. 74, Issue 20 https://doi.org/10.1021/ac025747h	journal	October 2002
A Statistical Model for Identifying Proteins by Tandem Mass Spectrometry Nesvizhskii, Alexey I.; Keller, Andrew; Kolker, Eugene Analytical Chemistry, Vol. 75, Issue 17 https://doi.org/10.1021/ac0341261	journal	September 2003
Decoy Methods for Assessing False Positives and False Discovery Rates in Shotgun Proteomics Wang, Guanghui; Wu, Wells W.; Zhang, Zheng Analytical Chemistry, Vol. 81, Issue 1 https://doi.org/10.1021/ac801664q	journal	December 2008
Beyond Target–Decoy Competition: Stable Validation of Peptide and Protein Identifications in Mass Spectrometry-Based Discovery Proteomics Couté, Yohann; Bruley, Christophe; Burger, Thomas Analytical Chemistry, Vol. 92, Issue 22 https://doi.org/10.1021/acs.analchem.0c00328	journal	September 2020
mokapot: Fast and Flexible Semisupervised Learning for Peptide Detection Fondrie, William E.; Noble, William S. Journal of Proteome Research, Vol. 20, Issue 4 https://doi.org/10.1021/acs.jproteome.0c01010	journal	February 2021
Improving Peptide-Level Mass Spectrometry Analysis via Double Competition Lin, Andy; Short, Temana; Noble, William Stafford Journal of Proteome Research, Vol. 21, Issue 10 https://doi.org/10.1021/acs.jproteome.2c00282	journal	September 2022
Quality Control for the Target Decoy Approach for Peptide Identification Debrie, Elke; Malfait, Milan; Gabriels, Ralf Journal of Proteome Research, Vol. 22, Issue 2 https://doi.org/10.1021/acs.jproteome.2c00423	journal	January 2023
Unbiased False Discovery Rate Estimation for Shotgun Proteomics Based on the Target-Decoy Approach Levitsky, Lev I.; Ivanov, Mark V.; Lobas, Anna A. Journal of Proteome Research, Vol. 16, Issue 2 https://doi.org/10.1021/acs.jproteome.6b00144	journal	December 2016
Combining High-Resolution and Exact Calibration To Boost Statistical Power: A Well-Calibrated Score Function for High-Resolution MS2 Data Lin, Andy; Howbert, J. Jeffry; Noble, William Stafford Journal of Proteome Research, Vol. 17, Issue 11 https://doi.org/10.1021/acs.jproteome.8b00206	journal	September 2018
Pyteomics 4.0: Five Years of Development of a Python Proteomics Framework Levitsky, Lev I.; Klein, Joshua A.; Ivanov, Mark V. Journal of Proteome Research, Vol. 18, Issue 2 https://doi.org/10.1021/acs.jproteome.8b00717	journal	December 2018
Bias in False Discovery Rate Estimation in Mass-Spectrometry-Based Peptide Identification Danilova, Yulia; Voronkova, Anastasia; Sulimov, Pavel Journal of Proteome Research, Vol. 18, Issue 5 https://doi.org/10.1021/acs.jproteome.8b00991	journal	April 2019
ThermoRawFileParser: Modular, Scalable, and Cross-Platform RAW File Conversion Hulstaert, Niels; Shofstahl, Jim; Sachsenberg, Timo Journal of Proteome Research, Vol. 19, Issue 1 https://doi.org/10.1021/acs.jproteome.9b00328	journal	November 2019
Repeat-Preserving Decoy Database for False Discovery Rate Estimation in Peptide Identification Moosa, Johra Muhammad; Guan, Shenheng; Moran, Michael F. Journal of Proteome Research, Vol. 19, Issue 3 https://doi.org/10.1021/acs.jproteome.9b00555	journal	February 2020
Tailor: A Nonparametric and Rapid Score Calibration Method for Database Search-Based Peptide Identification in Shotgun Proteomics Sulimov, Pavel; Kertész-Farkas, Attila Journal of Proteome Research, Vol. 19, Issue 4 https://doi.org/10.1021/acs.jproteome.9b00736	journal	March 2020
Faster SEQUEST Searching for Peptide Identification from Tandem Mass Spectra Diament, Benjamin J.; Noble, William Stafford Journal of Proteome Research, Vol. 10, Issue 9 https://doi.org/10.1021/pr101196n	journal	September 2011
On Using Samples of Known Protein Content to Assess the Statistical Calibration of Scores Assigned to Peptide-Spectrum Matches in Shotgun Proteomics Granholm, Viktor; Noble, William Stafford; Käll, Lukas Journal of Proteome Research, Vol. 10, Issue 5 https://doi.org/10.1021/pr1012619	journal	March 2011
MS Amanda, a Universal Identification Algorithm Optimized for High Accuracy Tandem Mass Spectra Dorfer, Viktoria; Pichler, Peter; Stranzl, Thomas Journal of Proteome Research, Vol. 13, Issue 8 https://doi.org/10.1021/pr500202e	journal	June 2014
Rapid and Accurate Peptide Identification from Tandem Mass Spectra Park, Christopher Y.; Klammer, Aaron A.; Käll, Lukas Journal of Proteome Research, Vol. 7, Issue 7 https://doi.org/10.1021/pr800127y	journal	May 2008
A Fast SEQUEST Cross Correlation Algorithm Eng, Jimmy K.; Fischer, Bernd; Grossmann, Jonas Journal of Proteome Research, Vol. 7, Issue 10 https://doi.org/10.1021/pr800420s	journal	October 2008
Mass-spectrometry-based draft of the human proteome Wilhelm, Mathias; Schlegl, Judith; Hahne, Hannes Nature, Vol. 509, Issue 7502 https://doi.org/10.1038/nature13319	journal	May 2014
MS-GF+ makes progress towards a universal database search tool for proteomics Kim, Sangtae; Pevzner, Pavel A. Nature Communications, Vol. 5, Issue 1 https://doi.org/10.1038/ncomms6277	journal	October 2014
Mass spectrometrists should search only for peptides they care about Noble, William Stafford Nature Methods, Vol. 12, Issue 7 https://doi.org/10.1038/nmeth.3450	journal	June 2015
OpenMS: a flexible open-source software platform for mass spectrometry data analysis Röst, Hannes L.; Sachsenberg, Timo; Aiche, Stephan Nature Methods, Vol. 13, Issue 9 https://doi.org/10.1038/nmeth.3959	journal	August 2016
MSFragger: ultrafast and comprehensive peptide identification in mass spectrometry–based proteomics Kong, Andy T.; Leprevost, Felipe V.; Avtonomov, Dmitry M. Nature Methods, Vol. 14, Issue 5 https://doi.org/10.1038/nmeth.4256	journal	April 2017
Mass spectrometrists should search for all peptides, but assess only the ones they care about Sticker, Adriaan; Martens, Lennart; Clement, Lieven Nature Methods, Vol. 14, Issue 7 https://doi.org/10.1038/nmeth.4338	journal	July 2017
Target-decoy search strategy for increased confidence in large-scale protein identifications by mass spectrometry Elias, Joshua E.; Gygi, Steven P. Nature Methods, Vol. 4, Issue 3 https://doi.org/10.1038/nmeth1019	journal	February 2007
Semi-supervised learning for peptide identification from shotgun proteomics datasets Käll, Lukas; Canterbury, Jesse D.; Weston, Jason Nature Methods, Vol. 4, Issue 11 https://doi.org/10.1038/nmeth1113	journal	October 2007
Philosopher: a versatile toolkit for shotgun proteomics data analysis da Veiga Leprevost, Felipe; Haynes, Sarah E.; Avtonomov, Dmitry M. Nature Methods, Vol. 17, Issue 9 https://doi.org/10.1038/s41592-020-0912-y	journal	July 2020
Computing Exact p-values for a Cross-correlation Shotgun Proteomics Score Function Howbert, J. Jeffry; Noble, William Stafford Molecular & Cellular Proteomics, Vol. 13, Issue 9 https://doi.org/10.1074/mcp.O113.036327	journal	September 2014
New mixture models for decoy-free false discovery rate estimation in mass spectrometry proteomics Peng, Yisu; Jain, Shantanu; Li, Yong Fuga Bioinformatics, Vol. 36, Issue Supplement_2 https://doi.org/10.1093/bioinformatics/btaa807	journal	December 2020
Group-walk: a rigorous approach to group-wise false discovery rate analysis by target-decoy competition Freestone, Jack; Short, Temana; Noble, William Stafford Bioinformatics, Vol. 38, Issue Supplement_2 https://doi.org/10.1093/bioinformatics/btac471	journal	September 2022
UniProt: a hub for protein information Consortium, UniPot Nucleic Acids Research, Vol. 43, Issue D1, p. D204-D212 https://doi.org/10.1093/nar/gku989	journal	October 2014
UniProt: a worldwide hub of protein knowledge Consortium, The UniProt Nucleic Acids Research https://doi.org/10.1093/nar/gky1049		November 2018
The PRIDE database and related tools and resources in 2019: improving support for quantification data Perez-Riverol, Yasset; Csordas, Attila; Bai, Jingwen Nucleic Acids Research, Vol. 47, Issue D1 https://doi.org/10.1093/nar/gky1106	journal	November 2018
False discovery rates in spectral identification Jeong, Kyowon; Kim, Sangtae; Bandeira, Nuno BMC Bioinformatics, Vol. 13, Issue Suppl 16 https://doi.org/10.1186/1471-2105-13-S16-S2	journal	January 2012
Controlling the false discovery rate via knockoffs Barber, Rina Foygel; Candès, Emmanuel J. The Annals of Statistics, Vol. 43, Issue 5 https://doi.org/10.1214/15-AOS1337	journal	October 2015
Simultaneous high-probability bounds on the false discovery proportion in structured, regression and online settings Katsevich, Eugene; Ramdas, Aaditya The Annals of Statistics, Vol. 48, Issue 6 https://doi.org/10.1214/19-AOS1938	journal	December 2020

Similar Records

Noble-Lab/crema

Software · Sun May 21 20:00:00 EDT 2023 · OSTI ID:code-107336

Improving Peptide-Level Mass Spectrometry Analysis via Double Competition

Journal Article · Mon Sep 26 20:00:00 EDT 2022 · Journal of Proteome Research · OSTI ID:1895576

Proteome-wide identification of proteins and their modifications with decreased ambiguities and improved false discovery rates using unique sequence tags

Journal Article · Sat Mar 15 00:00:00 EDT 2008 · Analytical Chemistry, 80(6):1871-82 · OSTI ID:926932

Related Subjects

59 BASIC BIOLOGICAL SCIENCES
FDR control
false discovery rate
target-decoy competition

Target–decoy false discovery rate estimation using Crema

Citation Formats

References (45)

Similar Records

Related Subjects