Crowdsourcing biocuration: The Community Assessment of Community Annotation with Ontologies (CACAO)

Ramsey, Jolene; McIntosh, Brenley; Renfro, Daniel; Aleksander, Suzanne A.; LaBonte, Sandra; Ross, Curtis; Zweifel, Adrienne E.; Liles, Nathan; Farrar, Shabnam; Gill, Jason J.; Erill, Ivan; Ades, Sarah; Berardini, Tanya Z.; Bennett, Jennifer A.; Brady, Siobhan; Britton, Robert; Carbon, Seth; Caruso, Steven M.; Clements, Dave; Dalia, Ritu; Defelice, Meredith; Doyle, Erin L.; Friedberg, Iddo; Gurney, Susan R.; Hughes, Lee; Johnson, Allison; Kowalski, Jason M.; Li, Donghui; Lovering, Ruth C.; Mans, Tamara L.; McCarthy, Fiona; Moore, Sean D.; Murphy, Rebecca; Paustian, Timothy D.; Perdue, Sarah; Peterson, Celeste N.; Prüß, Birgit M.; Saha, Margaret S.; Sheehy, Robert R.; Tansey, John T.; Temple, Louise; Thorman, Alexander William; Trevino, Saul; Vollmer, Amy Cheng; Walbot, Virginia; Willey, Joanne; Siegele, Deborah A.; Hu, James C.

doi:10.1371/journal.pcbi.1009463

Title: Crowdsourcing biocuration: The Community Assessment of Community Annotation with Ontologies (CACAO)

Journal Article · Thu Oct 28 00:00:00 UTC 2021 · PLoS Computational Biology (Online)

DOI: https://doi.org/10.1371/journal.pcbi.1009463 · OSTI ID:1833996

^[1]; McIntosh, Brenley ^[1];

^[1]; Aleksander, Suzanne A. ^[2];

^[1];

^[3];

^[1];

^[4];

^[5];

^[6];

^[7]; Brady, Siobhan ^[8];

^[9];

^[10];

^[4];

^[11]; Dalia, Ritu ^[12] more »

Texas A & M Univ., College Station, TX (United States)
Texas A & M Univ., College Station, TX (United States); Stanford University, Palo Alto, CA (United States)
Texas A & M Univ., College Station, TX (United States); Virginia Polytechnic Inst. and State Univ. (Virginia Tech), Blacksburg, VA (United States)
Univ. of Maryland Baltimore County (UMBC), Baltimore, MD (United States)
Pennsylvania State Univ., University Park, PA (United States)
Phoenix Bioinformatics, Newark, CA (United States)
Otterbein University, Westerville, OH (United States)
Univ. of California, Davis, CA (United States)
Baylor College of Medicine, Houston, TX (United States); Michigan State Univ., East Lansing, MI (United States)
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Johns Hopkins Univ., Baltimore, MD (United States)
MaST Community Charter School, Philadelphia, PA (United States); Drexel Univ., Philadelphia, PA (United States)
Doane University, Crete, NE (United States)
Iowa State Univ., Ames, IA (United States); Miami Univ., Oxford, OH (United States)
Univ. of St. Andrews, Scotland (United Kingdom); Drexel Univ., Philadelphia, PA (United States)
Univ. of North Texas, Denton, TX (United States)
Virginia Commonwealth Univ., Richmond, VA (United States)
Marian University, Fond du Lac, WI (United States); University of Wisconsin-Parkside, Kenosha, WI (United States)
University College London (United Kingdom)
North Hennepin Community College, Brooklyn Park, MN (United States); Minnesota State University Moorhead, Brooklyn Park, MN (United States)
Univ. of Arizona, Tucson, AZ (United States); Mississippi State Univ., Starkville, MS (United States)
Univ. of Central Florida, Orlando, FL (United States)
Centenary College of Louisiana, Shreveport, LA (United States)
Univ. of Wisconsin, Madison, WI (United States)
Univ. of Wisconsin, Madison, WI (United States); Univ. of Wisconsin-Parkside, Kenosha, WI (United States)
Suffolk University, Boston, MA (United States)
North Dakota State Univ., Fargo, ND (United States)
College of William and Mary, Williamsburg, VA (United States)
Radford University, VA (United States)
James Madison Univ., Harrisonburg, VA (United States)
Univ. of Cincinnati, OH (United States)
Houston Baptist University, TX (United States)
Swarthmore College, PA (United States)
Stanford Univ., CA (United States)
Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, NY (United States)

Experimental data about gene functions curated from the primary literature have enormous value for research scientists in understanding biology. Using the Gene Ontology (GO), manual curation by experts has provided an important resource for studying gene function, especially within model organisms. Unprecedented expansion of the scientific literature and validation of the predicted proteins have increased both data value and the challenges of keeping pace. Capturing literature-based functional annotations is limited by the ability of biocurators to handle the massive and rapidly growing scientific literature. Within the community-oriented wiki framework for GO annotation called the Gene Ontology Normal Usage Tracking System (GONUTS), we describe an approach to expand biocuration through crowdsourcing with undergraduates. This multiplies the number of high-quality annotations in international databases, enriches our coverage of the literature on normal gene function, and pushes the field in new directions. From an intercollegiate competition judged by experienced biocurators, Community Assessment of Community Annotation with Ontologies (CACAO), we have contributed nearly 5,000 literature-based annotations. Many of those annotations are to organisms not currently well-represented within GO. Over a 10-year history, our community contributors have spurred changes to the ontology not traditionally covered by professional biocurators. The CACAO principle of relying on community members to participate in and shape the future of biocuration in GO is a powerful and scalable model used to promote the scientific enterprise. It also provides undergraduate students with a unique and enriching introduction to critical reading of primary literature and acquisition of marketable skills.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)

Sponsoring Organization:: USDOE Office of Science (SC)

Grant/Contract Number:: AC02-05CH11231

OSTI ID:: 1833996

Journal Information:: PLoS Computational Biology (Online), Journal Name: PLoS Computational Biology (Online) Journal Issue: 10 Vol. 17; ISSN 1553-7358

Publisher:: Public Library of ScienceCopyright Statement

Country of Publication:: United States

Language:: English

References (55)

Mechanisms Connecting the Conserved Protein Kinases Ssp1, Kin1, and Pom1 in Fission Yeast Cell Polarity and Division Lee, Mid Eum; Rusin, Scott F.; Jenkins, Nicole Current Biology, Vol. 28, Issue 1 https://doi.org/10.1016/j.cub.2017.11.034	journal	January 2018
MetaGO: Predicting Gene Ontology of Non-homologous Proteins Through Low-Resolution Protein Structure Prediction and Protein–Protein Network Mapping Zhang, Chengxin; Zheng, Wei; Freddolino, Peter L. Journal of Molecular Biology, Vol. 430, Issue 15 https://doi.org/10.1016/j.jmb.2018.03.004	journal	July 2018
Applying the Gene Ontology in microbial annotation Giglio, Michelle G.; Collmer, Candace W.; Lomax, Jane Trends in Microbiology, Vol. 17, Issue 7 https://doi.org/10.1016/j.tim.2009.04.003	journal	July 2009
Quality of information sources about mental disorders: a comparison of Wikipedia with centrally controlled web and printed sources Reavley, N. J.; Mackinnon, A. J.; Morgan, A. J. Psychological Medicine, Vol. 42, Issue 8 https://doi.org/10.1017/S003329171100287X	journal	December 2011
Structure and Protein Interaction-Based Gene Ontology Annotations Reveal Likely Functions of Uncharacterized Proteins on Human Chromosome 17 Zhang, Chengxin; Wei, Xiaoqiong; Omenn, Gilbert S. Journal of Proteome Research, Vol. 17, Issue 12 https://doi.org/10.1021/acs.jproteome.8b00453	journal	September 2018
Internet encyclopaedias go head to head Giles, Jim Nature, Vol. 438, Issue 7070 https://doi.org/10.1038/438900a	journal	December 2005
No rest for the bio-wikis Callaway, Ewen Nature, Vol. 468, Issue 7322 https://doi.org/10.1038/468359a	journal	November 2010
Gene Ontology: tool for the unification of biology Ashburner, Michael; Ball, Catherine A.; Blake, Judith A. Nature Genetics, Vol. 25, Issue 1 https://doi.org/10.1038/75556	journal	May 2000
Sustainable digital infrastructure: Although databases and other online resources have become a central tool for biological research, their long‐term support and maintenance is far from secure Bastow, Ruth; Leonelli, Sabina EMBO reports, Vol. 11, Issue 10 https://doi.org/10.1038/embor.2010.145	journal	September 2010
Crystal structure of a monomeric retroviral protease solved by protein folding game players Khatib, Firas; DiMaio, Frank; Cooper, Seth Nature Structural & Molecular Biology, Vol. 18, Issue 10 https://doi.org/10.1038/nsmb.2119	journal	September 2011
SciPy 1.0: fundamental algorithms for scientific computing in Python Virtanen, Pauli; Gommers, Ralf; Oliphant, Travis E. Nature Methods https://doi.org/10.1038/s41592-019-0686-2	journal	February 2020
GOATOOLS: A Python library for Gene Ontology analyses Klopfenstein, D. V.; Zhang, Liangsheng; Pedersen, Brent S. Scientific Reports, Vol. 8, Issue 1 https://doi.org/10.1038/s41598-018-28948-z	journal	July 2018
The FAIR Guiding Principles for scientific data management and stewardship Wilkinson, Mark D.; Dumontier, Michel; Aalbersberg, IJsbrand Jan Scientific Data, Vol. 3, Issue 1 https://doi.org/10.1038/sdata.2016.18	journal	March 2016
Open-Source Genomic Analysis of Shiga-Toxin–Producing E. coli O104:H4 Rohde, Holger; Qin, Junjie; Cui, Yujun New England Journal of Medicine, Vol. 365, Issue 8 https://doi.org/10.1056/NEJMoa1107643	journal	August 2011
An inclusive Research Education Community (iREC): Impact of the SEA-PHAGES program on research outcomes and student learning Hanauer, David I.; Graham, Mark J.; Betancur, Laura Proceedings of the National Academy of Sciences, Vol. 114, Issue 51 https://doi.org/10.1073/pnas.1718188115	journal	December 2017
Exploring autophagy with Gene Ontology Denny, Paul; Feuermann, Marc; Hill, David P. Autophagy, Vol. 14, Issue 3 https://doi.org/10.1080/15548627.2017.1415189	journal	February 2018
Phylogenetic-based propagation of functional annotations within the Gene Ontology consortium Gaudet, P.; Livstone, M. S.; Lewis, S. E. Briefings in Bioinformatics, Vol. 12, Issue 5 https://doi.org/10.1093/bib/bbr042	journal	August 2011
Biopython: freely available Python tools for computational molecular biology and bioinformatics Cock, P. J. A.; Antao, T.; Chang, J. T. Bioinformatics, Vol. 25, Issue 11 https://doi.org/10.1093/bioinformatics/btp163	journal	March 2009
On expert curation and scalability: UniProtKB/Swiss-Prot as a case study Poux, Sylvain; Arighi, Cecilia N.; Magrane, Michele Bioinformatics, Vol. 33, Issue 21 https://doi.org/10.1093/bioinformatics/btx439	journal	July 2017
TreeGrafter: phylogenetic tree-based annotation of proteins with Gene Ontology terms and other annotations Tang, Haiming; Finn, Robert D.; Thomas, Paul D. Bioinformatics, Vol. 35, Issue 3 https://doi.org/10.1093/bioinformatics/bty625	journal	July 2018
Community curation in PomBase: enabling fission yeast experts to provide detailed, standardized, sharable annotation from research publications Lock, Antonia; Harris, Midori A.; Rutherford, Kim Database, Vol. 2020 https://doi.org/10.1093/database/baaa028	journal	January 2020
A guide to best practices for Gene Ontology (GO) manual annotation Balakrishnan, R.; Harris, M. A.; Huntley, R. Database, Vol. 2013, Issue 0 https://doi.org/10.1093/database/bat054	journal	January 2013
How much does curation cost? Karp, Peter D. Database, Vol. 2016 https://doi.org/10.1093/database/baw110	journal	January 2016
Crowd-sourcing and author submission as alternatives to professional curation Karp, Peter D. Database, Vol. 2016 https://doi.org/10.1093/database/baw149	journal	January 2016
The Science and Value of Diversity: Closing the Gaps in Our Understanding of Inclusion and Diversity Swartz, Talia H.; Palermo, Ann-Gel S.; Masur, Sandra K. The Journal of Infectious Diseases, Vol. 220, Issue Supplement_2 https://doi.org/10.1093/infdis/jiz174	journal	August 2019
ETE 3: Reconstruction, Analysis, and Visualization of Phylogenomic Data Huerta-Cepas, Jaime; Serra, François; Bork, Peer Molecular Biology and Evolution, Vol. 33, Issue 6 https://doi.org/10.1093/molbev/msw046	journal	February 2016
The Gene Ontology resource: enriching a GOld mine Carbon, Seth; Douglass, Eric; Good, Benjamin M. Nucleic Acids Research, Vol. 49, Issue D1 https://doi.org/10.1093/nar/gkaa1113	journal	December 2020
ViPR: an open bioinformatics database and analysis resource for virology research Pickett, Brett E.; Sadat, Eva L.; Zhang, Yun Nucleic Acids Research, Vol. 40, Issue D1 https://doi.org/10.1093/nar/gkr859	journal	October 2011
GONUTS: the Gene Ontology Normal Usage Tracking System Renfro, Daniel P.; McIntosh, Brenley K.; Venkatraman, Anand Nucleic Acids Research, Vol. 40, Issue D1 https://doi.org/10.1093/nar/gkr907	journal	November 2011
The Gene Wiki in 2011: community intelligence applied to human gene annotation Good, Benjamin M.; Clarke, Erik L.; de Alfaro, Luca Nucleic Acids Research, Vol. 40, Issue D1 https://doi.org/10.1093/nar/gkr925	journal	November 2011
The GOA database: Gene Ontology annotation updates for 2015 Huntley, Rachael P.; Sawford, Tony; Mutowo-Meullenet, Prudence Nucleic Acids Research, Vol. 43, Issue D1 https://doi.org/10.1093/nar/gku1113	journal	November 2014
Improvements to PATRIC, the all-bacterial Bioinformatics Database and Analysis Resource Center Wattam, Alice R.; Davis, James J.; Assaf, Rida Nucleic Acids Research, Vol. 45, Issue D1 https://doi.org/10.1093/nar/gkw1017	journal	November 2016
EuPathDB: the eukaryotic pathogen genomics database resource Aurrecoechea, Cristina; Barreto, Ana; Basenko, Evelina Y. Nucleic Acids Research, Vol. 45, Issue D1 https://doi.org/10.1093/nar/gkw1105	journal	November 2016
Influenza Research Database: An integrated bioinformatics resource for influenza virus research Zhang, Yun; Aevermann, Brian D.; Anderson, Tavis K. Nucleic Acids Research, Vol. 45, Issue D1 https://doi.org/10.1093/nar/gkw857	journal	September 2016
COFACTOR: improved protein function prediction by combining structure, sequence and protein–protein interaction information Zhang, Chengxin; Freddolino, Peter L.; Zhang, Yang Nucleic Acids Research, Vol. 45, Issue W1 https://doi.org/10.1093/nar/gkx366	journal	May 2017
UniProt: a worldwide hub of protein knowledge Consortium, The UniProt Nucleic Acids Research https://doi.org/10.1093/nar/gky1049		November 2018
The Gene Ontology Resource: 20 years and still GOing strong Nucleic Acids Research, Vol. 47, Issue D1, p. D330-D338 https://doi.org/10.1093/nar/gky1055	journal	November 2018
Alliance of Genome Resources Portal: unified model organism research platform Agapite, Julie; Albou, Laurent-Philippe; Aleksander, Suzi Nucleic Acids Research, Vol. 48, Issue D1 https://doi.org/10.1093/nar/gkz813	journal	September 2019
Matplotlib: A 2D Graphics Environment Hunter, John D. Computing in Science & Engineering, Vol. 9, Issue 3 https://doi.org/10.1109/MCSE.2007.55	journal	January 2007
Quantitative interaction proteomics reveals differences in the interactomes of amyloid precursor protein isoforms Andrew, Robert J.; Fisher, Kate; Heesom, Kate J. Journal of Neurochemistry, Vol. 149, Issue 3 https://doi.org/10.1111/jnc.14666	journal	February 2019
A Broadly Implementable Research Course in Phage Discovery and Genomics for First-Year Undergraduate Students Jordan, Tuajuanda C.; Burnett, Sandra H.; Carson, Susan mBio, Vol. 5, Issue 1 https://doi.org/10.1128/mBio.01051-13	journal	February 2014
Improving Interpretation of Cardiac Phenotypes and Enhancing Discovery With Expanded Knowledge in the Gene Ontology Lovering, Ruth C.; Roncaglia, Paola; Howe, Douglas G. Circulation: Genomic and Precision Medicine, Vol. 11, Issue 2 https://doi.org/10.1161/CIRCGEN.117.001813	journal	February 2018
How the gene ontology evolves Leonelli, Sabina; Diehl, Alexander D.; Christie, Karen R. BMC Bioinformatics, Vol. 12, Issue 1 https://doi.org/10.1186/1471-2105-12-325	journal	August 2011
Calling on a million minds for community annotation in WikiProteins Mons, Barend; Ashburner, Michael; Chichester, Christine Genome Biology, Vol. 9, Issue 5 https://doi.org/10.1186/gb-2008-9-5-r89	journal	January 2008
The C.R.E.A.T.E. Approach to Primary Literature Shifts Undergraduates’ Self-Assessed Ability to Read and Analyze Journal Articles, Attitudes about Science, and Epistemological Beliefs Hoskins, Sally G.; Lopatto, David; Stevens, Leslie M. CBE—Life Sciences Education, Vol. 10, Issue 4 https://doi.org/10.1187/cbe.11-03-0027	journal	December 2011
Figure Facts: Encouraging Undergraduates to Take a Data-Centered Approach to Reading Primary Literature Round, Jennifer E.; Campbell, A. Malcolm CBE—Life Sciences Education, Vol. 12, Issue 1 https://doi.org/10.1187/cbe.11-07-0057	journal	March 2013
On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other Mann, H. B.; Whitney, D. R. The Annals of Mathematical Statistics, Vol. 18, Issue 1 https://doi.org/10.1214/aoms/1177730491	journal	March 1947
From zebrafish heart jogging genes to mouse and human orthologs: using Gene Ontology to investigate mammalian heart development. Khodiyar, Varsha K.; Howe, Doug; Talmud, Philippa J. F1000Research, Vol. 2 https://doi.org/10.12688/f1000research.2-242.v1	journal	January 2013
Incorporating Genomics and Bioinformatics across the Life Sciences Curriculum Ditty, Jayna L.; Kvaal, Christopher A.; Goodner, Brad PLoS Biology, Vol. 8, Issue 8 https://doi.org/10.1371/journal.pbio.1000448	journal	August 2010
Quality of Computationally Inferred Gene Ontology Annotations Škunca, Nives; Altenhoff, Adrian; Dessimoz, Christophe PLoS Computational Biology, Vol. 8, Issue 5 https://doi.org/10.1371/journal.pcbi.1002533	journal	May 2012
A quick guide for student-driven community genome annotation Hosmani, Prashant S.; Shippy, Teresa; Miller, Sherry PLOS Computational Biology, Vol. 15, Issue 4 https://doi.org/10.1371/journal.pcbi.1006682	journal	April 2019
Galaxy and Apollo as a biologist-friendly interface for high-quality cooperative phage genome annotation Ramsey, Jolene; Rasche, Helena; Maughmer, Cory PLOS Computational Biology, Vol. 16, Issue 11 https://doi.org/10.1371/journal.pcbi.1008214	journal	November 2020
Global Analysis of Fission Yeast Mating Genes Reveals New Autophagy Factors Sun, Ling-Ling; Li, Ming; Suo, Fang PLoS Genetics, Vol. 9, Issue 8 https://doi.org/10.1371/journal.pgen.1003715	journal	August 2013
Science through Wikipedia: A novel representation of open knowledge through co-citation networks Arroyo-Machado, Wenceslao; Torres-Salinas, Daniel; Herrera-Viedma, Enrique PLOS ONE, Vol. 15, Issue 2 https://doi.org/10.1371/journal.pone.0228713	journal	February 2020
Exploring autophagy with Gene Ontology Denny, Paul; Feuermann, Marc; Hill, David P. Taylor & Francis https://doi.org/10.6084/m9.figshare.5899675.v1	dataset	January 2018

Similar Records

ECO, the Evidence & Conclusion Ontology: community standard for evidence information

Journal Article · Thu Nov 08 00:00:00 UTC 2018 · Nucleic Acids Research · OSTI ID:1625570

Giglio, Michelle; Tauber, Rebecca; Nadendla, Suvarna; +12 more

Strategies for community-sourced biocuration in bioinformatics: a case study on MIBiG 4.0

Journal Article · Thu Dec 11 00:00:00 UTC 2025 · Briefings in Bioinformatics · OSTI ID:3013841

Blin, Kai; Loureiro, Catarina; Louwen, Nico L. L.; +10 more

A quick guide for student-driven community genome annotation

Journal Article · Wed Apr 03 00:00:00 UTC 2019 · PLoS Computational Biology (Online) · OSTI ID:1627257

Hosmani, Prashant S.; Shippy, Teresa; Miller, Sherry; +8 more

Related Subjects

59 BASIC BIOLOGICAL SCIENCES
consortia
gene identification and analysis
gene ontologies
genome annotation
model organisms
ontologies
scientists
undergraduates

Title: Crowdsourcing biocuration: The Community Assessment of Community Annotation with Ontologies (CACAO)

Citation Formats

References (55)

Similar Records

Related Subjects