DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Choosing experiments to accelerate collective discovery

Abstract

Scientists perform a tiny subset of all possible experiments. What characterizes the experiments they choose? What are the consequences of those choices for the pace of scientific discovery? We model scientific knowledge as a network and science as a sequence of experiments designed to gradually uncover it. By analyzing millions of biomedical articles published over 30 y, we find that biomedical scientists pursue conservative research strategies exploring the local neighborhood of central, important molecules. Although such strategies probably serve scientific careers, we show that they slow scientific advance, especially in mature fields, where more risk and less redundant experimentation would accelerate discovery of the network. Lastly, we also consider institutional arrangements that could help science pursue these more efficient strategies.

Authors:
 [1];  [2];  [3]; ORCiD logo [4]
  1. Univ. of Chicago, IL (United States). Dept. of Medicine and Human Genetics; Univ. of Chicago and Argonne National Laboratory, Chicago, IL (United States); Univ. of Chicago, IL (United States). Inst. of Genomic and Systems Biology
  2. Univ. of California, Los Angeles, CA (United States)
  3. Univ. of Chicago and Argonne National Laboratory, Chicago, IL (United States); Argonne National Lab. (ANL), Argonne, IL (United States). Mathematics and Computer Science Division
  4. Univ. of Chicago and Argonne National Laboratory, Chicago, IL (United States); Argonne National Lab. (ANL), Argonne, IL (United States); Univ. of Chicago, IL (United States). Dept. of Sociology
Publication Date:
Research Org.:
Argonne National Laboratory (ANL), Argonne, IL (United States)
Sponsoring Org.:
USDOE Office of Science (SC); National Science Foundation (NSF); National Institutes of Health (NIH); US Air Force Office of Scientific Research (AFOSR)
OSTI Identifier:
1244548
Grant/Contract Number:  
AC02-06CH11357; SBE 0915730; 1P50MH094267; U01HL108634-01; W911NF1410333
Resource Type:
Accepted Manuscript
Journal Name:
Proceedings of the National Academy of Sciences of the United States of America
Additional Journal Information:
Journal Volume: 112; Journal Issue: 47; Journal ID: ISSN 0027-8424
Publisher:
National Academy of Sciences, Washington, DC (United States)
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; complex networks; computational biology; science of science; innovation; sociology of science

Citation Formats

Rzhetsky, Andrey, Foster, Jacob G., Foster, Ian T., and Evans, James A. Choosing experiments to accelerate collective discovery. United States: N. p., 2015. Web. doi:10.1073/pnas.1509757112.
Rzhetsky, Andrey, Foster, Jacob G., Foster, Ian T., & Evans, James A. Choosing experiments to accelerate collective discovery. United States. https://doi.org/10.1073/pnas.1509757112
Rzhetsky, Andrey, Foster, Jacob G., Foster, Ian T., and Evans, James A. Tue . "Choosing experiments to accelerate collective discovery". United States. https://doi.org/10.1073/pnas.1509757112. https://www.osti.gov/servlets/purl/1244548.
@article{osti_1244548,
title = {Choosing experiments to accelerate collective discovery},
author = {Rzhetsky, Andrey and Foster, Jacob G. and Foster, Ian T. and Evans, James A.},
abstractNote = {Scientists perform a tiny subset of all possible experiments. What characterizes the experiments they choose? What are the consequences of those choices for the pace of scientific discovery? We model scientific knowledge as a network and science as a sequence of experiments designed to gradually uncover it. By analyzing millions of biomedical articles published over 30 y, we find that biomedical scientists pursue conservative research strategies exploring the local neighborhood of central, important molecules. Although such strategies probably serve scientific careers, we show that they slow scientific advance, especially in mature fields, where more risk and less redundant experimentation would accelerate discovery of the network. Lastly, we also consider institutional arrangements that could help science pursue these more efficient strategies.},
doi = {10.1073/pnas.1509757112},
journal = {Proceedings of the National Academy of Sciences of the United States of America},
number = 47,
volume = 112,
place = {United States},
year = {Tue Nov 24 00:00:00 EST 2015},
month = {Tue Nov 24 00:00:00 EST 2015}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Citation Metrics:
Cited by: 105 works
Citation information provided by
Web of Science

Save / Share:

Works referenced in this record:

The Increasing Dominance of Teams in Production of Knowledge
journal, May 2007


Theory Choice and Problem Choice in Science
journal, July 1978


Exploration and Exploitation in Organizational Learning
journal, February 1991


Swift/T: Large-scale Application Composition via Distributed-memory Dataflow Processing
text, January 2013


Why Most Published Research Findings Are False
journal, January 2019


Tradition and Innovation in Scientists’ Research Strategies
journal, September 2015

  • Foster, Jacob G.; Rzhetsky, Andrey; Evans, James A.
  • American Sociological Review, Vol. 80, Issue 5
  • DOI: 10.1177/0003122415601618

Collaborative learning in networks
journal, December 2011

  • Mason, W.; Watts, D. J.
  • Proceedings of the National Academy of Sciences, Vol. 109, Issue 3
  • DOI: 10.1073/pnas.1110069108

The 'wired' universe of organic chemistry
journal, April 2009

  • Grzybowski, Bartosz A.; Bishop, Kyle J. M.; Kowalczyk, Bartlomiej
  • Nature Chemistry, Vol. 1, Issue 1
  • DOI: 10.1038/nchem.136

Rewiring Chemistry: Algorithmic Discovery and Experimental Validation of One-Pot Reactions in the Network of Organic Chemistry
journal, July 2012

  • Gothard, Chris M.; Soh, Siowling; Gothard, Nosheen A.
  • Angewandte Chemie, Vol. 124, Issue 32
  • DOI: 10.1002/ange.201202155

Atypical Combinations and Scientific Impact
journal, October 2013


Laboratory Life
journal, June 2019


The Temporal Structure of Scientific Consensus Formation
journal, December 2010


PharmGKB: the Pharmacogenetics Knowledge Base
journal, January 2002


Navigation in a small world
book, December 2011


Probing genetic overlap among complex human phenotypes
journal, July 2007

  • Rzhetsky, A.; Wajngurt, D.; Park, N.
  • Proceedings of the National Academy of Sciences, Vol. 104, Issue 28
  • DOI: 10.1073/pnas.0704820104

Probing genetic overlap among complex human phenotypes
text, January 2006

  • Rzhetsky, Andrey; Wajngurt, David; Park, Naeun
  • Columbia University
  • DOI: 10.7916/d8ms3rpr

Matthew: Effect or Fable?
journal, January 2014


Why Most Published Research Findings Are False
journal, September 2005


The Division of Cognitive Labor
journal, January 1990

  • Kitcher, Philip
  • The Journal of Philosophy, Vol. 87, Issue 1
  • DOI: 10.2307/2026796

Sampling from large graphs
conference, January 2006

  • Leskovec, Jure; Faloutsos, Christos
  • Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD '06
  • DOI: 10.1145/1150402.1150479

Mapping the Semantic Structure of Cognitive Neuroscience
text, January 2014

  • Elizabeth, Beam,; Jordynn, Jack,; A., Huettel, Scott
  • The University of North Carolina at Chapel Hill University Libraries
  • DOI: 10.17615/dw65-ed34

Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market
journal, February 2006


Perceived Criteria for Research Problem Choice in the Agricultural Sciences--A Research Note
journal, September 1983


Using BLAST for identifying gene and protein names in journal articles
journal, December 2000


powerlaw: A Python Package for Analysis of Heavy-Tailed Distributions
journal, January 2014


Gender Differences in Patenting in the Academic Life Sciences
journal, August 2006


A probabilistic similarity metric for Medline records: A model for author name disambiguation
journal, January 2004

  • Torvik, Vetle I.; Weeber, Marc; Swanson, Don R.
  • Journal of the American Society for Information Science and Technology, Vol. 56, Issue 2
  • DOI: 10.1002/asi.20105

Navigation in a small world
journal, August 2000


An approximation of the inverse moments of the positive hypergeometric distribution
journal, January 1978


Evolution of the social network of scientific collaborations
journal, August 2002

  • Barabási, A. L.; Jeong, H.; Néda, Z.
  • Physica A: Statistical Mechanics and its Applications, Vol. 311, Issue 3-4
  • DOI: 10.1016/S0378-4371(02)00736-7

Why most published research findings are false
journal, September 2008


The Temporal Structure of Scientific Consensus Formation
text, January 2010


Priorities in Scientific Discovery: A Chapter in the Sociology of Science
journal, December 1957

  • Merton, Robert K.
  • American Sociological Review, Vol. 22, Issue 6
  • DOI: 10.2307/2089193

On Simultaneous Confidence Intervals for Multinomial Proportions
journal, May 1965


Power-Law Distributions in Empirical Data
journal, November 2009

  • Clauset, Aaron; Shalizi, Cosma Rohilla; Newman, M. E. J.
  • SIAM Review, Vol. 51, Issue 4
  • DOI: 10.1137/070710111

Problem Retention and Problem Change in Science
journal, July 1978


Weaving the fabric of science: Dynamic network models of science's unfolding structure
journal, October 2015


Main Trends in Recent Philosophy: Two Dogmas of Empiricism
journal, January 1951

  • Quine, W. V.
  • The Philosophical Review, Vol. 60, Issue 1
  • DOI: 10.2307/2181906

The specificity of the scientific field and the social conditions of the progress of reason
journal, January 1975


Collective dynamics of ‘small-world’ networks
journal, June 1998

  • Watts, Duncan J.; Strogatz, Steven H.
  • Nature, Vol. 393, Issue 6684
  • DOI: 10.1038/30918

Power-law distributions in empirical data
text, January 2018


Why Most Published Research Findings Are False
journal, August 2005


Mechanisms for (mis)allocating scientific credit
conference, January 2011

  • Kleinberg, Jon; Oren, Sigal
  • Proceedings of the 43rd annual ACM symposium on Theory of computing - STOC '11
  • DOI: 10.1145/1993636.1993707

Mapping the Dynamics of Science and Technology
book, January 1986


Matthew: Effect or Fable?
report, December 2012


Social distancing and epidemic resurgence in agent-based susceptible-infectious-recovered models
journal, January 2021

  • Mukhamadiarov, Ruslan I.; Deng, Shengfeng; Serrao, Shannon R.
  • Scientific Reports, Vol. 11, Issue 1
  • DOI: 10.1038/s41598-020-80162-y

Structural Holes and Good Ideas
journal, September 2004

  • Burt, Ronald S.
  • American Journal of Sociology, Vol. 110, Issue 2
  • DOI: 10.1086/421787

Rewiring Chemistry: Algorithmic Discovery and Experimental Validation of One-Pot Reactions in the Network of Organic Chemistry
journal, July 2012

  • Gothard, Chris M.; Soh, Siowling; Gothard, Nosheen A.
  • Angewandte Chemie International Edition, Vol. 51, Issue 32
  • DOI: 10.1002/anie.201202155

Swift/T: Large-Scale Application Composition via Distributed-Memory Dataflow Processing
conference, May 2013

  • Wozniak, J. M.; Armstrong, T. G.; Wilde, M.
  • 2013 13th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), 2013 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing
  • DOI: 10.1109/CCGrid.2013.99

Epistemic Landscapes and the Division of Cognitive Labor*
journal, April 2009

  • Weisberg, Michael; Muldoon, Ryan
  • Philosophy of Science, Vol. 76, Issue 2
  • DOI: 10.1086/644786

The Social Process of Scientific Investigation
book, January 1980


How to Make More Published Research True
journal, October 2014


The Structure and Function of Complex Networks
journal, January 2003


The Scientist as an Analogical Reasoner: A Critique of the Metaphor Theory of Innovation
book, January 1980


Emergent behavior of growing knowledge about molecular interactions
journal, October 2005

  • Cokol, Murat; Iossifov, Ivan; Weinreb, Chani
  • Nature Biotechnology, Vol. 23, Issue 10
  • DOI: 10.1038/nbt1005-1243

Different personal propensities among scientists relate to deeper vs. broader knowledge contributions
journal, March 2015


The Burden of Knowledge and the 'Death of the Renaissance Man': Is Innovation Getting Harder?
journal, January 2005


The dynamics of correlated novelties
journal, July 2014

  • Tria, F.; Loreto, V.; Servedio, V. D. P.
  • Scientific Reports, Vol. 4, Issue 1
  • DOI: 10.1038/srep05890

Turbine: a distributed-memory dataflow engine for extreme-scale many-task applications
conference, January 2012

  • Wozniak, Justin M.; Armstrong, Timothy G.; Maheshwari, Ketan
  • Proceedings of the 1st ACM SIGMOD Workshop on Scalable Workflow Execution Engines and Technologies - SWEET '12
  • DOI: 10.1145/2443416.2443421

The Burden of Knowledge and the “Death of the Renaissance Man”: Is Innovation Getting Harder?
journal, January 2009


Emergence of Scaling in Random Networks
journal, October 1999


Mapping the Semantic Structure of Cognitive Neuroscience
journal, September 2014

  • Beam, Elizabeth; Appelbaum, L. Gregory; Jack, Jordynn
  • Journal of Cognitive Neuroscience, Vol. 26, Issue 9
  • DOI: 10.1162/jocn_a_00604

Knowledge Specialization, Knowledge Brokerage and the Uneven Growth of Technology Domains
journal, December 2009


Industry collaboration, scientific sharing, and the dissemination of knowledge
journal, September 2010


Power-law distributions in empirical data
text, January 2018


Perceived Criteria for Research Problem Choice in the Agricultural Sciences-A Research Note
journal, September 1983

  • Busch, Lawrence; Lacy, William B.; Sachs, Carolyn
  • Social Forces, Vol. 62, Issue 1
  • DOI: 10.2307/2578355

Optimization by Simulated Annealing
journal, May 1983


Reputation and impact in academic careers
journal, October 2014

  • Petersen, A. M.; Fortunato, S.; Pan, R. K.
  • Proceedings of the National Academy of Sciences, Vol. 111, Issue 43
  • DOI: 10.1073/pnas.1323111111

Agent-Based Models of Science
book, October 2011


On Simultaneous Confidence Intervals for Multinomial Proportions
journal, May 1965


Power-law distributions in empirical data
text, January 2007


Powerlaw: a Python package for analysis of heavy-tailed distributions
text, January 2013


Evolution of the social network of scientific collaborations
text, January 2001


Works referencing / citing this record:

Quantifying patterns of research-interest evolution
journal, March 2017

  • Jia, Tao; Wang, Dashun; Szymanski, Boleslaw K.
  • Nature Human Behaviour, Vol. 1, Issue 4
  • DOI: 10.1038/s41562-017-0078

The Possibility of Systematic Research Fraud Targeting Under-Studied Human Genes: Causes, Consequences, and Potential Solutions
journal, January 2019


Peer Review and Scholarly Originality: Let 1,000 Flowers Bloom, but Don’t Step on Any
journal, August 2016


Organisational factors and academic research agendas: an analysis of academics in the social sciences
journal, May 2019


Toward a more scientific science
journal, September 2018


The association of thinking styles with research agendas among academics in the social sciences
journal, April 2020

  • Santos, João M.; Horta, Hugo; Zhang, Li‐fang
  • Higher Education Quarterly, Vol. 74, Issue 2
  • DOI: 10.1111/hequ.12240

Early-career setback and future career impact
journal, October 2019


Science of science
journal, May 2021


Competition for novelty reduces information sampling in a research game - a registered report
journal, May 2019

  • Tiokhin, Leonid; Derex, Maxime
  • Royal Society Open Science, Vol. 6, Issue 5
  • DOI: 10.1098/rsos.180934

Explore with caution: mapping the evolution of scientific interest in physics
journal, September 2019


Network analysis of synthesizable materials discovery
journal, May 2019


A Nobel opportunity for interdisciplinarity
journal, November 2018


Theoretical research without projects
journal, March 2019


Scientific prize network predicts who pushes the boundaries of science
journal, December 2018

  • Ma, Yifang; Uzzi, Brian
  • Proceedings of the National Academy of Sciences, Vol. 115, Issue 50
  • DOI: 10.1073/pnas.1800485115

Science of science
journal, February 1974


Large-scale investigation of the reasons why potentially important genes are ignored
journal, September 2018


Increasing trend of scientists to switch between topics
journal, July 2019


Scientific productivity: An exploratory study of metrics and incentives
journal, April 2018


Science of science
journal, March 2018


Evolution of semantic networks in biomedical texts
journal, June 2019


Validation and Topic-driven Ranking for Biomedical Hypothesis Generation Systems
posted_content, February 2018


Literature-based automated discovery of tumor suppressor p53 phosphorylation and inhibition by NEK2
journal, September 2018

  • Choi, Byung-Kwon; Dayaram, Tajhal; Parikh, Neha
  • Proceedings of the National Academy of Sciences, Vol. 115, Issue 42
  • DOI: 10.1073/pnas.1806643115

Neuroanatomical Substrates for Risk Behavior
journal, May 2016


Efficient team structures in an open-ended cooperative creativity experiment
journal, October 2019

  • Monechi, Bernardo; Pullano, Giulia; Loreto, Vittorio
  • Proceedings of the National Academy of Sciences, Vol. 116, Issue 44
  • DOI: 10.1073/pnas.1909827116

Network dynamics of innovation processes
text, January 2017


Network analysis of synthesizable materials discovery
text, January 2018


Theoretical research without projects
text, January 2018


Increasing trend of scientists to switch between topics
text, January 2018


Early-career setback and future career impact
text, January 2019


Explore with caution: mapping the evolution of scientific interest in Physics
preprint, January 2019


Supporting novel biomedical research via multilayer collaboration networks
journal, November 2016

  • Kuzmin, Konstantin; Lu, Xiaoyan; Mukherjee, Partha Sarathi
  • Applied Network Science, Vol. 1, Issue 1
  • DOI: 10.1007/s41109-016-0015-y

Increasing trend of scientists to switch between topics
journal, July 2019


Quantifying patterns of research-interest evolution
journal, March 2017

  • Jia, Tao; Wang, Dashun; Szymanski, Boleslaw K.
  • Nature Human Behaviour, Vol. 1, Issue 4
  • DOI: 10.1038/s41562-017-0078

Literature-based automated discovery of tumor suppressor p53 phosphorylation and inhibition by NEK2
journal, September 2018

  • Choi, Byung-Kwon; Dayaram, Tajhal; Parikh, Neha
  • Proceedings of the National Academy of Sciences, Vol. 115, Issue 42
  • DOI: 10.1073/pnas.1806643115

Competition for novelty reduces information sampling in a research game - a registered report
journal, May 2019

  • Tiokhin, Leonid; Derex, Maxime
  • Royal Society Open Science, Vol. 6, Issue 5
  • DOI: 10.1098/rsos.180934

Validation and Topic-driven Ranking for Biomedical Hypothesis Generation Systems
posted_content, February 2018


Neuroanatomical Substrates for Risk Behavior
journal, May 2016


The Possibility of Systematic Research Fraud Targeting Under-Studied Human Genes: Causes, Consequences, and Potential Solutions
journal, January 2019


Theoretical research without projects
text, January 2018


Early-career setback and future career impact
text, January 2019


Science and Technology Advance through Surprise
preprint, January 2019


The effect of novelty on the future impact of scientific grants
preprint, January 2019


A community-powered search of machine learning strategy space to find NMR property prediction models
text, January 2020


Biomedical Convergence Facilitated by the Emergence of Technological and Informatic Capabilities
preprint, January 2021