skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Do citations and readership identify seminal publications?

Abstract

Here, this work presents a new approach for analysing the ability of existing research metrics to identify research which has strongly influenced future developments. More specifically, we focus on the ability of citation counts and Mendeley reader counts to distinguish between publications regarded as seminal and publications regarded as literature reviews by field experts. The main motivation behind our research is to gain a better understanding of whether and how well the existing research metrics relate to research quality. For this experiment we have created a new dataset which we call TrueImpactDataset and which contains two types of publications, seminal papers and literature reviews. Using the dataset, we conduct a set of experiments to study how citation and reader counts perform in distinguishing these publication types, following the intuition that causing a change in a field signifies research quality. Finally, our research shows that citation counts work better than a random baseline (by a margin of 10%) in distinguishing important seminal research papers from literature reviews while Mendeley reader counts do not work better than the baseline.

Authors:
ORCiD logo [1]; ORCiD logo [2]; ORCiD logo [3]; ORCiD logo [2]
  1. Open Univ., Milton Keynes (United Kingdom); Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
  2. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
  3. Open Univ., Milton Keynes (United Kingdom)
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
1425336
Grant/Contract Number:  
AC05-00OR22725
Resource Type:
Journal Article: Accepted Manuscript
Journal Name:
Scientometrics
Additional Journal Information:
Journal Volume: 115; Journal Issue: 1; Journal ID: ISSN 0138-9130
Country of Publication:
United States
Language:
English
Subject:
99 GENERAL AND MISCELLANEOUS; Information retrieval; Scholarly communication; Publication datasets; Data mining; Research evaluation; Bibliometrics; Altmetrics

Citation Formats

Herrmannova, Drahomira, Patton, Robert M., Knoth, Petr, and Stahl, Christopher G. Do citations and readership identify seminal publications?. United States: N. p., 2018. Web. doi:10.1007/s11192-018-2669-y.
Herrmannova, Drahomira, Patton, Robert M., Knoth, Petr, & Stahl, Christopher G. Do citations and readership identify seminal publications?. United States. doi:10.1007/s11192-018-2669-y.
Herrmannova, Drahomira, Patton, Robert M., Knoth, Petr, and Stahl, Christopher G. Sat . "Do citations and readership identify seminal publications?". United States. doi:10.1007/s11192-018-2669-y. https://www.osti.gov/servlets/purl/1425336.
@article{osti_1425336,
title = {Do citations and readership identify seminal publications?},
author = {Herrmannova, Drahomira and Patton, Robert M. and Knoth, Petr and Stahl, Christopher G.},
abstractNote = {Here, this work presents a new approach for analysing the ability of existing research metrics to identify research which has strongly influenced future developments. More specifically, we focus on the ability of citation counts and Mendeley reader counts to distinguish between publications regarded as seminal and publications regarded as literature reviews by field experts. The main motivation behind our research is to gain a better understanding of whether and how well the existing research metrics relate to research quality. For this experiment we have created a new dataset which we call TrueImpactDataset and which contains two types of publications, seminal papers and literature reviews. Using the dataset, we conduct a set of experiments to study how citation and reader counts perform in distinguishing these publication types, following the intuition that causing a change in a field signifies research quality. Finally, our research shows that citation counts work better than a random baseline (by a margin of 10%) in distinguishing important seminal research papers from literature reviews while Mendeley reader counts do not work better than the baseline.},
doi = {10.1007/s11192-018-2669-y},
journal = {Scientometrics},
number = 1,
volume = 115,
place = {United States},
year = {Sat Feb 10 00:00:00 EST 2018},
month = {Sat Feb 10 00:00:00 EST 2018}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Save / Share: