DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: A meta-analysis of semantic classification of citations

Journal Article · · Quantitative science studies

The aim of this literature review is to examine the current state of the art in the area of citation classification. In particular, we investigate the approaches for characterizing citations based on their semantic type. We conduct this literature review as a meta-analysis covering 60 scholarly articles in this domain. Although we included some of the manual pioneering works in this review, more emphasis is placed on the later automated methods, which use Machine Learning and Natural Language Processing (NLP) for analyzing the fine-grained linguistic features in the surrounding text of citations. The sections are organized based on the steps involved in the pipeline for citation classification. Specifically, we explore the existing classification schemes, data sets, pre-processing methods, extraction of contextual and non-contextual features, and the different types of classifiers and evaluation approaches. The review highlights the importance of identifying the citation types for research evaluation, the challenges faced by the researchers in the process, and the existing research gaps in this field.

Research Organization:
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
Sponsoring Organization:
Jisc; USDOE
Grant/Contract Number:
AC05-00OR22725
OSTI ID:
1831680
Journal Information:
Quantitative science studies, Journal Name: Quantitative science studies Journal Issue: 4 Vol. 2; ISSN 2641-3337
Publisher:
MIT PressCopyright Statement
Country of Publication:
United States
Language:
English

References (56)

Measuring academic influence: Not all citations are equal: Measuring Academic Influence journal May 2014
Highly cited old papers and the reasons why they continue to be cited journal September 1978
Private acts and public objects: An investigation of citer motivations journal July 1985
The norms of citation behavior: Prolegomena to the footnote journal July 1965
Counting citations in texts rather than reference lists to improve the accuracy of assessing scientific contribution: Citation frequency of individual articles in other papers more fairly measures their scientific contribution than mere presence in reference lists journal August 2011
Is citation analysis a legitimate evaluation tool? journal May 1979
Neural ParsCit: a deep learning-based reference string parser journal May 2018
CERMINE: automatic extraction of structured metadata from scientific literature journal July 2015
The ACL anthology network corpus journal January 2013
The linguistic patterns and rhetorical structure of citation context: an approach using n-grams journal September 2016
Do citations and readership identify seminal publications? journal February 2018
A novel machine-learning approach to measuring scientific knowledge flows using citation context analysis journal May 2018
Citance-based retrieval and summarization using IR and machine learning journal July 2018
Identification of important citations by exploiting research articles’ metadata and cue-terms from content journal November 2018
What do citation counts measure? An updated review of studies on citations in scientific documents published between 2006 and 2018 journal September 2019
Important citation identification by exploiting the syntactic and contextual information of citations journal September 2020
Multi-task learning model based on recurrent convolutional neural networks for citation sentiment and purpose classification journal March 2019
An interview-based study of the functions of citations in academic writing across two disciplines journal March 2009
Survey about citation context analysis: Tasks, techniques, and resources journal November 2015
NLP-driven citation analysis for scientometrics journal January 2016
Citation function, polarity and influence classification journal April 2017
Semantic Enrichment of Scientific Publications and Metadata: Citation Analysis Through Contextual and Cognitive Analysis journal July 2012
Citation Analysis and Discourse Analysis Revisited journal March 2004
Citation Analysis and Discourse Analysis journal January 1986
Citations, contexts, and humanistic discourse: Toward automatic extraction and classification journal May 2014
What do citation counts measure? A review of studies on citing behavior journal January 2008
Citation Context Analysis using Word-Graph conference March 2019
Semi-Automatic Annotation for Citation Function Classification conference May 2018
Important Citation Identification by Exploiting the Optimal In-text Citation Frequency conference February 2020
Identifying Important Citations Using Contextual Information from Full Text conference June 2017
ACT: An Annotation Platform for Citation Typing at Scale conference June 2019
Neural Multi-task Learning for Citation Function and Provenance conference June 2019
Citation Analysis as a Tool in Journal Evaluation: Journals can be ranked by frequency and impact of citations for science policy studies. journal November 1972
PDFX: fully-automated PDF-to-XML conversion of scientific literature
  • Constantin, Alexandru; Pettifer, Steve; Voronkov, Andrei
  • DocEng '13: ACM Symposium on Document Engineering 2013, Proceedings of the 2013 ACM symposium on Document engineering https://doi.org/10.1145/2494266.2494271
conference September 2013
An Overview of Microsoft Academic Service (MAS) and Applications
  • Sinha, Arnab; Shen, Zhihong; Song, Yang
  • WWW '15: 24th International World Wide Web Conference, Proceedings of the 24th International Conference on World Wide Web https://doi.org/10.1145/2740908.2742839
conference May 2015
Investigating Convolutional Networks and Domain-Specific Embeddings for Semantic Classification of Citations
  • Lauscher, Anne; Glavaš, Goran; Ponzetto, Simone Paolo
  • WOSP 2017: 6th International Workshop on Mining Scientific Publications, Proceedings of the 6th International Workshop on Mining Scientific Publications https://doi.org/10.1145/3127526.3127531
conference December 2017
An Authoritative Approach to Citation Classification
  • Pride, David; Knoth, Petr
  • JCDL '20: The ACM/IEEE Joint Conference on Digital Libraries in 2020, Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020 https://doi.org/10.1145/3383583.3398617
conference August 2020
Sharing Is Caring: The Future of Shared Tasks journal December 2017
Microsoft Academic Graph: When experts are not enough journal February 2020
Measuring the Evolution of a Scientific Field through Citation Frames journal December 2018
Some Results on the Function and Quality of Citations journal February 1975
Content Analysis of References: Adjunct or Alternative to Citation Counting? journal November 1975
Science Studies: Bibliometric and Content Analysis journal February 1977
Referencing as Persuasion journal February 1977
Citations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories journal January 2019
CiTO, the Citation Typing Ontology journal January 2010
Concentration of the Most-Cited Papers in the Scientific Literature: Analysis of Journal Ecosystems journal December 2006
Important citation identification by exploiting content and section-wise in-text citation count journal March 2020
S2ORC: The Semantic Scholar Open Research Corpus conference January 2020
SciBERT: A Pretrained Language Model for Scientific Text
  • Beltagy, Iz; Lo, Kyle; Cohan, Arman
  • Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) https://doi.org/10.18653/v1/D19-1371
conference January 2019
Structural Scaffolds for Citation Intent Classification in Scientific Publications conference January 2019
Citation Analysis with Neural Attention Models
  • Munkhdalai, Tsendsuren; Lalor, John; Yu, Hong
  • Proceedings of the Seventh International Workshop on Health Text Mining and Information Analysis https://doi.org/10.18653/v1/W16-6109
conference January 2016
Citation Block Determination Using Textual Coherence journal January 2016
Automatic classification of citation function conference January 2006
An annotation scheme for citation function conference January 2006
Classification of research papers using citation links and citation types: Towards automatic review article generation. journal November 2000