A structural equation modeling approach to leveraging the power of extant sentiment analysis tools

Lebakula, Viswadeep; Porter, Ben; Stubbs-Richardson, Megan; Ray, Taylor; Cosby, Arthur; Bethel, Cindy

doi:10.1007/s42001-024-00334-y

Title: A structural equation modeling approach to leveraging the power of extant sentiment analysis tools

Journal Article · 25 November 2024 · Journal of computational social science (Online)

DOI: https://doi.org/10.1007/s42001-024-00334-y · OSTI ID:2480054

^[1]; Porter, Ben ^[2]; Stubbs-Richardson, Megan ^[2]; Ray, Taylor ^[2]; Cosby, Arthur ^[2]; Bethel, Cindy ^[2]

Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
Mississippi State Univ., Mississippi State, MS (United States)

Machine-derived sentiment analysis has become a pervasive and useful tool to address a wide array of issues in natural language processing. Leading technology companies such as Google now provide sentiment analysis tools (SATs) as readily accessible online products. Academic researchers develop and make available SATs to support the research enterprise. One of the major challenges with SATs is the inconsistencies in results among the various SATs. Consequently, the selection of a SAT for a specific purpose may significantly impact the application. This study addresses the foregoing problem by utilizing structural equation modeling to merge the outputs of SATs to develop a combined sentiment metric without the need for a labeled training dataset. This method is applicable to a wide range of text-based problems, is data-driven, and replicable. It was tested using three publicly available datasets and compared against seven different SATs. The results indicate that as a continous measure, the proposed method outperformed other SATs in the movie reviews and SemEval datasets, and achieved a tie for first place with IBM Watson on the Sentiment 140 dataset. Also, compared to the published major alternatives, the arithmetic mean solution, this approach performed better across these three datasets.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Organization:: USDOE

Grant/Contract Number:: AC05-00OR22725

OSTI ID:: 2480054

Journal Information:: Journal of computational social science (Online), Journal Name: Journal of computational social science (Online) Journal Issue: 1 Vol. 8; ISSN 2432-2725

Publisher:: Springer NatureCopyright Statement

Country of Publication:: United States

Language:: English

References (33)

Index for rating diagnostic tests Youden, W. J. Cancer, Vol. 3, Issue 1 https://doi.org/10.1002/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3	journal	January 1950
A General Method for Estimating a Linear Structural Equation System* Jöreskog, Karl G. ETS Research Bulletin Series, Vol. 1970, Issue 2 https://doi.org/10.1002/j.2333-8504.1970.tb00783.x	journal	December 1970
Combining Sentiment Analysis Scores to Improve Accuracy of Polarity Classification in MOOC Posts Pinto, Herbert Laroca; Rocio, Vitor Lecture Notes in Computer Science, p. 35-46 https://doi.org/10.1007/978-3-030-30241-2_4	book	August 2019
Successes and challenges in developing a hybrid approach to sentiment analysis Appel, Orestes; Chiclana, Francisco; Carter, Jenny Applied Intelligence https://doi.org/10.1007/s10489-017-0966-4	journal	June 2017
Sentiment analysis of political communication: combining a dictionary approach with crowdcoding Haselmayer, Martin; Jenny, Marcelo Quality & Quantity, Vol. 51, Issue 6 https://doi.org/10.1007/s11135-016-0412-4	journal	September 2016
Multilingual Sentiment Analysis: State of the Art and Independent Comparison of Techniques Dashtipour, Kia; Poria, Soujanya; Hussain, Amir Cognitive Computation, Vol. 8, Issue 4 https://doi.org/10.1007/s12559-016-9415-7	journal	June 2016
A review of sentiment analysis techniques for opinionated web text Singh, Jaspreet; Singh, Gurvinder; Singh, Rajinder CSI Transactions on ICT, Vol. 4, Issue 2-4 https://doi.org/10.1007/s40012-016-0107-y	journal	December 2016
An empirical study of sentiment analysis utilizing machine learning and deep learning algorithms Erkantarci, Betul; Bakal, Gokhan Journal of Computational Social Science, Vol. 7, Issue 1 https://doi.org/10.1007/s42001-023-00236-5	journal	December 2023
Tutorial on maximum likelihood estimation Myung, In Jae Journal of Mathematical Psychology, Vol. 47, Issue 1, p. 90-100 https://doi.org/10.1016/S0022-2496(02)00028-7	journal	February 2003
Sentiment analysis in medical settings: New opportunities and challenges Denecke, Kerstin; Deng, Yihan Artificial Intelligence in Medicine, Vol. 64, Issue 1 https://doi.org/10.1016/j.artmed.2015.03.006	journal	May 2015
Sentiment analysis algorithms and applications: A survey Medhat, Walaa; Hassan, Ahmed; Korashy, Hoda Ain Shams Engineering Journal, Vol. 5, Issue 4 https://doi.org/10.1016/j.asej.2014.04.011	journal	December 2014
The evolution of sentiment analysis—A review of research topics, venues, and top cited papers Mäntylä, Mika V.; Graziotin, Daniel; Kuutila, Miikka Computer Science Review, Vol. 27 https://doi.org/10.1016/j.cosrev.2017.10.002	journal	February 2018
Sentiment analysis: A review and comparative analysis of web services Serrano-Guerrero, Jesus; Olivas, Jose A.; Romero, Francisco P. Information Sciences, Vol. 311 https://doi.org/10.1016/j.ins.2015.03.040	journal	August 2015
A survey on sentiment analysis challenges Hussein, Doaa Mohey El-Din Mohamed Journal of King Saud University - Engineering Sciences, Vol. 30, Issue 4 https://doi.org/10.1016/j.jksues.2016.04.002	journal	October 2018
Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives Hu, Li‐tze; Bentler, Peter M. Structural Equation Modeling: A Multidisciplinary Journal, Vol. 6, Issue 1 https://doi.org/10.1080/10705519909540118	journal	January 1999
Arabic language sentiment analysis on health services Alayba, Abdulaziz M.; Palade, Vasile; England, Matthew 2017 1st International Workshop on Arabic Script Analysis and Recognition (ASAR) https://doi.org/10.1109/ASAR.2017.8067771	conference	April 2017
Prediction of Indian election using sentiment analysis on Hindi Twitter Sharma, Parul; Moh, Teng-Sheng 2016 IEEE International Conference on Big Data (Big Data) https://doi.org/10.1109/BigData.2016.7840818	conference	December 2016
Stock Price Prediction Using News Sentiment Analysis Mohan, Saloni; Mullapudi, Sahitya; Sammeta, Sudheer 2019 IEEE Fifth International Conference on Big Data Computing Service and Applications (BigDataService) https://doi.org/10.1109/BigDataService.2019.00035	conference	April 2019
State-of-the-art review on Twitter Sentiment Analysis Alshammari, Norah Fahad; AlMansour, Amal Abdullah 2019 2nd International Conference on Computer Applications & Information Security (ICCAIS) https://doi.org/10.1109/CAIS.2019.8769465	conference	May 2019
Issues of Social Data Analytics with a New Method for Sentiment Analysis of Social Media Data Wang, Zhaoxia; Joo, Victor; Tong, Chuan 2014 IEEE 6th International Conference on Cloud Computing Technology and Science https://doi.org/10.1109/CloudCom.2014.40	conference	December 2014
Sentiment extraction from natural audio streams Kaushik, Lakshmish; Sangwan, Abhijeet; Hansen, John H. L. 2013 IEEE International Conference on Acoustics, Speech and Signal Processing https://doi.org/10.1109/ICASSP.2013.6639321	conference	May 2013
Using SentiWordNet for multilingual sentiment analysis Denecke, Kerstin 2008 IEEE 24th International Conference on Data Engineering Workshop https://doi.org/10.1109/ICDEW.2008.4498370	conference	April 2008
Research on Sentiment Analysis: The First Decade Ahlgren, Oskar 2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW) https://doi.org/10.1109/ICDMW.2016.0131	conference	December 2016
Choosing your weapons: On sentiment analysis tools for software engineering research Jongeling, Robbert; Datta, Subhajit; Serebrenik, Alexander 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME) https://doi.org/10.1109/ICSM.2015.7332508	conference	September 2015
Multimodal Sentiment Analysis of Spanish Online Videos Perez Rosas, Veronica; Mihalcea, Rada; Morency, Louis-Philippe IEEE Intelligent Systems, Vol. 28, Issue 3 https://doi.org/10.1109/MIS.2013.9	journal	May 2013
Sentiment Analysis and Topic Recognition in Video Transcriptions Stappen, Lukas; Baird, Alice; Cambria, Erik IEEE Intelligent Systems, Vol. 36, Issue 2 https://doi.org/10.1109/MIS.2021.3062200	journal	March 2021
Studying the Scope of Negation for Spanish Sentiment Analysis on Twitter Jimenez Zafra, Salud Maria; Martin Valdivia, M. Teresa; Martinez Camara, Eugenio IEEE Transactions on Affective Computing, Vol. 10, Issue 1 https://doi.org/10.1109/TAFFC.2017.2693968	journal	January 2019
Comparing and combining sentiment analysis methods Gonçalves, Pollyanna; Araújo, Matheus; Benevenuto, Fabrício Proceedings of the first ACM conference on Online social networks https://doi.org/10.1145/2512938.2512951	conference	October 2013
VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text Hutto, C.; Gilbert, Eric Proceedings of the International AAAI Conference on Web and Social Media, Vol. 8, Issue 1 https://doi.org/10.1609/icwsm.v8i1.14550	journal	May 2014
SemEval-2017 Task 4: Sentiment Analysis in Twitter Rosenthal, Sara; Farra, Noura; Nakov, Preslav Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017) https://doi.org/10.18653/v1/S17-2088	conference	January 2017
The Measurement of Observer Agreement for Categorical Data Landis, J. Richard; Koch, Gary G. Biometrics, Vol. 33, Issue 1 https://doi.org/10.2307/2529310	journal	March 1977
Automated Measures of Sentiment via Transformer- and Lexicon-Based Sentiment Analysis (TLSA) Zhao, Xinyan; Wong, Chau-Wai https://doi.org/10.36227/techrxiv.21781109.v2	preprint	November 2023
Approaches, Tools and Applications for Sentiment Analysis Implementation D'Andrea, Alessia; Ferri, Fernando; Grifoni, Patrizia International Journal of Computer Applications, Vol. 125, Issue 3 https://doi.org/10.5120/ijca2015905866	journal	September 2015

Similar Records

Community Sentiment Analysis with Focus on CCS

Conference · 2024 · OSTI ID:2426314

White, Casey; Sharma, Maneesh; Bauer, Jennifer; +1 more

A spark‐based big data analysis framework for real‐time sentiment prediction on streaming data

Journal Article · 2019 · Software, Practice and Experience · OSTI ID:1529994

Kılınç, Deniz

A Survey of Intrusion Detection Systems Leveraging Host Data

Journal Article · 2019 · ACM Computing Surveys · OSTI ID:1965280

Bridges, Robert A.; Glass-Vanderlan, Tarrah R.; Iannacone, Michael D.; +2 more

Related Subjects

97 MATHEMATICS AND COMPUTING
Amazon comprehend
Combining sentiment analysis tools
Google nlp
Ibm watson
Natural language processing
Sentiment 140
Sentiment analysis
Sentiment tools
Stanford corenlp
Structural equation modeling
Text blob
Text processing
Vader

Title: A structural equation modeling approach to leveraging the power of extant sentiment analysis tools

Citation Formats

References (33)

Similar Records

Related Subjects