A spark‐based big data analysis framework for real‐time sentiment prediction on streaming data

doi:10.1002/spe.2724

This content will become publicly available on June 26, 2020

Title: A spark‐based big data analysis framework for real‐time sentiment prediction on streaming data

Authors:

^[1]

Department of Software Engineering, Faculty of TechnologyManisa Celal Bayar University Manisa Turkey

Publication Date:: 2019-06-27

Sponsoring Org.:: USDOE Office of Electricity Delivery and Energy Reliability (OE), Power Systems Engineering Research and Development (R&D) (OE-10)

OSTI Identifier:: 1529994

Resource Type:: Publisher's Accepted Manuscript

Journal Name:: Software, Practice and Experience

Additional Journal Information:: Journal Name: Software, Practice and Experience Journal Volume: 49 Journal Issue: 9; Journal ID: ISSN 0038-0644

Publisher:: Wiley Blackwell (John Wiley & Sons)

Country of Publication:: United Kingdom

Language:: English

Citation Formats


                    Kılınç, Deniz. A spark‐based big data analysis framework for real‐time sentiment prediction on streaming data.  United Kingdom: N. p., 2019. 
        Web.  doi:10.1002/spe.2724.

Copy to clipboard


                    Kılınç, Deniz. A spark‐based big data analysis framework for real‐time sentiment prediction on streaming data.  United Kingdom.  doi:10.1002/spe.2724.

Copy to clipboard


                    Kılınç, Deniz. Thu .  
        "A spark‐based big data analysis framework for real‐time sentiment prediction on streaming data".  United Kingdom.  doi:10.1002/spe.2724.

Copy to clipboard


                    
@article{osti_1529994,

  title        = {A spark‐based big data analysis framework for real‐time sentiment prediction on streaming data},

  author       = {Kılınç, Deniz},

  abstractNote = {},

  doi          = {10.1002/spe.2724},

  journal      = {Software, Practice and Experience},

  number       = 9,

  volume       = 49,

  place        = {United Kingdom},

  year         = {2019},

  month        = {6}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

This content will become publicly available on June 26, 2020

Publisher's Version of Record

DOI: 10.1002/spe.2724

Other availability

Search WorldCat to find libraries that may hold this journal

Save / Share:

Export Metadata

Save to My Library

Works referenced in this record:

Big Data Stream Analytics for Near Real-Time Sentiment Analysis
journal, January 2015

Cheng, Otto K. M.; Lau, Raymond
Journal of Computer and Communications, Vol. 03, Issue 05
DOI: 10.4236/jcc.2015.35024

Preparation of Improved Turkish DataSet for Sentiment Analysis in Social Media
journal, January 2017

Makinist, Semiha; Hallaç, İbrahim Rıza; Ay Karakuş, Betül
ITM Web of Conferences, Vol. 13
DOI: 10.1051/itmconf/20171301030

TTC-3600: A new benchmark dataset for Turkish text categorization
journal, December 2015

Kılınç, Deniz; Özçift, Akın; Bozyigit, Fatma
Journal of Information Science, Vol. 43, Issue 2
DOI: 10.1177/0165551515620551

Sentiment Analysis and Opinion Mining
journal, May 2012

Liu, Bing
Synthesis Lectures on Human Language Technologies, Vol. 5, Issue 1
DOI: 10.2200/S00416ED1V01Y201204HLT016

Big data analytics on Apache Spark
journal, October 2016

Salloum, Salman; Dautov, Ruslan; Chen, Xiaojun
International Journal of Data Science and Analytics, Vol. 1, Issue 3-4
DOI: 10.1007/s41060-016-0027-9

Statistical Features-Based Real-Time Detection of Drifted Twitter Spam
journal, April 2017

Chen, Chao; Wang, Yu; Zhang, Jun
IEEE Transactions on Information Forensics and Security, Vol. 12, Issue 4
DOI: 10.1109/TIFS.2016.2621888

Applying spark based machine learning model on streaming big data for health status prediction
journal, January 2018

Nair, Lekha R.; Shetty, Sujala D.; Shetty, Siddhanth D.
Computers & Electrical Engineering, Vol. 65
DOI: 10.1016/j.compeleceng.2017.03.009

A survey on platforms for big data analytics
journal, October 2014

Singh, Dilpreet; Reddy, Chandan K.
Journal of Big Data, Vol. 2, Issue 1
DOI: 10.1186/s40537-014-0008-6

Spammers Are Becoming "Smarter" on Twitter
journal, March 2016

Chen, Chao; Zhang, Jun; Xiang, Yang
IT Professional, Vol. 18, Issue 2
DOI: 10.1109/MITP.2016.36

A comparison study on active learning integrated ensemble approaches in sentiment analysis
journal, January 2017

Aldoğan, Deniz; Yaslan, Yusuf
Computers & Electrical Engineering, Vol. 57
DOI: 10.1016/j.compeleceng.2016.11.015

Uncovering social network Sybils in the wild
journal, February 2014

Yang, Zhi; Wilson, Christo; Wang, Xiao
ACM Transactions on Knowledge Discovery from Data, Vol. 8, Issue 1
DOI: 10.1145/2556609

Clash of the titans: MapReduce vs. Spark for large scale data analytics
journal, September 2015

Shi, Juwei; Qiu, Yunjie; Minhas, Umar Farooq
Proceedings of the VLDB Endowment, Vol. 8, Issue 13
DOI: 10.14778/2831360.2831365

Term-weighting approaches in automatic text retrieval
journal, January 1988

Salton, Gerard; Buckley, Christopher
Information Processing & Management, Vol. 24, Issue 5
DOI: 10.1016/0306-4573(88)90021-0

Similar records in OSTI.GOV collections:

Matisse: A Visual Analytics System for Exploring Emotion Trends in Social Media Text Streams

Conference Steed, Chad A ; Drouhard, Margaret MEG G ; Beaver, Justin M ; ...

Dynamically mining textual information streams to gain real-time situational awareness is especially challenging with social media systems where throughput and velocity properties push the limits of a static analytical approach. In this paper, we describe an interactive visual analytics system, called Matisse, that aids with the discovery and investigation of trends in streaming text. Matisse addresses the challenges inherent to text stream mining through the following technical contributions: (1) robust stream data management, (2) automated sentiment/emotion analytics, (3) interactive coordinated visualizations, and (4) a flexible drill-down interaction scheme that accesses multiple levels of detail. In addition to positive/negative sentiment prediction,more »« less
Full Text Available
Accelerating Data Acquisition, Reduction, and Analysis at the Spallation Neutron Source

Conference Campbell, Stuart I ; Kohl, James Arthur ; Granroth, Garrett E ; ...

ORNL operates the world's brightest neutron source, the Spallation Neutron Source (SNS). Funded by the US DOE Office of Basic Energy Science, this national user facility hosts hundreds of scientists from around the world, providing a platform to enable break-through research in materials science, sustainable energy, and basic science. While the SNS provides scientists with advanced experimental instruments, the deluge of data generated from these instruments represents both a big data challenge and a big data opportunity. For example, instruments at the SNS can now generate multiple millions of neutron events per second providing unprecedented experiment fidelity but leaving themore »« less
Cloud Based Analytical Framework for Synchrophasor Data Analysis

Conference Etingov, Pavel V. ; Hou, Zhangshuan ; Wang, Heng ; ...

The paper presents initial results from the development of a cloud-based, Big Data analysis platform for power systems. The computational pipeline uses the Apache Spark framework running in an OpenStack cloud infrastructure. A real-world phasor measurement unit (PMU) dataset has been used to carry out the analysis. Actual examples of power system events detection using synchrophasor data are presented. It has been shown that applications of the cloud based computing environment and the Apache Spark framework enable a significant increase in the computational efficiency of large-scale PMU data analysis.
Statistical Techniques For Real-time Anomaly Detection Using Spark Over Multi-source VMware Performance Data

Journal Article Solaimani, Mohiuddin ; Iftekhar, Mohammed ; Khan, Latifur ; ... - Sandia journal manuscript; Not yet accepted for publication

Anomaly detection refers to the identi cation of an irregular or unusual pat- tern which deviates from what is standard, normal, or expected. Such deviated patterns typically correspond to samples of interest and are assigned different labels in different domains, such as outliers, anomalies, exceptions, or malware. Detecting anomalies in fast, voluminous streams of data is a formidable chal- lenge. This paper presents a novel, generic, real-time distributed anomaly detection framework for heterogeneous streaming data where anomalies appear as a group. We have developed a distributed statistical approach to build a model and later use it to detect anomaly. Asmore »« less
Full Text Available
Uncovering the relationships between military community health and affects expressed in social media

Journal Article Volkova, Svitlana ; Charles, Lauren E. ; Harrison, Josh ; ... - EPJ Data Science

Military populations present a small, unique community whose mental and physical health impacts the security of the nation. Recent literature has explored social media's ability to enhance disease surveillance and characterize distinct communities with encouraging results. We present a novel analysis of the relationships between influenza-like illnesses (ILI) clinical data and affects (i.e., emotions and sentiments) extracted from social media around military facilities. Our analyses examine (1) differences in affects expressed by military and control populations, (2) affect changes over time by users, (3) differences in affects expressed during high and low ILI seasons, and (4) correlations and cross-correlations betweenmore »« less
Cited by 2
DOI: 10.1140/epjds/s13688-017-0102-z

Full Text Available

Similar Records

This content will become publicly available on June 26, 2020

Title: A spark‐based big data analysis framework for real‐time sentiment prediction on streaming data

Citation Formats

Big Data Stream Analytics for Near Real-Time Sentiment Analysis journal, January 2015

Preparation of Improved Turkish DataSet for Sentiment Analysis in Social Media journal, January 2017

TTC-3600: A new benchmark dataset for Turkish text categorization journal, December 2015

Sentiment Analysis and Opinion Mining journal, May 2012

Big data analytics on Apache Spark journal, October 2016

Statistical Features-Based Real-Time Detection of Drifted Twitter Spam journal, April 2017

Applying spark based machine learning model on streaming big data for health status prediction journal, January 2018

A survey on platforms for big data analytics journal, October 2014

Spammers Are Becoming "Smarter" on Twitter journal, March 2016

A comparison study on active learning integrated ensemble approaches in sentiment analysis journal, January 2017

Uncovering social network Sybils in the wild journal, February 2014

Clash of the titans: MapReduce vs. Spark for large scale data analytics journal, September 2015

Term-weighting approaches in automatic text retrieval journal, January 1988

Big Data Stream Analytics for Near Real-Time Sentiment Analysis
journal, January 2015

Preparation of Improved Turkish DataSet for Sentiment Analysis in Social Media
journal, January 2017

TTC-3600: A new benchmark dataset for Turkish text categorization
journal, December 2015

Sentiment Analysis and Opinion Mining
journal, May 2012

Big data analytics on Apache Spark
journal, October 2016

Statistical Features-Based Real-Time Detection of Drifted Twitter Spam
journal, April 2017

Applying spark based machine learning model on streaming big data for health status prediction
journal, January 2018

A survey on platforms for big data analytics
journal, October 2014

Spammers Are Becoming "Smarter" on Twitter
journal, March 2016

A comparison study on active learning integrated ensemble approaches in sentiment analysis
journal, January 2017

Uncovering social network Sybils in the wild
journal, February 2014

Clash of the titans: MapReduce vs. Spark for large scale data analytics
journal, September 2015

Term-weighting approaches in automatic text retrieval
journal, January 1988