A spark‐based big data analysis framework for real‐time sentiment prediction on streaming data

doi:10.1002/spe.2724

Title: A spark‐based big data analysis framework for real‐time sentiment prediction on streaming data

Authors:

^[1]

Department of Software Engineering, Faculty of TechnologyManisa Celal Bayar University Manisa Turkey

Publication Date:: 2019-06-27

Sponsoring Org.:: USDOE Office of Electricity (OE), Advanced Grid Research & Development. Power Systems Engineering Research

OSTI Identifier:: 1529994

Resource Type:: Publisher's Accepted Manuscript

Journal Name:: Software, Practice and Experience

Additional Journal Information:: Journal Name: Software, Practice and Experience Journal Volume: 49 Journal Issue: 9; Journal ID: ISSN 0038-0644

Publisher:: Wiley Blackwell (John Wiley & Sons)

Country of Publication:: United Kingdom

Language:: English

Citation Formats


                    Kılınç, Deniz. A spark‐based big data analysis framework for real‐time sentiment prediction on streaming data.  United Kingdom: N. p., 2019. 
        Web.  doi:10.1002/spe.2724.

Copy to clipboard


                    Kılınç, Deniz. A spark‐based big data analysis framework for real‐time sentiment prediction on streaming data.  United Kingdom.  doi:10.1002/spe.2724.

Copy to clipboard


                    Kılınç, Deniz. Thu .  
        "A spark‐based big data analysis framework for real‐time sentiment prediction on streaming data".  United Kingdom.  doi:10.1002/spe.2724.

Copy to clipboard


                    
@article{osti_1529994,

  title        = {A spark‐based big data analysis framework for real‐time sentiment prediction on streaming data},

  author       = {Kılınç, Deniz},

  abstractNote = {},

  doi          = {10.1002/spe.2724},

  journal      = {Software, Practice and Experience},

  number       = 9,

  volume       = 49,

  place        = {United Kingdom},

  year         = {2019},

  month        = {6}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (Publisher)

Publisher's Version of Record
DOI: 10.1002/spe.2724

Other availability

Search WorldCat to find libraries that may hold this journal

Save / Share:

Export Metadata

Save to My Library

Works referenced in this record:

Big Data Stream Analytics for Near Real-Time Sentiment Analysis
journal, January 2015

Cheng, Otto K. M.; Lau, Raymond
Journal of Computer and Communications, Vol. 03, Issue 05
DOI: 10.4236/jcc.2015.35024

Twitter fake account detection
conference, October 2017

Ersahin, Buket; Aktas, Ozlem; Kilinc, Deniz
2017 International Conference on Computer Science and Engineering (UBMK)
DOI: 10.1109/UBMK.2017.8093420

SentiWordNet for New Language: Automatic Translation Approach
conference, January 2016

Ucan, Alaettin; Naderalvojoud, Behzad; Sezer, Ebru Akcapinar
2016 12th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS)
DOI: 10.1109/SITIS.2016.57

Preparation of Improved Turkish DataSet for Sentiment Analysis in Social Media
journal, January 2017

Makinist, Semiha; Hallaç, İbrahim Rıza; Ay Karakuş, Betül
ITM Web of Conferences, Vol. 13
DOI: 10.1051/itmconf/20171301030

TTC-3600: A new benchmark dataset for Turkish text categorization
journal, December 2015

Kılınç, Deniz; Özçift, Akın; Bozyigit, Fatma
Journal of Information Science, Vol. 43, Issue 2
DOI: 10.1177/0165551515620551

Sentiment Analysis and Opinion Mining
journal, May 2012

Liu, Bing
Synthesis Lectures on Human Language Technologies, Vol. 5, Issue 1
DOI: 10.2200/S00416ED1V01Y201204HLT016

Big data analytics on Apache Spark
journal, October 2016

Salloum, Salman; Dautov, Ruslan; Chen, Xiaojun
International Journal of Data Science and Analytics, Vol. 1, Issue 3-4
DOI: 10.1007/s41060-016-0027-9

Naive Bayes classifier for continuous variables using novel method (NBC4D) and distributions
conference, June 2014

Yildirim, Pelin; Birant, Derya
2014 IEEE International Symposium on Innovations in Intelligent Systems and Applications (INISTA) Proceedings
DOI: 10.1109/INISTA.2014.6873605

Statistical Features-Based Real-Time Detection of Drifted Twitter Spam
journal, April 2017

Chen, Chao; Wang, Yu; Zhang, Jun
IEEE Transactions on Information Forensics and Security, Vol. 12, Issue 4
DOI: 10.1109/TIFS.2016.2621888

Text Mining Analysis in Turkish Language Using Big Data Tools
conference, June 2016

Cakir, Mehmet Ulas; Guldamlasioglu, Seren
2016 IEEE 40th Annual Computer Software and Applications Conference (COMPSAC)
DOI: 10.1109/COMPSAC.2016.203

Data analytic on diabetic awareness with Hadoop streaming using map reduce in python
conference, October 2016

Ramsingh, J.; Bhuvaneswari, V.
2016 IEEE International Conference on Advances in Computer Applications (ICACA)
DOI: 10.1109/ICACA.2016.7887979

Applying spark based machine learning model on streaming big data for health status prediction
journal, January 2018

Nair, Lekha R.; Shetty, Sujala D.; Shetty, Siddhanth D.
Computers & Electrical Engineering, Vol. 65
DOI: 10.1016/j.compeleceng.2017.03.009

Real-Time Sentiment Prediction on Streaming Social Network Data Using In-Memory Processing
conference, February 2017

Nirmal, V. Jude; Amalarethinam, D. I. George
2017 World Congress on Computing and Communication Technologies (WCCCT)
DOI: 10.1109/WCCCT.2016.26

A survey on platforms for big data analytics
journal, October 2014

Singh, Dilpreet; Reddy, Chandan K.
Journal of Big Data, Vol. 2, Issue 1
DOI: 10.1186/s40537-014-0008-6

Spammers Are Becoming "Smarter" on Twitter
journal, March 2016

Chen, Chao; Zhang, Jun; Xiang, Yang
IT Professional, Vol. 18, Issue 2
DOI: 10.1109/MITP.2016.36

Distributed real-time sentiment analysis for big data social streams
conference, November 2014

Rahnama, Amir Hossein Akhavan
2014 International Conference on Control, Decision and Information Technologies (CoDIT)
DOI: 10.1109/CoDIT.2014.6996998

Asymmetric self-learning for tackling Twitter Spam Drift
conference, April 2015

Chen, Chao; Zhang, Jun; Xiang, Yang
IEEE INFOCOM 2015 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), 2015 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS)
DOI: 10.1109/INFCOMW.2015.7179386

A comparison study on active learning integrated ensemble approaches in sentiment analysis
journal, January 2017

Aldoğan, Deniz; Yaslan, Yusuf
Computers & Electrical Engineering, Vol. 57
DOI: 10.1016/j.compeleceng.2016.11.015

Feature hashing for large scale multitask learning
conference, January 2009

Weinberger, Kilian; Dasgupta, Anirban; Langford, John
Proceedings of the 26th Annual International Conference on Machine Learning - ICML '09
DOI: 10.1145/1553374.1553516

Uncovering social network Sybils in the wild
journal, February 2014

Yang, Zhi; Wilson, Christo; Wang, Xiao
ACM Transactions on Knowledge Discovery from Data, Vol. 8, Issue 1
DOI: 10.1145/2556609

Detecting spammers on social networks
conference, January 2010

Stringhini, Gianluca; Kruegel, Christopher; Vigna, Giovanni
Proceedings of the 26th Annual Computer Security Applications Conference on - ACSAC '10
DOI: 10.1145/1920261.1920263

Scalable and Real-Time Sentiment Analysis of Twitter Data
conference, December 2016

Karanasou, Maria; Ampla, Anneta; Doulkeridis, Christos
2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW)
DOI: 10.1109/ICDMW.2016.0138

Clash of the titans: MapReduce vs. Spark for large scale data analytics
journal, September 2015

Shi, Juwei; Qiu, Yunjie; Minhas, Umar Farooq
Proceedings of the VLDB Endowment, Vol. 8, Issue 13
DOI: 10.14778/2831360.2831365

Studying User Footprints in Different Online Social Networks
conference, August 2012

Malhotra, A.; Totti, L.; Meira, W.
2012 International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2012), 2012 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining
DOI: 10.1109/ASONAM.2012.184

Term-weighting approaches in automatic text retrieval
journal, January 1988

Salton, Gerard; Buckley, Christopher
Information Processing & Management, Vol. 24, Issue 5
DOI: 10.1016/0306-4573(88)90021-0

FakeBook: Detecting Fake Profiles in On-Line Social Networks
conference, August 2012

Conti, M.; Poovendran, R.; Secchiero, M.
2012 International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2012), 2012 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining
DOI: 10.1109/ASONAM.2012.185

Similar Records in DOE PAGES and OSTI.GOV collections:

Comand: Context-Aware ML Middleware for Real-Time Distributed Streaming Big Data Analysis

Technical Report Xie, Bin

DOE and many applications needs big data streaming capability to process massive energy or relevant data sets using Machine Learning (ML) algorithms in a context-aware and real-time fashion. Such a capability is able to quickly process and analyze the big data in a real-time fashion to extract useful information in support of decision making. Previous approaches are very limited in such a sophisticate capability. In this project, InfoBeyond advocates the development of COMAND technology to enable context-aware and real-time ML streaming capability. Context means that the big data exhibit different statistic properties along the time and other environmental factors. Meanwhile,more »« less
A Real-Time Big Data Control-Theoretical Framework for Cyber-Physical-Human Systems [Book Chapter]

Other Gusrialdi, Azwirman ; Xu, Ying ; Qu, Zhihua ; ...

It is known that cyber-physical-human systems naturally arise from interdependent infrastructure systems and smart connected communities. Such applications require ubiquitous information sensing and processing, intelligent machine-to-machine communication for a seamless coordination, as well as intelligent interactions between humans and machines. This chapter presents a control-theoretical framework to model heterogeneous physical dynamic systems, information and communication, as well as cooperative controls and/or distributed optimization of such interconnected systems. It is shown that efficient analytical and computational algorithms can be modularly designed and hierarchically implemented to operate and optimize cyber-physical-human systems, first to quantify individually the input–output relationship of nonlinear dynamic behaviorsmore »« less
DOI: 10.1007/978-3-030-25446-9_7
Real-Time Pathogen Detection in the Era of Whole-Genome Sequencing and Big Data: Comparison of k-mer and Site-Based Methods for Inferring the Genetic Distances among Tens of Thousands of Salmonella Samples

Journal Article Pettengill, James B. ; Pightling, Arthur W. ; Baugher, Joseph D. ; ... - PLoS ONE

The adoption of whole-genome sequencing within the public health realm for molecular characterization of bacterial pathogens has been followed by an increased emphasis on real-time detection of emerging outbreaks (e.g., food-borne Salmonellosis). In turn, large databases of whole-genome sequence data are being populated. These databases currently contain tens of thousands of samples and are expected to grow to hundreds of thousands within a few years. For these databases to be of optimal use one must be able to quickly interrogate them to accurately determine the genetic distances among a set of samples. Being able to do so is challenging duemore »« less
Cited by 2
DOI: 10.1371/journal.pone.0166162

Full Text Available
Quality Control Analysis in Real-time (QC-ART): A Tool for Real-time Quality Control Assessment of Mass Spectrometry-based Proteomics Data [QC-ART: A Tool for Real-time Quality Control Assessment of Mass Spectrometry-based Proteomics Data]

Journal Article Stanfill, Bryan A. ; Nakayasu, Ernesto S. ; Bramer, Lisa M. ; ... - Molecular and Cellular Proteomics

Liquid chromatography-mass spectrometry (LC-MS)-based proteomics studies of large sample cohorts can easily require from months to years to complete. Acquiring consistent, high-quality data in such large-scale studies is challenging because of normal variations in instrumentation performance over time, as well as artifacts introduced by the samples themselves, such as those because of collection, storage and processing. Existing quality control methods for proteomics data primarily focus on post-hoc analysis to remove low-quality data that would degrade downstream statistics; they are not designed to evaluate the data in near real-time, which would allow for interventions as soon as deviations in data qualitymore »« less
Cited by 1
DOI: 10.1074/mcp.RA118.000648

Full Text Available
Spark-based Anomaly Detection Over Multi-source VMware Performance Data In Real-time.

Conference Solaimani, Mohiuddin ; Khan, Latifur ; Thuraisingham, Bhavani

Abstract not provided.
Full Text Available

Similar Records

Title: A spark‐based big data analysis framework for real‐time sentiment prediction on streaming data

Citation Formats

Big Data Stream Analytics for Near Real-Time Sentiment Analysis journal, January 2015

Twitter fake account detection conference, October 2017

SentiWordNet for New Language: Automatic Translation Approach conference, January 2016

Preparation of Improved Turkish DataSet for Sentiment Analysis in Social Media journal, January 2017

TTC-3600: A new benchmark dataset for Turkish text categorization journal, December 2015

Sentiment Analysis and Opinion Mining journal, May 2012

Big data analytics on Apache Spark journal, October 2016

Naive Bayes classifier for continuous variables using novel method (NBC4D) and distributions conference, June 2014

Statistical Features-Based Real-Time Detection of Drifted Twitter Spam journal, April 2017

Text Mining Analysis in Turkish Language Using Big Data Tools conference, June 2016

Data analytic on diabetic awareness with Hadoop streaming using map reduce in python conference, October 2016

Applying spark based machine learning model on streaming big data for health status prediction journal, January 2018

Real-Time Sentiment Prediction on Streaming Social Network Data Using In-Memory Processing conference, February 2017

A survey on platforms for big data analytics journal, October 2014

Spammers Are Becoming "Smarter" on Twitter journal, March 2016

Distributed real-time sentiment analysis for big data social streams conference, November 2014

Asymmetric self-learning for tackling Twitter Spam Drift conference, April 2015

A comparison study on active learning integrated ensemble approaches in sentiment analysis journal, January 2017

Feature hashing for large scale multitask learning conference, January 2009

Uncovering social network Sybils in the wild journal, February 2014

Detecting spammers on social networks conference, January 2010

Scalable and Real-Time Sentiment Analysis of Twitter Data conference, December 2016

Clash of the titans: MapReduce vs. Spark for large scale data analytics journal, September 2015

Studying User Footprints in Different Online Social Networks conference, August 2012

Term-weighting approaches in automatic text retrieval journal, January 1988

FakeBook: Detecting Fake Profiles in On-Line Social Networks conference, August 2012

Big Data Stream Analytics for Near Real-Time Sentiment Analysis
journal, January 2015

Twitter fake account detection
conference, October 2017

SentiWordNet for New Language: Automatic Translation Approach
conference, January 2016

Preparation of Improved Turkish DataSet for Sentiment Analysis in Social Media
journal, January 2017

TTC-3600: A new benchmark dataset for Turkish text categorization
journal, December 2015

Sentiment Analysis and Opinion Mining
journal, May 2012

Big data analytics on Apache Spark
journal, October 2016

Naive Bayes classifier for continuous variables using novel method (NBC4D) and distributions
conference, June 2014

Statistical Features-Based Real-Time Detection of Drifted Twitter Spam
journal, April 2017

Text Mining Analysis in Turkish Language Using Big Data Tools
conference, June 2016

Data analytic on diabetic awareness with Hadoop streaming using map reduce in python
conference, October 2016

Applying spark based machine learning model on streaming big data for health status prediction
journal, January 2018

Real-Time Sentiment Prediction on Streaming Social Network Data Using In-Memory Processing
conference, February 2017

A survey on platforms for big data analytics
journal, October 2014

Spammers Are Becoming "Smarter" on Twitter
journal, March 2016

Distributed real-time sentiment analysis for big data social streams
conference, November 2014

Asymmetric self-learning for tackling Twitter Spam Drift
conference, April 2015

A comparison study on active learning integrated ensemble approaches in sentiment analysis
journal, January 2017

Feature hashing for large scale multitask learning
conference, January 2009

Uncovering social network Sybils in the wild
journal, February 2014

Detecting spammers on social networks
conference, January 2010

Scalable and Real-Time Sentiment Analysis of Twitter Data
conference, December 2016

Clash of the titans: MapReduce vs. Spark for large scale data analytics
journal, September 2015

Studying User Footprints in Different Online Social Networks
conference, August 2012

Term-weighting approaches in automatic text retrieval
journal, January 1988

FakeBook: Detecting Fake Profiles in On-Line Social Networks
conference, August 2012