GBSVM: Sentiment Classification from Unstructured Reviews Using Ensemble Classifier
Source abstract
Abstract
User reviews on social networking platforms like Twitter, Facebook, and Google+, etc. have been gaining growing interest on account of their wide usage in sentiment analysis which serves as the feedback to both public and private companies, as well as, governments. The analysis of such reviews not only plays a noteworthy role to improve the quality of such services and products but helps to devise marketing and financial strategies to increase the profit for companies and customer satisfaction. Although many analysis models have been proposed, yet, there is still room for improving the processing, classification, and analysis of user reviews which can assist managers to interpret customers feedback and elevate the quality of products. This study first evaluates the performance of a few machine learning models which are among the most widely used models and then presents a voting classifier Gradient Boosted Support Vector Machine (GBSVM) which is constituted of gradient boosting and support vector machines. The proposed model has been evaluated on two different datasets with term frequency and three variants of term frequency-inverse document frequency including uni-, bi-, and tri-gram as features. The performance is compared with other state-of-the-art techniques which prove that GBSVM outperforms these models.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Sentiment Analysis and Opinion Mining · Spam and Phishing Detection · Text and Document Classification Technologies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science, Khawaja Fareed University of Engineering and Information Technology, Rahim Yar Khan 64200, Pakistan
Department of Information and Communication Engineering, Yeungnam University, Gyeongsan 38541, Korea
Department of Computer Science & Information Technology, The Islamia University of Bahawalpur, Bahawalpur 63100, Pakistan
Funding
MSIT (Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) · Yeungnam University Research Grant
References (63)
Show source-provided references
1. Chen, L., Wang, W., Nagarajan, M., Wang, S., and Sheth, A.P. (2012, January 4–7). Extracting diverse sentiment expressions with target-dependent polarity from twitter. Proceedings of the Sixth International AAAI Conference on Weblogs and Social Media, Dublin, Ireland.
2. Liu, B. (2009). Handbook Chapter: Sentiment Analysis and Subjectivity. Handbook of Natural Language Processing. Handbook of Natural Language Processing, Marcel Dekker, Inc.
3. Dave, K., Lawrence, S., and Pennock, D.M. (2003, January 20–24). Mining the peanut gallery: Opinion extraction and semantic classification of product reviews. Proceedings of the 12th International Conference on World Wide Web, Budapest, Hungary.10.1145/775152.775226 ↗
4. Kasper, W., and Vela, M. (2011, January 17–19). Sentiment analysis for hotel reviews. Proceedings of the Computational Linguistics-Applications Conference, Jachranka, Poland.
5. A survey on sentiment analysis challenges
6. Mukherjee, A., Venkataraman, V., Liu, B., and Glance, N. (2013, January 8–11). What yelp fake review filter might be doing?. Proceedings of the Seventh International AAAI Conference on Weblogs and Social Media, Cambridge, MA, USA.
7. Thumbs up? Sentiment classification using machine learning techniques
8. Opinion mining and sentiment analysis10.1561/1500000011 ↗
9. Rustam, F., Ashraf, I., Mehmood, A., Ullah, S., and Choi, G.S. (2019). Tweets Classification on the Base of Sentiments for US Airline Companies. Entropy, 21.10.3390/e21111078 ↗
10. Neethu, M., and Rajasree, R. (2013, January 4–6). Sentiment analysis in twitter using machine learning techniques. Proceedings of the 2013 Fourth International Conference on Computing, Communications and Networking Technologies (ICCCNT), Tiruchengode, India.10.1109/ICCCNT.2013.6726818 ↗
11. Sentiment analysis in Facebook and its application to e-learning10.1016/j.chb.2013.05.024 ↗
12. Bakshi, R.K., Kaur, N., Kaur, R., and Kaur, G. (2016, January 16–18). Opinion mining and sentiment analysis. Proceedings of the 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom), New Delhi, India.
13. Barnaghi, P., Ghaffari, P., and Breslin, J.G. (April, January 29). Opinion mining and sentiment polarity on twitter and correlation between events and sentiment. Proceedings of the 2016 IEEE Second International Conference on Big Data Computing Service and Applications (BigDataService), Oxford, UK.10.1109/BigDataService.2016.36 ↗
14. Ahmed, S., and Danti, A. (2015, January 21–22). A novel approach for Sentimental Analysis and Opinion Mining based on SentiWordNet using web data. Proceedings of the 2015 International Conference on Trends in Automation, Communications and Computing Technology (I-TACT-15), Bangalore, India.10.1109/ITACT.2015.7492646 ↗
15. Duwairi, R.M., and Qarqaz, I. (2014, January 27–29). Arabic sentiment analysis using supervised classification. Proceedings of the 2014 International Conference on Future Internet of Things and Cloud, Barcelona, Spain.10.1109/FiCloud.2014.100 ↗
16. Le, H.S., Van Le, T., and Pham, T.V. (2015, January 23–25). Aspect analysis for opinion mining of Vietnamese text. Proceedings of the 2015 International Conference on Advanced Computing and Applications (ACOMP), Ho Chi Minh City, Vietnam.10.1109/ACOMP.2015.21 ↗
17. Chumwatana, T. (2015, January 11–13). Using sentiment analysis technique for analyzing Thai customer satisfaction from social media. Proceedings of the 5th International Conference on Computing and Informatics (ICOCI), Istanbul, Turkey.
18. A machine learning approach to sentiment analysis in multilingual Web texts10.1007/s10791-008-9070-z ↗
19. A study of the effects of preprocessing strategies on sentiment analysis for Arabic text10.1177/0165551514534143 ↗
20. The impact of preprocessing on text classification10.1016/j.ipm.2013.08.006 ↗
21. Kalra, V., and Aggarwal, R. (2017, January 22–23). Importance of Text Data Preprocessing & Implementation in RapidMiner. Proceedings of the First International Conference on Information Technology and Knowledge Management, New Dehli, India.10.15439/2017KM46 ↗
22. A novel probabilistic feature selection method for text classification10.1016/j.knosys.2012.06.005 ↗
23. Hackeling, G. (2017). Mastering Machine Learning with Scikit-Learn, Packt Publishing Ltd.
24. Sentiment classification: The contribution of ensemble learning10.1016/j.dss.2013.08.002 ↗
25. Whitehead, M., and Yaeger, L. (2010). Sentiment mining using ensemble classification models. Innovations and Advances in Computer Sciences and Engineering, Springer.10.1007/978-90-481-3658-2_89 ↗
26. Ensemble Learning
27. Deng, X.B., Ye, Y.M., Li, H.B., and Huang, J.Z. (2008, January 12–15). An improved random forest approach for detection of hidden web search interfaces. Proceedings of the 2008 International Conference on Machine Learning and Cybernetics, Kunming, China.
28. Random Forests10.1023/A:1010933404324 ↗
29. The importance of logistic regression implementations in the Turkish livestock sector and logistic regression implementations/fields
30. Learning nonlinear functions using regularized greedy forest10.1109/TPAMI.2013.159 ↗
31. Greedy function approximation: A gradient boosting machine10.1214/aos/1013203451 ↗
32. Gradient boosting machines, a tutorial10.3389/fnbot.2013.00021 ↗
33. Bissacco, A., Yang, M.H., and Soatto, S. (2007, January 17–22). Fast human pose estimation using appearance and motion via multi-dimensional boosting regression. Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA.10.1109/CVPR.2007.383129 ↗
34. Hutchinson, R.A., Liu, L.P., and Dietterich, T.G. (2011, January 7–11). Incorporating boosted regression trees into ecological latent variable models. Proceedings of the Twenty-Fifth Aaai Conference on Artificial Intelligence, San Francisco, CA, USA.10.1609/aaai.v25i1.7801 ↗
35. Support-vector networks10.1007/BF00994018 ↗
36. Data mining algorithms and their applications in education data mining
37. A survey on pattern recognition applications of support vector machines10.1142/S0218001403002460 ↗
38. Burges, C.J. (1999). Geometry and Invariance in Kernel Based Methods, Advances in Kernel Methods: Support Vector Learning, MIT Press.10.7551/mitpress/1130.003.0010 ↗
39. Shmilovici, A. (2009). Support vector machines. Data Mining and Knowledge Discovery Handbook, Springer.10.1007/978-0-387-09823-4_12 ↗
40. Smola, A., and Schölkopf, B. (1998). A Tutorial on Support Vector Regression, Royal Holloway Coll. Univ.. Available online: http://www.kernel-machines.
41. The connection between regularization operators and support vector kernels10.1016/S0893-6080(98)00032-X ↗
42. Non-parametric estimation of a multivariate probability density10.1137/1114019 ↗
43. Zhang, Y. (2012). Support vector machine classification algorithm and its application. International Conference on Information Computing and Applications, Springer.10.1007/978-3-642-34041-3_27 ↗
44. Improvements to the SMO algorithm for SVM regression10.1109/72.870050 ↗
45. Text mining: Open source tokenization tools-an analysis
46. Text mining of Twitter data using a latent Dirichlet allocation topic model and sentiment analysis
47. Anandarajan, M., and Nolan, T. (2019). Practical Text Analytics. Maximizing the Value of Text Data. Advances in Analytics and Data Science, Springer Nature Switzerland AG.10.1007/978-3-319-95663-3 ↗
48. Support vector machines: Hype or hallelujah?10.1145/380995.380999 ↗
49. Soft voting technique to improve the performance of global filter based feature selection in text corpus10.1007/s10489-018-1349-1 ↗
50. Kaggle (2019, November 21). Google Play Store Apps. Available online: https://www.kaggle.com/lava18/google-play-store-apps.
51. Tellex, S., Katz, B., Lin, J., Fernandes, A., and Marton, G. (August, January 28). Quantitative evaluation of passage retrieval algorithms for question answering. Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Informaion Retrieval, Toronto, ON, Canada.
52. Fuzzy bag-of-words model for document representation10.1109/TFUZZ.2017.2690222 ↗
53. Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv.
54. Huang, L. (2019, November 21). Measuring Similarity Between Texts in Python. Available online: https://sites.temple.edu/tudsc/2017/03/30/measuring-similarity-between-texts-in-python/.
55. Joulin, A., Grave, E., Bojanowski, P., and Mikolov, T. (2016). Bag of tricks for efficient text classification. arXiv.10.18653/v1/E17-2068 ↗
56. Sisodia, D.S., Nikhil, S., Kiran, G.S., and Shrawgi, H. (2020). Performance Evaluation of Learners for Analyzing the Hotel Customer Sentiments Based on Text Reviews. Performance Management of Integrated Systems and its Applications in Software Engineering, Springer.10.1007/978-981-13-8253-6_20 ↗
57. Performance evaluation of the data mining classification methods
58. Han, J., Pei, J., and Kamber, M. (2011). Data Mining: Concepts and Techniques, Elsevier.
59. Shalev-Shwartz, S., and Ben-David, S. (2014). Understanding Machine Learning: From Theory to Algorithms, Cambridge University Press.10.1017/CBO9781107298019 ↗
60. Danesh, A., Moshiri, B., and Fatemi, O. (2007, January 9–12). Improve text classification accuracy based on classifier fusion methods. Proceedings of the 2007 10th International Conference on Information Fusion, Quebec, QC, Canada.10.1109/ICIF.2007.4408196 ↗
61. Kaggle (2020, January 14). 20 Newsgroups. Available online: https://www.kaggle.com/crawford/20-newsgroups.
62. Wu, F., Zhang, T., Souza, A.H.D., Fifty, C., Yu, T., and Weinberger, K.Q. (2019). Simplifying graph convolutional networks. arXiv.
63. Yamada, I., and Shindo, H. (2019). Neural Attentive Bag-of-Entities Model for Text Classification. arXiv.10.18653/v1/K19-1052 ↗