Comparative Study on Perceived Trust of Topic Modeling Based on Affective Level of Educational Text
Source abstract
Abstract
Latent dirichlet allocation (LDA) is a representative topic model to extract keywords related to latent topics embedded in a document set. Despite its effectiveness in finding underlying topics in documents, the traditional algorithm of LDA does not have a process to reflect sentimental meanings in text for topic extraction. Focusing on this issue, this study aims to investigate the usability of both LDA and sentiment analysis (SA) algorithms based on the affective level of text. This study defines the affective level of a given set of paragraphs and attempts to analyze the perceived trust of the methodologies in regards to usability. In our experiments, the text of the college scholastic ability test was selected as the set of evaluation paragraphs, and the affective level of the paragraphs was manipulated into three levels (low, medium, and high) as an independent variable. The LDA algorithm was used to extract the keywords of the paragraph, while SA was used to identify the positive or negative mood of the extracted subject word. In addition, the perceived trust score of the algorithm was evaluated by the subjects, and this study verifies whether there is a difference in the score according to the affective levels of the paragraphs. The results show that paragraphs with low affect lead to the high perceived trust of LDA from the participants. However, the perceived trust of SA does not show a statistically significant difference between the affect levels. The findings from this study indicate that LDA is more effective to find topics in text that mainly contains objective information.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Sentiment Analysis and Opinion Mining · Advanced Text Analysis Techniques · Topic Modeling
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Division of Design Engineering, Dong-eui University, Busan 47340, Korea
Department of Industrial and Management Engineering, Incheon National University, Incheon 22012, Korea
Funding
Incheon National University
References (41)
Show source-provided references
1. Latent dirichlet allocation
2. A Language-based Approach to Measuring Scholarly Impact
3. Probabilistic Topic Decomposition of An Eighteenth-century American Newspaper10.1002/asi.20342 ↗
4. Performance analysis of topic modeling algorithms for news articles
5. Hu, Y., John, A., and Seligmann, D.D. (2011, January 1). Event Analytics via Social Media. Proceedings of the Workshop on Social and Behavioural Networked Media Access, Scottsdale, AZ, USA.10.1145/2072627.2072638 ↗
6. AI and opinion mining
7. Mcauliffe, J.D., and Blei, D.M. (, January 8–11December). Supervised Topic Models. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada.
8. Michelson, M., and Macskassy, S.A. (2010, January 26). Discovering Users’ Topics of Interest on Twitter: A First Look. Proceedings of the Fourth Workshop on Analytics for Noisy Unstructured Text Data, Toronto, ON, Canada.10.1145/1871840.1871852 ↗
9. Hoque, E., and Carenini, G. (2016, January 7–10). MultiConVis: A Visual Text Analytics System for Exploring a Collection of Online Conversations. Proceedings of the 21st International Conference on Intelligent User Interfaces, Sonoma, CA, USA.10.1145/2856767.2856782 ↗
10. Multi-label classification using stacked hierarchical Dirichlet processes with reduced sampling complexity10.1007/s10115-018-1204-z ↗
11. Dense distributions from sparse samples: Improved Gibbs sampling parameter estimators for LDA
12. Teh, Y.W., Newman, D., and Welling, M. (2007, January 3–6). A Collapsed Variational Bayesian Inference Algorithm for Latent Dirichlet Allocation. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada.10.21236/ADA629956 ↗
13. Canini, K., Shi, L., and Griffiths, T. (2009). Online Inference of Topics with Latent Dirichlet Allocation. Artificial Intelligence and Statistics, Springer.
14. Song, Y., Pan, S., Liu, S., Zhou, M.X., and Qian, W. (2009, January 2–6). Topic and keyword re-ranking for LDA-based topic modeling. Proceedings of the 18th ACM Conference on Information and Knowledge Management, Hong Kong, China.10.1145/1645953.1646223 ↗
15. Anandkumar, A., Foster, D.P., Hsu, D.J., Kakade, S.M., and Liu, Y.K. (2012, January 3–8). A spectral algorithm for latent dirichlet allocation. Proceedings of the Advances in Neural Information Processing Systems, Stateline, NV, USA.
16. Wei, X., and Croft, W.B. (2006, January 6–11). LDA-based document models for ad-hoc retrieval. Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Seattle, WA, USA.10.1145/1148170.1148204 ↗
17. Reliability Analysis of VOC Data for Opinion Mining
18. Signals of public opinion in online communication: A comparison of methods and data sources10.1177/0002716215569192 ↗
19. Hu, M., and Liu, B. (2004, January 22–25). Mining and summarizing customer reviews. Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Seattle, WA, USA.10.1145/1014052.1014073 ↗
20. Estimating the helpfulness and economic impact of product reviews: Mining text and reviewer characteristics10.1109/TKDE.2010.188 ↗
21. Esuli, A., and Sebastiani, F. (2006, January 5–6). Determining Term Subjectivity and Term Orientation for Opinion Mining. Proceedings of the 11th Conference of the European Chapter of the Association for Computational Linguistics, Trento, Italy.
22. Mei, Q., Ling, X., Wondra, M., Su, H., and Zhai, C. (2007, January 8–12). Topic sentiment mixture: Modeling facets and opinions in weblogs. Proceedings of the 16th international conference on World Wide Web, Banff, AB, Canada.10.1145/1242572.1242596 ↗
23. New avenues in opinion mining and sentiment analysis10.1109/MIS.2013.30 ↗
24. The Corpus Based Approach to Sentiment Analysis in Modern Standard Arabic and Arabic Dialects: A Literature Review
25. Pak, A., and Paroubek, P. (2010, January 19–21). Twitter as a corpus for sentiment analysis and opinion mining. Proceedings of the Seventh Conference on International Language Resources and Evaluation, Valletta, Malta.
26. Identifying affect elements based on a conceptual model of affect: A case study on a smartphone10.1016/j.ergon.2016.01.003 ↗
27. Analyzing thumb interaction on mobile touchpad devices10.1016/j.ergon.2018.05.018 ↗
28. Developing elements of user experience for mobile phones and services: Survey, interview, and observation approaches10.1002/hfm.20316 ↗
29. Modeling user experience: A case study on a mobile device10.1016/j.ergon.2013.01.005 ↗
30. (2019, August 09). Available online: http://www.ebsi.co.kr/ebs/xip/xipc/previousPaperList.ebs.
31. A corpus-based lexical analysis of the foreign language (English) test for the college scholastic ability test (CSAT)
32. The internal analysis of the validation on item-types of Foreign (English) Language Domain of the current 2005 CSAT for designing the level-differentiated English tests of the 2014 CSAT10.18649/jkees.2013.12.2.1 ↗
33. Prediction of Correct Answer Rate and Identification of Significant Factors for CSAT English Test Based on Data Mining Techniques10.3745/KTSDE.2015.4.11.509 ↗
34. Empirical study of constructing a knowledge organization system of patent documents using topic modeling10.1007/s11192-014-1328-1 ↗
35. Building emotional dictionary for sentiment analysis of online news10.1007/s11280-013-0221-9 ↗
36. Turney, P.D. (2002, January 7–12). Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews. In Proceeding of the 40th Annual Meeting on Association for Computational Linguistics, Philadelphia, PA, USA.10.3115/1073083.1073153 ↗
37. A Korean Product Review Analysis System Using a Semi-Automatically Constructed Semantic Dictionary
38. Poria, S., Chaturvedi, I., Cambria, E., and Bisio, F. (2016, January 24–29). Sentic LDA: Improving on LDA with semantic similarity for aspect-based sentiment analysis. Proceedings of the International Joint Conference on Neural Networks (IJCNN), Vancouver, BC, Canada.10.1109/IJCNN.2016.7727784 ↗
39. Shams, M., Shakery, A., and Faili, H. (2012, January 2–3). A non-parametric LDA-based induction method for sentiment analysis. Proceedings of the 16th CSI International Symposium on Artificial Intelligence and Signal Processing (AISP), Shiraz, Iran.10.1109/AISP.2012.6313747 ↗
40. Yuan, B., and Wu, G. (2017). A Hybrid HDP-ME-LDA Model for Sentiment Analysis. 2nd International Conference on Automation, Mechanical Control and Computational Engineering (AMCCE), Atlantis Press.10.2991/amcce-17.2017.114 ↗
41. Blei, D., and Lafferty, J. (2005, January 5–8). Correlated Topic Models. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada.