journal article·2025-11-25Open access

Irony and Sarcasm Detection in Turkish Texts: A Comparative Study of Transformer-Based Models and Ensemble Learning

Murat Eser, Metin Bilgin

Applied Sciences· Volume 15· Issue 23

Abstract

Irony and sarcasm are forms of expression that emphasize the inconsistency between what is said and what is meant. Correctly classifying such expressions is an important text mining problem, especially on user-centered platforms such as social media. Due to the increasing prevalence of implicit expressions, this topic has become a significant area of research in Natural Language Processing (NLP). However, the simultaneous detection of ironic and sarcastic expressions is highly challenging, as both types of implicit sentiments often convey closely related meanings. To address the detection of irony and sarcasm, this study compares the performance of transformer-based models and an ensemble learning method on Turkish texts, using five textual datasets—monogram, bigram, trigram, quadrigram, and omnigram—that share the same textual content but differ in context length. To improve classification performance, an ensemble learning approach based on the Artificial Rabbit Optimization (ARO) algorithm was implemented, combining the outputs of the models to produce final predictions. The experimental results indicate that as the context width of the datasets increases, the models achieve better predictions, leading to improvements across all performance metrics. The ensemble learning method outperformed individual models in all metrics, with performance increasing as the context expanded, achieving the highest success in the omnigram dataset with 76.71% accuracy, 74.64% precision, 73.29% sensitivity, and 73.96% F-Score. This study demonstrates that both model architecture and data structure are decisive factors in text classification performance, showing that community methods can make significant contributions to the effectiveness of deep learning solutions in low-resource languages.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Sentiment Analysis and Opinion Mining · Topic Modeling · Language, Metaphor, and Cognition

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Computer Engineering, Bursa Uludag University, Bursa 16059, Turkey

Funding

Bursa Uludağ University Science and Technology Centre

References (100+)

Show source-provided references
1. A comparative analysis of transformer based models for figurative language classification10.1016/j.compeleceng.2022.108051 ↗
2. Sarcastic sentiment detection in tweets streamed in real time: A big data approach10.1016/j.dcan.2016.06.002 ↗
3. From humor recognition to irony detection: The figurative language of social media10.1016/j.datak.2012.02.005 ↗
4. Turkish abstractive text summarization using pretrained sequence-to-sequence models10.1017/S1351324922000195 ↗
5. Sarcasm classification: A novel approach by using Content Based Feature Selection Method10.1016/j.procs.2018.10.409 ↗
6. Making objective decisions from subjective data: Detecting irony in customer reviews10.1016/j.dss.2012.05.027 ↗
7. Sentiment analysis: Bayesian Ensemble Learning10.1016/j.dss.2014.10.004 ↗
8. Sentiment classification: The contribution of ensemble learning10.1016/j.dss.2013.08.002 ↗
9. Poria, S., Cambria, E., Hazarika, D., and Vij, P. (2016). A deeper look into sarcastic tweets using deep convolutional neural networks. COLING 2016—26th International Conference on Computational Linguistics, Proceedings of COLING 2016: Technical Papers, Osaka, Japan, 11–16 December 2016, The COLING 2016 Organizing Committee.
10. Figurative messages and affect in Twitter: Differences between #irony, #sarcasm and #not10.1016/j.knosys.2016.05.035 ↗
11. Filatova, E. (2012, January 23–25). Irony and sarcasm: Corpus generation and analysis using crowdsourcing. Proceedings of the 8th International Conference on Language Resources and Evaluation, LREC 2012, Istanbul, Turkey.
12. Ravi, K., and Ravi, V. (2018, January 16–18). Irony Detection Using Neural Network Language Model, Psycholinguistic Features and Text Mining. Proceedings of the 2018 IEEE 17th International Conference on Cognitive Informatics and Cognitive Computing, ICCI*CC 2018, Berkeley, CA, USA.10.1109/ICCI-CC.2018.8482094 ↗
13. A novel automatic satire and irony detection using ensembled feature selection and data mining10.1016/j.knosys.2016.12.018 ↗
14. Sarcasm detection in natural language processing
15. Lad, R. (2023, November 12). Sarcasm Detection in English and Arabic Tweets Using Transformer Models. Available online: https://digitalcommons.dartmouth.edu/cs_senior_theses/20/.
16. An emoji-aware multitask framework for multimodal sarcasm detection10.1016/j.knosys.2022.109924 ↗
17. Irony as relevant inappropriateness10.1016/S0378-2166(99)00070-3 ↗
18. Sarcasm detection using deep learning and ensemble learning10.1007/s11042-022-12930-z ↗
19. Kader, F.B., Nujat, N.H., Sogir, T.B., Kabir, M., Mahmud, H., and Hasan, K. (2022). Computational Sarcasm Analysis on Social Media: A Systematic Review. arXiv, Available online: https://arxiv.org/abs/2209.06170.
20. Taşlıoğlu, H. (2014). Ironi Detection on Turkish Microblog Texts, Middle East Technical University.
21. KnowleNet: Knowledge fusion network for multimodal sarcasm detection10.1016/j.inffus.2023.101921 ↗
22. Rohanian, O., Taslimipoor, S., Evans, R., and Mitkov, R. (2018). WLV at SemEval-2018 Task 3: Dissecting Tweets in Search of Irony. NAACL HLT 2018—International Workshop on Semantic Evaluation, SemEval 2018—Proceedings of the 12th Workshop, New Orleans, LA, USA, 5–6 June 2018, Association for Computational Linguistics.10.18653/v1/S18-1090 ↗
23. Sarcasm Detection with Sentiment Semantics Enhanced Multi-level Memory Network10.1016/j.neucom.2020.03.081 ↗
24. Chaudhari, P., and Chandankhede, C. (2017, January 22–24). Literature survey of sarcasm detection. Proceedings of the 2017 International Conference on Wireless Communications, Signal Processing and Networking, WiSPNET 2017, Chennai, India.10.1109/WiSPNET.2017.8300120 ↗
25. The Differential Role of Ridicule in Sarcasm and Irony10.1207/s15327868ms1301_1 ↗
26. Irony in conversation10.1017/S0047404599004029 ↗
27. De Freitas, L.A., Vanin, A.A., Hogetop, D.N., Bochernitsan, M.N., and Vieira, R. (2014, January 24–28). Pathways for irony detection in tweets. Proceedings of the ACM Symposium on Applied Computing, Gyeongju, Republic of Korea.10.1145/2554850.2555048 ↗
28. A survey of figurative language and its computational detection in online social networks10.1145/3375547 ↗
29. SARCASM: MOCK POLITENESS in THE BIG BANG THEORY10.24252/10.24252/elite.v7i2a7 ↗
30. Sarcasm Detection with A New CNN + BiLSTM Hybrid Neural Network and BERT Classification Model
31. Nagwanshi, P., and Madhavan, C.E.V. (2014, January 21–24). Sarcasm detection using sentiment and semantic features. Proceedings of the International Conference on Knowledge Discovery and Information Retrieval—KDIR 2014, Rome, Italy.10.5220/0005153504180424 ↗
32. Oraby, S., Harrison, V., Misra, A., Riloff, E., and Walker, M. (2017). Are you serious? Rhetorical questions and sarcasm in social media dialog. SIGDIAL 2017—18th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Proceedings of the Conference, Saarbrücken, Germany, 15–17 August 2017, Association for Computational Linguistics.10.18653/v1/W17-5537 ↗
33. Savini, E., and Caragea, C. (2022). Intermediate-Task Transfer Learning with BERT for Sarcasm Detection. Mathematics, 10.10.3390/math10050844 ↗
34. Frenda, S., Patti, V., and Rosso, P. (2023). When Sarcasm Hurts: Irony-Aware Models for Abusive Language Detection, Springer.10.1007/978-3-031-42448-9_4 ↗
35. Computational Linguistics with Optimal Deep Belief Network Based Irony Detection in Social Media
36. Maladry, A., Lefever, E., Van Hee, C., and Hoste, V. (2023, January 14). A Fine Line Between Irony and Sincerity: Identifying Bias in Transformer Models for Irony Detection. Proceedings of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, Toronto, ON, Canada.10.18653/v1/2023.wassa-1.28 ↗
37. Malik, M., Tomás, D., and Rosso, P. (2023). How Challenging is Multimodal Irony Detection?. Language Processing and Information Systems, Springer. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinfor-matics).10.1007/978-3-031-35320-8_2 ↗
38. Ortega-Bueno, R., Rosso, P., and Fersini, E. (2023). Cross-Domain and Cross-Language Irony Detection: The Impact of Bias on Models’ Generalization. Natural Language Processing and Information Systems, Springer. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinfor-matics).10.1007/978-3-031-35320-8_10 ↗
39. Ensemble-based domain adaptation on social media posts for irony detection10.1007/s11042-023-16180-5 ↗
40. Exploiting stacked embeddings with LSTM for multilingual humor and irony detection10.1007/s13278-023-01049-0 ↗
41. Transformer-based models for multimodal irony detection10.1007/s12652-022-04447-y ↗
42. ‘Irony is easy to understand’: The role of emoji in irony detection10.1515/ip-2023-5001 ↗
43. da Rocha Junqueira, J., Junior, C.L., Silva, F.L.V., Côrrea, U.B., and de Freitas, L.A. (2023, January 25–29). Albertina in action: An investigation of its abilities in aspect extraction, hate speech detection, irony detection, and question-answering. Proceedings of the Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana (STIL), SBC, Belo Horizonte, Brazil.10.5753/stil.2023.234159 ↗
44. Băroiu, A.C., and Trăușan-Matu, Ș. (2022). Automatic Sarcasm Detection: Systematic Literature Review. Information, 13.10.3390/info13080399 ↗
45. A machine learning approach in analysing the effect of hyperboles using negative sentiment tweets for sarcasm detection10.1016/j.jksuci.2022.01.008 ↗
46. Misra, R. (2022). News Headlines Dataset for Sarcasm Detection. arXiv, Available online: http://arxiv.org/abs/2212.06035.
47. BERT-LSTM model for sarcasm detection in code-mixed social media post10.1007/s10844-022-00755-z ↗
48. Qiao, Y., Jing, L., Song, X., Chen, X., Zhu, L., and Nie, L. (2023, January 7–14). Mutual-Enhanced Incongruity Learning Network for Multi-Modal Sarcasm Detection. Proceedings of the 37th AAAI Conference on Artificial Intelligence, AAAI 2023, Washington, DC, USA.10.1609/aaai.v37i8.26138 ↗
49. Sharma, D.K., Singh, B., Agarwal, S., Kim, H., and Sharma, R. (2022). Sarcasm Detection over Social Media Platforms Using Hybrid Auto-Encoder-Based Model. Electronics, 11.10.3390/electronics11182844 ↗
50. An intelligent machine learning-based sarcasm detection and classification model on social networks10.1007/s11227-022-04312-x ↗
51. González-Ibáñez, R., Muresan, S., and Wacholder, N. (2011, January 19–24). Identifying sarcasm in Twitter: A closer look. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies—ACL-HLT 2011, Portland, OR, USA.
52. Extracting relevant knowledge for the detection of sarcasm and nastiness in the social web10.1016/j.knosys.2014.05.021 ↗
53. Automatic detection of satire in Twitter: A psycholinguistic-based approach10.1016/j.knosys.2017.04.009 ↗
54. Al-Ghadhban, D., Alnkhilan, E., Tatwany, L., and Alrazgan, M. (2017, January 8–10). Arabic sarcasm detection in Twitter. Proceedings of the 2017 International Conference on Engineering and MIS, ICEMIS 2017, Monastir, Tunisia.10.1109/ICEMIS.2017.8272990 ↗
55. da Rocha Junqueira, J., Da Silva, F., Costa, W., Carvalho, R., Bender, A., Correa, U., and Freitas, L. (2023, January 14–17). BERTimbau in Action: An Investigation of its Abilities in Sentiment Analysis, Aspect Extraction, Hate Speech Detection, and Irony Detection. Proceedings of the International Florida Artificial Intelligence Research Society Conference, FLAIRS, Clearwater Beach, FL, USA.10.32473/flairs.36.133186 ↗
56. Ramteke, J., Shah, S., Godhia, D., and Shaikh, A. (2016, January 26–27). Election result prediction using Twitter sentiment analysis. Proceedings of the International Conference on Inventive Computation Technologies, ICICT 2016, Coimbatore, India.10.1109/INVENTIVE.2016.7823280 ↗
57. A Pattern-Based Approach for Sarcasm Detection on Twitter10.1109/ACCESS.2016.2594194 ↗
58. Forslid, E., and Wikén, N. (2015). Automatic Irony- and Sarcasm Detection in Social Media, Uppsala Universitet.
59. Nayak, D.K., and Bolla, B.K. (2022). Efficient Deep Learning Methods for Sarcasm Detection of News Headlines. Machine Learning and Autonomous Systems, Springer. Smart Innovation, Systems and Technologies.10.1007/978-981-16-7996-4_26 ↗
60. Shrikhande, P., Setty, V., and Sahani, A. (2020, January 26–28). Sarcasm Detection in Newspaper Headlines. Proceedings of the 2020 IEEE 15th International Conference on Industrial and Information Systems, ICIIS 2020—Proceedings, Rupnagar, India.10.1109/ICIIS51140.2020.9342742 ↗
61. Zanchak, M., Vysotska, V., and Albota, S. (2021, January 22–25). The sarcasm detection in news headlines based on machine learning technology. Proceedings of the International Scientific and Technical Conference on Computer Sciences and Information Technologies, Lviv, Ukraine.10.1109/CSIT52700.2021.9648710 ↗
62. Irony and Sarcasm Detection on Public Figure Speech
63. Sentiment analysis of tweets using refined neutrosophic sets10.1016/j.compind.2019.103180 ↗
64. ABCDM: An Attention-based Bidirectional CNN-RNN Deep Model for sentiment analysis10.1016/j.future.2020.08.005 ↗
65. Energy choices in Alaska: Mining people’s perception and attitudes from geotagged tweets10.1016/j.rser.2020.109781 ↗
66. AraCust: A Saudi Telecom Tweets corpus for sentiment analysis10.7717/peerj-cs.510 ↗
67. The paradox of sarcasm: Theory of mind and sarcasm use in adults10.1016/j.paid.2020.110035 ↗
68. A Comprehensive Study on Sarcasm Detection Techniques in Sentiment Analysis
69. TÜRKÇE DİLİNDE YAZILAN BİLİMSEL METİNLERİN DERİN ÖĞRENME TEKNİĞİ UYGULANARAK ÇOKLU SINIFLANDIRILMASI10.21923/jesd.973181 ↗
70. Dimovska, J., Angelovska, M., Gjorgjevikj, D., and Madjarov, G. (2018). Sarcasm and Irony Detection in English Tweets. ICT Innovations 2018. Engineering and Life Science, Springer. Communications in Computer and Information Science.10.1007/978-3-030-00825-3_11 ↗
71. Samonte, M.J.C., Dollete, C.J.T., Capanas, P.M.M., Flores, M.L.C., and Soriano, C.B. (2018, January 24–26). Sentence-level sarcasm detection in English and Filipino tweets. Proceedings of the 4th International Conference on Industrial and Business Engineering, Macao.10.1145/3288155.3288172 ↗
72. O2-Bert: Two-Stage Target-Based Sentiment Analysis10.1007/s12559-023-10191-y ↗
73. Identifying sarcasm using heterogeneous word embeddings: A hybrid and ensemble perspective10.1007/s00500-023-08368-6 ↗
74. Quintero, Y.C., and García, L.A. (2018). Irony detection based on character language model classifiers. Progress in Artificial Intelligence and Pattern Recognition, Springer. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics).
75. Bölücü, N., and Can, B. (2020, January 5–7). Sarcasm target identification with lstm networks. Proceedings of the 2020 28th Signal Processing and Communications Applications Conference (SIU), Türkiye, Gaziantep.10.1109/SIU49456.2020.9302321 ↗
76. Mohammed, P., Eid, Y., Badawy, M., and Hassan, A. (2019, January 26–28). Evaluation of Different Sarcasm Detection Models for Arabic News Headlines. Proceedings of the International Conference on Advanced Intelligent Systems and Informatics 2019, Cairo, Egypt.10.1007/978-3-030-31129-2_38 ↗
77. Talafha, B., Za’ter, M.E., Suleiman, S., Al-Ayyoub, M., and Al-Kabi, M.N. (2021, January 1–3). Sarcasm Detection and Quantification in Arabic Tweets. Proceedings of the International Conference on Tools with Artificial Intelligence, ICTAI, Washington, DC, USA.10.1109/ICTAI52525.2021.00177 ↗
78. Zhang, L., Zhao, X., Song, X., Fang, Y., Li, D., and Wang, H. (2022). A Novel Chinese Sarcasm Detection Model Based on Retrospective Reader. MultiMedia Modeling, Springer. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics).10.1007/978-3-030-98355-0_23 ↗
79. Liu, L., Chen, X., and He, B. (2020). End-to-End Multi-task Learning for Allusion Detection in Ancient Chinese Poems. Knowledge Science, Engineering and Management, Springer. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics).10.1007/978-3-030-55393-7_27 ↗
80. Okimoto, Y., Suwa, K., Zhang, J., and Li, L. (2021). Sarcasm Detection for Japanese Text Using BERT and Emoji. Database and Expert Systems Applications, Springer. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics).10.1007/978-3-030-86472-9_11 ↗
81. Sharma, Y., and Mandalam, A.V. (2021, January 2–4). Irony Detection in Non-English Tweets. Proceedings of the 2021 6th International Conference for Convergence in Technology, I2CT 2021, Pune, India.10.1109/I2CT51068.2021.9418186 ↗
82. Hyperbolic Feature-based Sarcasm Detection in Telugu Conversation Sentences
83. Bharti, S.K., Babu, K.S., and Jena, S.K. (2017). Harnessing Online News for Sarcasm Detection in Hindi Tweets. Pattern Recognition and Machine Intelligence, Springer. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics).10.1007/978-3-319-69900-4_86 ↗
84. Sarcasm Detection for Sentiment Analysis in Indonesian Tweets10.22146/ijccs.41136 ↗
85. Lunando, E., and Purwarianti, A. (2013, January 28–29). Indonesian social media sentiment analysis with sarcasm detection. Proceedings of the 2013 International Conference on Advanced Computer Science and Information Systems, ICACSIS 2013, Sanur Bali, Indonesia.10.1109/ICACSIS.2013.6761575 ↗
86. Anan, R., Apon, T.S., Hossain, Z.T., Modhu, E.A., Mondal, S., and Alam, M.D.G.R. (2023, January 8–11). Interpretable Bangla Sarcasm Detection using BERT and Explainable AI. Proceedings of the 2023 IEEE 13th Annual Computing and Communication Workshop and Conference (CCWC), Las Vegas, NV, USA.10.1109/CCWC57344.2023.10099331 ↗
87. Sarcasm detection in mash-up language using soft-attention based bi-directional LSTM and feature-rich CNN10.1016/j.asoc.2020.106198 ↗
88. Karabaş, A., and Dırı, B. (2020, January 5–7). Irony Detection with Deep Learning in Turkish Microblogs. Proceedings of the 2020 28th Signal Processing and Communications Applications Conference (SIU), IEEE, Gaziantep, Turkey.10.1109/SIU49456.2020.9302253 ↗
89. Satire identification in Turkish news articles based on ensemble of classifiers10.3906/elk-1907-11 ↗
90. Semantically-Informed Graph Neural Networks for Irony Detection in Turkish
91. Öztürk, A.U. (2024). Designing and Debiasing Binary Classifiers for Irony and Satire Detection, Middle East Technical University.
92. Riloff, E., Qadir, A., Surve, P., De Silva, L., Gilbert, N., and Huang, R. (2013, January 18–21). Sarcasm as contrast between a positive sentiment and negative situation. Proceedings of the EMNLP 2013—2013 Conference on Empirical Methods in Natural Language Processing, Seattle, WA, USA.10.18653/v1/D13-1066 ↗
93. Amir, S., Wallace, B.C., Lyu, H., Carvalho, P., and Silva, M.J. (2016, January 7–12). Modelling context with user embeddings for sarcasm detection in social media. Proceedings of the CoNLL 2016—20th SIGNLL Conference on Computational Natural Language Learning, Berlin, Germany.10.18653/v1/K16-1017 ↗
94. Bamman, D., and Smith, N.A. (2015, January 26–29). Contextualized sarcasm detection on twitter. Proceedings of the 9th International Conference on Web and Social Media, ICWSM 2015, Oxford, UK.
95. Barbieri, F., Saggion, H., and Ronzano, F. (2014, January 27). Modelling Sarcasm in Twitter, a Novel Approach. Proceedings of the 5th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, Baltimore, MD, USA.10.3115/v1/W14-2609 ↗
96. Ptáček, T., Habernal, I., and Hong, J. (2014). Sarcasm detection on Czech and English twitter. COLING 2014—25th International Conference on Computational Linguistics, Proceedings of COLING 2014: Technical Papers, Dublin, Ireland, 23–29 August 2014, Dublin City University and Association for Computational Linguistics.
97. Schifanella, R., De Juan, P., Tetreault, J., and Cao, L. (2016, January 15–19). Detecting sarcasm in multimodal social platforms. Proceedings of the 2016 ACM Multimedia Conference—MM 2016, Amsterdam, The Netherlands.
98. Joshi, A., Tripathi, V., Patel, K., Bhattacharyya, P., and Carman, M. (2016). Are word embedding-based features useful for sarcasm detection?. EMNLP 2016—Conference on Empirical Methods in Natural Language Processing, Proceedings, Association for Computational Linguistics.10.18653/v1/D16-1104 ↗
99. Sarcasm detection using soft attention-based bidirectional long short-term memory model with convolution network10.1109/ACCESS.2019.2899260 ↗
100. Subramanian, J., Sridharan, V., Shu, K., and Liu, H. (2019). Exploiting emojis for sarcasm detection. Social, Cultural, and Behavioral Modeling, Springer. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics).10.1007/978-3-030-21741-9_8 ↗