Edge Convolutional Networks for Style Change Detection in Arabic Multi-Authored Text
Source abstract
Abstract
The style change detection (SCD) task asks to find the positions of authors’ style changes within multi-authored texts. It has several application areas, such as forensics, cybercrime, and literary analysis. Since 2017, SCD solutions in English have been actively investigated. However, to the best of our knowledge, this task has not yet been investigated in Arabic text. Moreover, most existing SCD solutions represent boundaries surrounding segments by concatenating them. This shallow concatenation may lose style patterns within each segment and also increase input lengths while several embedding models restrict these lengths. This study seeks to bridge these gaps by introducing an Edge Convolutional Neural Network for the Arabic SCD task (ECNN-ASCD) solution. It represents boundaries as standalone learnable parameters across layers based on graph neural networks. ECNN-ASCD was trained on an Arabic dataset containing three classes of instances according to difficulty level: easy, medium, and hard. The results show that ECNN-ASCD achieved a high F1 score of 0.9945%, 0.9381%, and 0.9120% on easy, medium, and hard instances, respectively. The ablation experiments demonstrated the effectiveness of ECNN-ASCD components. As the first publicly available solution for Arabic SCD, ECNN-ASCD would open the door for more active research on solving this task and contribute to boosting research in Arabic NLP.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Authorship Attribution and Profiling · Natural Language Processing Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science, King Saud University, Riyadh 11451, Saudi Arabia
Computer Science Department, Imam Mohammad Ibn Saud Islamic University (IMSIU), Riyadh 11432, Saudi Arabia
References (100+)
Show source-provided references
1. Overview of the Author Identification Task at PAN-2017: Style Breach Detection and Author Clustering
2. Overview of the Author Identification Task at PAN-2018
3. Overview of the Style Change Detection Task at PAN 2019
4. Overview of the Style Change Detection Task at PAN 2020
5. Overview of the Style Change Detection Task at PAN 2021
6. Overview of the Style Change Detection Task at PAN 2022
7. Zangerle, E., Mayerl, M., Potthast, M., and Stein, B. (2023). Overview of the Multi-Author Writing Style Analysis Task at PAN 2023. Working Notes of CLEF 2023—Conference and Labs of the Evaluation Forum, CEUR-WS.org. CEUR Workshop Proceedings.
8. Authorship Identification of Documents with High Content Similarity10.1007/s11192-018-2661-6 ↗
9. Akiva, N., and Koppel, M. (2012, January 22–24). Identifying Distinct Components of a Multi-author Document. Proceedings of the 2012 European Intelligence and Security Informatics Conference, Odense, Denmark.10.1109/EISIC.2012.16 ↗
10. Ensemble-Based Clustering for Writing Style Change Detection in Multi-Authored Textual Documents
11. Overview of the Multi-Author Writing Style Analysis Task at PAN 2024
12. The Graph Neural Network Model10.1109/TNN.2008.2005605 ↗
13. Graph-Based Siamese Network for Authorship Verification
14. Graph-Based Siamese Network for Authorship Verification
15. Valdez-Valenzuela, A., Martinez-Galicia, J.A., and Gomez-Adorno, H. (2023). Heterogeneous-Graph Convolutional Network for Authorship Verification. Working Notes of CLEF 2023—Conference and Labs of the Evaluation Forum, CEUR-WS.org. CEUR Workshop Proceedings.
16. Zhang, Y., Qi, P., and Manning, C.D. (November, January 31). Graph Convolution over Pruned Dependency Trees Improves Relation Extraction. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium.10.18653/v1/D18-1244 ↗
17. Yu, B., Mengge, X., Zhang, Z., Liu, T., Yubin, W., and Wang, B. (2020, January 8–13). Learning to Prune Dependency Trees with Rethinking for Neural Relation Extraction. Proceedings of the 28th International Conference on Computational Linguistics, Barcelona, Spain.10.18653/v1/2020.coling-main.341 ↗
18. Li, B., Fan, Y., Sataer, Y., Gao, Z., and Gui, Y. (2022). Improving Semantic Dependency Parsing with Higher-Order Information Encoded by Graph Neural Networks. Appl. Sci., 12.10.3390/app12084089 ↗
19. A Graph Convolutional Network With Multiple Dependency Representations for Relation Extraction10.1109/ACCESS.2021.3086480 ↗
20. Relation Extraction with Convolutional Network over Learnable Syntax-Transport Graph
21. Zhou, L., Wang, T., Qu, H., Huang, L., and Liu, Y. (2020). A Weighted GCN with Logical Adjacency Matrix for Relation Extraction. ECAI 2020, iOS Press.10.3233/FAIA200360 ↗
22. Mandya, A., Bollegala, D., and Coenen, F. (2020, January 8–13). Graph Convolution over Multiple Dependency Sub-graphs for Relation Extraction. Proceedings of the 28th International Conference on Computational Linguistics, Barcelona, Spain.10.18653/v1/2020.coling-main.565 ↗
23. Jin, Z., Yang, Y., Qiu, X., and Zhang, Z. (2020). Relation of the Relations: A New Paradigm of the Relation Extraction Problem. arXiv.
24. Tian, Y., Chen, G., Song, Y., and Wan, X. (2021, January 1–6). Dependency-driven Relation Extraction with Attentive Graph Convolutional Networks. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Online.10.18653/v1/2021.acl-long.344 ↗
25. Representation Iterative Fusion Based on Heterogeneous Graph Neural Network for Joint Entity and Relation Extraction10.1016/j.knosys.2021.106888 ↗
26. Liu, P., Wang, L., Zhao, Q., Chen, H., Feng, Y., Lin, X., and He, L. (2021, January 10–31). ECNU_ICA_1 SemEval-2021 Task 4: Leveraging Knowledge-enhanced Graph Attention Networks for Reading Comprehension of Abstract Meaning. Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), Online.10.18653/v1/2021.semeval-1.20 ↗
27. Guo, Z., Zhang, Y., and Lu, W. (August, January 28). Attention Guided Graph Convolutional Networks for Relation Extraction. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy.
28. Improve Relation Extraction with Dual Attention-Guided Graph Convolutional Networks10.1007/s00521-020-05087-z ↗
29. Arabic Natural Language Processing: Challenges and Solutions10.1145/1644879.1644881 ↗
30. Habash, N., Soudi, A., and Buckwalter, T. (2007). On Arabic transliteration. Arabic Computational Morphology: Knowledge-Based and Empirical Methods, Springer.
31. Zitouni, I. (2014). Morphological processing of semitic languages. Natural Language Processing of Semitic Languages, Springer.10.1007/978-3-642-45358-8 ↗
32. Ahmed, M.A., and Trausan-Matu, S. (2017, January 21–23). Using natural language processing for analyzing Arabic poetry rhythm. Proceedings of the 2017 16th RoEduNet Conference: Networking in Education and Research (RoEduNet), Targu Mures, Romania.10.1109/ROEDUNET.2017.8123759 ↗
33. Style Breach Detection: An Unsupervised Detection Model
34. A Model for Style Change Detection at a Glance
35. OPI-JSA at CLEF 2017: Author Clustering and Style Breach Detection
36. Ramachandran, K.M., and Tsokos, C.P. (2020). Mathematical Statistics with Applications in R, Elsevier. [3rd ed.].
37. The Regression Analysis of Binary Sequences10.1111/j.2517-6161.1959.tb00334.x ↗
38. Ho, T.K. (1995, January 14–16). Random decision forests. Proceedings of the 3rd International Conference on Document Analysis and Recognition, Montreal, Canada.
39. Support-vector Networks10.1007/BF00994018 ↗
40. Style Change Detection using Discourse Markers
41. Jacobo, G.X., Dehesa-Corona, V., Rojas-Reyes, A., and Gomez-Adorno, H. (2023). Authorship Verification Machine Learning Methods For Style Change Detection In Texts. Working Notes of CLEF 2023—Conference and Labs of the Evaluation Forum, CEUR-WS.org. CEUR Workshop Proceedings.
42. Nath, S. (2021). Style Change Detection. [Ph.D. Thesis, Université de Neuchâtel].
43. Writing Style Change Detection on Multi-Author Documents
44. Detecting a Change of Style Using Text Statistics
45. An Ensemble-Rich Multi-Aspect Approach for Robust Style Change Detection
46. Some Methods for Classification and Analysis of Multivariate Observations
47. Elamine, M., Mechti, S., and Belguith, L.H. (2019, January 3–7). An Unsupervised Method for Detecting Style Breaches in a Document. Proceedings of the 2019 IEEE/ACS 16th International Conference on Computer Systems and Applications (AICCSA), Abu Dhabi, United Arab Emirates.10.1109/AICCSA47632.2019.9035264 ↗
48. Mandic, L., Milkovic, F., and Doria, S. (2019). Combining the Powers of Clustering Affinities in Style Change Detection. Course Project Reports, University of Zagreb.
49. Style Change Detection with Feed-forward Neural Networks
50. Style Change Detection by Threshold Based and Window Merge Clustering Methods
51. Mixed Style Feature Representation and B0-maximal Clustering for Style Change Detection
52. Style Change Detection using Siamese Neural Networks
53. Long Short-Term Memory10.1162/neco.1997.9.8.1735 ↗
54. Lecun, Y. (1989). Generalization and Network Design Strategies. Connectionism in Perspective, Elsevier.
55. Style Change Detection on Real-World Data using an LSTM-powered Attribution Algorithm
56. Character-based Convolutional Neural Network for Style Change Detection
57. Müller, P. (2019). Style Change Detection. [Bachelor’s Thesis, ETH Zurich].
58. Jurafsky, D., and Martin, J.H. (2000). Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition, Prentice Hall.
59. Bromley, J., Guyon, I., LeCun, Y., Säckinger, E., and Shah, R. (December, January 29). Signature Verification using a ‘Siamese’ Time Delay Neural Network. Proceedings of the the 6th International Conference on Neural Information Processing Systems, San Francisco, CA, USA. NIPS’93.
60. A Parallel Hierarchical Attention Network for Style Change Detection
61. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2019, January 2–7). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA.
62. Style Change Detection Based On Bert And Conv1d
63. Style Change Detection Based On Bi-LSTM And Bert
64. Liu, G., Yan, Z., Wang, T., and Zhan, K. (2022, January 26–28). ARTW: A Model of Author Recognition based on Writing Style by Recognizing Style Crack. Proceedings of the ICISS 2022: 2022 the 5th International Conference on Information Science and Systems, Beijing, China.10.1145/3561877.3561898 ↗
65. Style Breach Detection with Neural Sentence Embeddings
66. Kiros, R., Zhu, Y., Salakhutdinov, R., Zemel, R.S., Torralba, A., Urtasun, R., and Fidler, S. (2015, January 8–13). Skip-Thought Vectors. Proceedings of the 28th International Conference on Neural Information Processing Systems, Cambridge, MA, USA.
67. Three Style Similarity: Sentence-embedding, Auxiliary Words, Punctuation
68. Reimers, N., and Gurevych, I. (2019, January 3–7). Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China.10.18653/v1/D19-1410 ↗
69. Enriching Word Vectors with Subword Information
70. He, P., Liu, X., Gao, J., and Chen, W. (2021, January 4–8). DeBERTa: Decoding-enhanced BERT with Disentangled Attention. Proceedings of the International Conference on Learning Representations ICLR 2021, Vienna, Austria.
71. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv.
72. Clark, K., Luong, M.T., Le, Q.V., and Manning, C.D. (2020, January 26–30). ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators. Proceedings of the 8th International Conference on Learning Representations, Addis Ababa, Ethiopia.
73. Muennighoff, N., Wang, T., Sutawika, L., Roberts, A., Biderman, S., Le Scao, T., Bari, M.S., Shen, S., Yong, Z.X., and Schoelkopf, H. (2023, January 9–14). Crosslingual Generalization through Multitask Finetuning. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, Toronto, ON, Canada.10.18653/v1/2023.acl-long.891 ↗
74. Style Change Detection Using BERT
75. Style Change Detection based on Prompt
76. Style Change Detection Based On Writing Style Similarity
77. Ensemble Pre-trained Transformer Models for Writing Style Change Detection
78. Multi-label Style Change Detection by Solving a Binary Classification Problem
79. Style Change Detection: Method Based On Pre-trained Model And Similarity Recognition
80. Kucukkaya, I.E., Sahin, U., and Toraman, C. (2023). ARC-NLP at PAN 2023: Transition-Focused Natural Language Inference for Writing Style Detection. Working Notes of CLEF 2023—Conference and Labs of the Evaluation Forum, CEUR-WS.org.
81. Ye, Z., Zhong, C., Qi, H., and Han, Y. (2023). Supervised Contrastive Learning for Multi-Author Writing Style Analysis. Working Notes of CLEF 2023—Conference and Labs of the Evaluation Forum, CEUR-WS.org. CEUR Workshop Proceedings.
82. Hashemi, A., and Shi, W. (2023). EnhancingWriting Style Change Detection using Transformer-based Models and Data Augmentation. Working Notes of CLEF 2023—Conference and Labs of the Evaluation Forum, CEUR-WS.org. CEUR Workshop Proceedings.
83. Huang, M., Huang, Z., and Kong, L. (2023). Encoded Classifier Using Knowledge Distillation for Multi-Author Writing Style Analysis. Working Notes of CLEF 2023—Conference and Labs of the Evaluation Forum, CEUR-WS.org. CEUR Workshop Proceedings.
84. Knowledge Graphs: Opportunities and Challenges10.1007/s10462-023-10465-9 ↗
85. Kipf, T.N., and Welling, M. (2017). Semi-Supervised Classification with Graph Convolutional Networks. arXiv.
86. Dynamic Graph CNN for Learning on Point Clouds
87. Deep Learning-based Text Classification: A Comprehensive Review10.1145/3439726 ↗
88. Team foshan-university-of-guangdong at PAN: Adaptive Entropy-Based Stability-Plasticity for Multi-Author Writing Style Analysis
89. BERT-Based Similarity Measures Oriented Approach for Style Change Detection
90. Team NYCU-NLP at PAN 2024: Integrating Transformers with Similarity Adjustments for Multi-Author Writing Style Analysis
91. Team Text Understanding and Analysis at PAN: Utilizing BERT Series Pre-training Model for Multi-Author Writing Style Analysis
92. Team bingezzzleep at PAN: A Writing Style Change Analysis Model Based on RoBERTa Encoding and Contrastive Learning for Multi-Author Writing Style Analysis
93. Team Chen at PAN: Integrating R-Drop and Pre-trained Language Model for Multi-author Writing Style Analysis
94. Team baker at PAN: Enhancing Writing Style Change Detection with Virtual Softmax
95. Team karami-sh at PAN: Transformer-based Ensemble Learning for Multi-Author Writing Style Analysis
96. Team Liuc0757 at PAN: A Writing Style Embedding Method Based on Contrastive Learning for Multi-Author Writing Style Analysis
97. Zamir, M.T., Ayub, M.A., Gul, A., Ahmad, N., and Ahmad, K. (2024). Stylometry Analysis of Multi-authored Documents for Authorship and Author Style Change Detection. arXiv.
98. Fine-Tuned Reasoning for Writing Style Analysis
99. Team fosu-stu at PAN: Supervised Fine-Tuning of Large Language Models for Multi Author Writing Style Analysis
100. eam riyahsanjesh at PAN: Multi-feature with CNN and Bi-LSTM Neural Network Approach to Style Change Detection