Post-Authorship Attribution Using Regularized Deep Neural Network
Source abstract
Abstract
Post-authorship attribution is a scientific process of using stylometric features to identify the genuine writer of an online text snippet such as an email, blog, forum post, or chat log. It has useful applications in manifold domains, for instance, in a verification process to proactively detect misogynistic, misandrist, xenophobic, and abusive posts on the internet or social networks. The process assumes that texts can be characterized by sequences of words that agglutinate the functional and content lyrics of a writer. However, defining an appropriate characterization of text to capture the unique writing style of an author is a complex endeavor in the discipline of computational linguistics. Moreover, posts are typically short texts with obfuscating vocabularies that might impact the accuracy of authorship attribution. The vocabularies include idioms, onomatopoeias, homophones, phonemes, synonyms, acronyms, anaphora, and polysemy. The method of the regularized deep neural network (RDNN) is introduced in this paper to circumvent the intrinsic challenges of post-authorship attribution. It is based on a convolutional neural network, bidirectional long short-term memory encoder, and distributed highway network. The neural network was used to extract lexical stylometric features that are fed into the bidirectional encoder to extract a syntactic feature-vector representation. The feature vector was then supplied as input to the distributed high networks for regularization to minimize the network-generalization error. The regularized feature vector was ultimately passed to the bidirectional decoder to learn the writing style of an author. The feature-classification layer consists of a fully connected network and a SoftMax function to make the prediction. The RDNN method was tested against thirteen state-of-the-art methods using four benchmark experimental datasets to validate its performance. Experimental results have demonstrated the effectiveness of the method when compared to the existing state-of-the-art methods on three datasets while producing comparable results on one dataset.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Authorship Attribution and Profiling · Hate Speech and Cyberbullying Detection · Spam and Phishing Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science, University of Pretoria, Lynnwood Road, Pretoria 0002, South Africa
School of Computer Science and Applied Mathematics, University of the Witwatersrand, Johannesburg 2000, South Africa
School of Electrical and Information Engineering, University of the Witwatersrand, Johannesburg 2000, South Africa
MICT SETA Center of Excellence in 4IR, Durban University of Technology, Durban 4001, South Africa
References (100+)
Show source-provided references
1. Learning stylometric representations for authorship analysis10.1109/TCYB.2017.2766189 ↗
2. Detecting traffic information from social media texts with deep learning approaches10.1109/TITS.2018.2871269 ↗
3. Java, A., Song, X., Finin, T., and Tseng, B. (2007). Why We Twitter: An Analysis of a Microblogging Community. International Workshop on Social Network Mining and Analysis, Springer.10.1145/1348549.1348556 ↗
4. Wikipedia (2022, June 14). Twitter—Wikipedia. Available online: https://en.wikipedia.org/wiki/Twitter.
5. Mishne, G. (2022, June 14). Applied Text Analytics for Blogs. Universiteit van Amsterdam. Available online: http://brenocon.com/gilad_mishne_phd_thesis_ch6.pdf.
6. Lin, Y. (2022, June 14). 10 Blogging Statistics You Need to Know in 2021 [Infographic]. Available online: https://www.oberlo.com/blog/blogging-statistics.
7. A framework for authorship identification of online messages: Writing-style features and classification techniques10.1002/asi.20316 ↗
8. Abiodun, M., Olugbara, O.O., and Ojo, S.O. (2010, January 6–7). Identifying Advanced Fee Fraud Activities on Internet Using Machine Learning Algorithms. Proceedings of the 3rd International Conference on Computational Intelligence and Industrial Application, Wuhan, China.
9. Afroz, S., Brennan, M., and Greenstadt, R. (2012, January 20–23). Detecting Hoaxes, Frauds, and Deception in Writing Style Online. Proceedings of the 2012 IEEE Symposium on Security and Privacy, San Francisco, CA, USA.10.1109/SP.2012.34 ↗
10. Diale, M., van der Walt, C., Celik, T., and Modupe, A. (December, January 30). Feature Selection and Support Vector Machine Hyper-Parameter Optimi-sation for Spam Detection. Proceedings of the 2016 Pattern Recognition Association of South Africa and Robotics and Mechatronics International Conference (PRASA-RobMech), Stellenbosch, South Africa.10.1109/RoboMech.2016.7813162 ↗
11. Harnessing Big Data for Communicable Tropical and Sub-Tropical Disorders: Implications from a Systematic Review of the Literature10.3389/fpubh.2018.00090 ↗
12. Writer identification using microblog-ging texts for social media forensics10.1109/TBIOM.2021.3078073 ↗
13. Who am I? Analyzing digital personas in cy-bercrime investigations10.1109/MC.2013.68 ↗
14. Can anonymous posters on medical forums be reidentified?10.2196/jmir.2514 ↗
15. Who am I online? Understanding the meaning of online contexts for identity development10.1007/s40894-016-0025-0 ↗
16. Adversarial stylometry: Circumventing authorship recognition to preserve privacy and anonymity10.1145/2382448.2382450 ↗
17. Afroz, S., Islam, A.C., Stolerman, A., Greenstadt, R., and McCoy, D. (2014, January 18–21). Doppelgänger Finder: Taking Stylometry to the Underground. Proceedings of the IEEE Symposium on Security and Privacy, San Jose, CA, USA.10.1109/SP.2014.21 ↗
18. Arabic Authorship Attribution: An Extensive Study on Twitter Posts10.1145/3236391 ↗
19. Authorship attribution for social media forensics10.1109/TIFS.2016.2603960 ↗
20. Yu, C., and Ginsberg, A. (2018, January 28–30). Social Networking without Sacrificing Privacy. Proceedings of the 2018 International Conference on System Science and Engineering (ICSSE), New Taipei City, Taiwan.10.1109/ICSSE.2018.8520188 ↗
21. A survey of modern authorship attribution methods10.1002/asi.21001 ↗
22. Computational methods in authorship attribution10.1002/asi.20961 ↗
23. Surveying stylometry techniques and applications10.1145/3132039 ↗
24. Code authorship attribution: Methods and challenges10.1145/3292577 ↗
25. The rhetorical parsing of unrestricted texts: A surface-based approach10.1162/089120100561755 ↗
26. An effective and scalable framework for authorship attribution query processing10.1109/ACCESS.2018.2869198 ↗
27. A novel approach of mining write-prints for authorship attribution in e-mail forensics10.1016/j.diin.2008.05.001 ↗
28. Linguistic data mining with complex networks: A stylometric-oriented approach10.1016/j.ins.2019.01.040 ↗
29. Modupe, A., Olugbara, O.O., and Ojo, S.O. (2011, January 11). Exploring Support Vector Machines and Random Forests to Detect Advanced Fee Fraud Activities on Internet. Proceedings of the 2011 IEEE 11th International Conference on Data Mining Workshops, Vancouver, BC, Canada.10.1109/ICDMW.2011.81 ↗
30. Modupe, A., Olugbara, O.O., and Ojo, S.O. (2012, January 2–4). Comparing Supervised Learning Classifiers to Detect Advanced Fee Fraud Activities on Internet. Proceedings of the International Conference on Computer Science and Information Technology, Bangalore, India.10.1007/978-3-642-27317-9_10 ↗
31. A visualizable evidence-driven approach for authorship attribution10.1145/2699910 ↗
32. Kešelj, V., Peng, F., Cercone, N., and Thomas, C. (2003, January 22–25). N-gram-based author profiles for authorship attribution. Proceedings of the Conference Pacific Association for Computational Linguistics, Halifax, NS, Canada.
33. Plagiarism and authorship analysis: Introduction to the special issue10.1007/s10579-011-9136-1 ↗
34. Ramnial, H., Panchoo, S., and Pudaruth, S. (2016). Authorship attribution using stylometry and machine learning techniques. Intelligent Systems Technologies and Applications, Springer.10.1007/978-3-319-23036-8_10 ↗
35. Fabien, M., Villatoro-Tello, E., Motlicek, P., and Parida, S. (2020, January 18–21). BertAA: BERT fine-tuning for authorship attribution. Proceedings of the 17th International Conference on Natural Language Processing (ICON), Bihar, India.
36. Pennacchiotti, M., and Popescu, A.-M. (2011, January 17–21). A Machine Learning Approach to Twitter user Classification. Proceedings of the Inter-national AAAI Conference on Web and Social Media, Barcelona, Spain.
37. Examining multiple features for author profiling
38. Golbeck, J., and Hansen, D. (2011, January 7–12). Computing political preference among Twitter followers. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Vancouver, BC, Canada.10.1145/1978942.1979106 ↗
39. A method for computing political preference among Twitter followers10.1016/j.socnet.2013.07.004 ↗
40. Conover, M., Ratkiewicz, J., Francisco, M., Gonçalves, B., Menczer, F., and Flammini, A. (2011, January 17–21). Political polarization on Twitter. Proceedings of the International AAAI Conference on Web and Social Media, Barcelona, Spain.
41. Ma, W., Liu, R., Wang, L., and Vosoughi, S. (2020). Towards improved model design for authorship identification: A survey on writing style understanding: A survey on writing style understanding. arXiv.
42. Searching with style: Authorship attribution in classic literature
43. Bigrams of syntactic labels for authorship discrimination of short texts10.1093/llc/fqm023 ↗
44. Frantzeskou, G., Stamatatos, E., Gritzalis, S., and Katsikas, S. (2006, January 20–28). Effective Identification of Source Code Authors Using Byte-Level Information. Proceedings of the 28th International Conference on Software Engineering, Shanghai, China.10.1145/1134285.1134445 ↗
45. Frantzeskou, G., Stamatatos, E., Gritzalis, S., and Katsikas, S. (2006). Source Code Author Identification Based on N-Gram Author Profiles. IFIP International Conference on Artificial Intelligence Applications and Innovations, Springer.10.1007/0-387-34224-9_59 ↗
46. Identifying Authorship by Byte-Level n-Grams: The Source Code Author Profile (Scap) Method
47. Escalante, J., Solorio, T., and Montes, M. (2011, January 19–24). Local Histograms of Character N-grams for Authorship Attribution. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Portland, OR, USA.
48. Rcv1: A new benchmark collection for text categorization research
49. Koppel, M., Schler, J., Argamon, S., and Messeri, E. (2006, January 6–11). Authorship Attribution with Thousands of Candidate Authors. Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Seattle, WA, USA.10.1145/1148170.1148304 ↗
50. Authorship attribution in the wild10.1007/s10579-009-9111-2 ↗
51. Sapkota, U., Bethard, S., Montes, M., and Solorio, T. (June, January 31). Not all character n-grams are created equal: A study in authorship attribution. Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Denver, CO, USA.10.3115/v1/N15-1010 ↗
52. Plakias, S., and Stamatatos, E. (2008, January 2–4). Tensor space models for authorship identification. Proceedings of the Hellenic Conference on Artificial Intelligence, Syros, Greece.
53. Muttenthaler, L., Lucas, G., and Amann, J. (2022, January 20). Authorship Attribution in Fan-Fictional Texts Given Variable Length Character and Word N-grams. Notebook for PAN at CLEF 2019. Available online: http://ceur-ws.org/Vol-2380/paper_49.pdf.
54. PAN (2022, January 20). Cross-Domain Authorship Attribution 2019. Available online: https://pan.webis.de/clef19/pan19-web/authorship-attribution.html.
55. Markov, I., Stamatatos, E., and Sidorov, G. (2017, January 17–23). Improving Cross-Topic Authorship Attribution: The Role of Pre-Processing. Proceedings of the International Conference on Computational Linguistics and Intelligent Text Processing, Budapest, Hungary.10.1007/978-3-319-77116-8_21 ↗
56. Authorship attribution with topic models10.1162/COLI_a_00173 ↗
57. Rosen-Zvi, M., Griffiths, T., Steyvers, M., and Smyth, P. (2004, January 7–11). The Author-Topic Model for Authors and Documents. Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence, Banff, AB, Canada.
58. Latent Dirichlet allocation
59. Character-level text classification via convolutional neural network and gated recurrent unit10.1007/s13042-020-01084-9 ↗
60. Sari, Y., Vlachos, A., and Stevenson, M. (2017, January 3–7). Continuous n-gram representations for authorship attribution. Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, Valencia, Spain.10.18653/v1/E17-2043 ↗
61. Source Code Authorship Attribution Using Hybrid Approach of Program Dependence Graph and Deep Learning Model10.1109/ACCESS.2019.2943639 ↗
62. Alrabaee, S., Shirani, P., Debbabi, M., and Wang, L. (2016). On the Feasibility of Malware Authorship Attribution. International Symposium on Foundations and Practice of Security, Springer.10.1007/978-3-319-51966-1_17 ↗
63. Burrows, S., and Tahaghoghi, S.M.M. (2007, January 10). Source Code Authorship Attribution Using N-grams. Proceedings of the Twelth Australasian Document Computing Symposium, Melbourne, Australia.
64. Source Code Author Identification with Unsupervised Feature Learning10.1016/j.patrec.2012.10.027 ↗
65. Alsulami, B., Dauber, E., Harang, R., Mancoridis, S., and Greenstadt, R. (2017). Source Code Authorship Attribution Using Long Short-Term Memory Based Networks. European Symposium on Research in Computer Security, Springer.10.1007/978-3-319-66402-6_6 ↗
66. Zhang, C., Wang, S., Wu, J., and Niu, Z. (2017). Authorship Identification of Source Codes. Asia-Pacific Web (APWeb) and Web-Age Information Management (WAIM) Joint Conference on Web and Big Data, Springer.10.1007/978-3-319-63579-8_22 ↗
67. Transparency and accountability in AI decision support: Explaining and visualizing convolutional neural networks for text information10.1016/j.dss.2020.113302 ↗
68. Layton, R., McCombie, S., and Watters, P. (2012, January 29–30). Authorship attribution of IRC messages using inverse author frequency. Proceedings of the 2012 Third Cybercrime and Trustworthy Computing Workshop, Ballarat, Australia.10.1109/CTC.2012.11 ↗
69. Juola, P. (2008). Authorship Attribution, Now Publishers Inc.10.1561/9781601981196 ↗
70. Recentred local profiles for authorship attribution10.1017/S1351324911000180 ↗
71. Schwartz, R., Tsur, O., Rappoport, A., and Koppel, M. (2013, January 18–21). Authorship attribution of micro-messages. Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, WA, USA.
72. Donais, J.A., Frost, R.A., Peelar, S.M., and Roddy, R.A. (2013, January 25–28). A system for the automated author attribution of text and instant messages. Proceedings of the 2013 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, Niagara, ON, Canada.10.1145/2492517.2500308 ↗
73. Okuno, S., Asai, H., and Yamana, H. (2014, January 27–30). A challenge of authorship identification for ten-thousand-scale microblog users. Proceedings of the 2014 IEEE International Conference on Big Data (Big Data), Washington, DC, USA.10.1109/BigData.2014.7004491 ↗
74. Bhargava, M., Mehndiratta, P., and Asawa, K. (2013, January 6–9). Stylometric analysis for authorship attribution on Twitter. Proceedings of the International Conference on Big Data Analytics, Silicon Valley, CA, USA.10.1007/978-3-319-03689-2_3 ↗
75. Ruder, S., Ghaffari, P., and Breslin, J.G. (2016). Character-level and multi-channel convolutional neural networks for large-scale authorship attribution. arXiv.
76. Shrestha, P., Sierra, S., González, F.A., Montes-y-Gómez, M., Rosso, P., and Solorio, T. (2017, January 3–7). Convolutional neural networks for au-thorship attribution of short texts. Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics EACL, Valencia, Spain.10.18653/v1/E17-2106 ↗
77. Zhang, R., Hu, Z., Guo, H., and Mao, Y. (November, January 31). Syntax encoding with application in authorship attribution. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium.10.18653/v1/D18-1294 ↗
78. Jafariakinabad, F., and Hua, K.A. (2019, January 16–19). Style-Aware Neural Model with Application in Authorship Attribution. Proceedings of the 2019 18th IEEE International Conference on Machine Learning and Applications (ICMLA), Boca Raton, FL, USA.10.1109/ICMLA.2019.00061 ↗
79. Jafariakinabad, F., Tarnpradab, S., and Hua, K.A. (2020, January 17–18). Syntactic neural model for authorship attribution. Proceedings of the Thirty-Third International Flairs Conference, Miami, FL, USA.
80. Seroussi, Y., Zukerman, I., and Bohnert, F. (2011, January 23–24). Authorship attribution with latent Dirichlet allocation. Proceedings of the Fifteenth Conference on Computational Natural Language Learning, Portland, OR, USA.
81. Kim, Y. (2014, January 25–29). Convolutional neural networks for sentence classification. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.10.3115/v1/D14-1181 ↗
82. al Zamal, F., Liu, W., and Ruths, D. (2012, January 4–7). Homophily and latent attribute inference: Inferring latent attributes of Twitter users from neighbors. Proceedings of the International AAAI Conference on Web and Social Media, Dublin, Ireland.
83. Hitschler, J., van den Berg, E., and Rehbein, I. (2017, January 8). Authorship attribution with convolutional neural networks and POS-Eliding. Proceedings of the Workshop on Stylistic Variation, Copenhagen, Denmark.10.18653/v1/W17-4907 ↗
84. Bird, S., Dale, R., Dorr, B.J., Gibson, B., Joseph, M.T., Kan, M., Lee, D., Powley, B., Radev, D.R., and Tan, Y.F. (2008, January 28–30). The ACL anthology reference corpus: A reference dataset for bibliographic research in computational linguistics. Proceedings of the Sixth In-ternational Conference on Language Resources and Evaluation (LREC’08), Marrakech, Morocco.
85. Hu, Z., Lee, R.K.-W., Wang, L., Lim, E., and Dai, B. (2020, January 12–14). Deepstyle: User style embedding for authorship attribution of short texts. Proceedings of the Asia-Pacific Web (APWeb) and Web-Age Information Management (WAIM) Joint International Conference on Web and Big Data, Tianjin, China.10.1007/978-3-030-60290-1_17 ↗
86. Schler, J., Koppel, M., Argamon, S., and Pennebaker, J.W. (2006, January 27–29). Effects of age and gender on blogging. Proceedings of the AAAI Spring Symposium: Computational Approaches to Analyzing Weblogs, Stanford, CA, USA.
87. Murauer, B., and Specht, G. (2021, January 10–11). Developing a benchmark for reducing data bias in authorship attribution. Proceedings of the 2nd Workshop on Evaluation and Comparison of NLP Systems, Punta Cana, Dominican Republic.10.18653/v1/2021.eval4nlp-1.18 ↗
88. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language under-standing. arXiv.
89. Sanh, V., Debut, L., Chaumond, J., and Wolf, T. (2019). DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter. arXiv.
90. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). Roberta: A robustly optimized Bert pretraining approach. arXiv.
91. Exploring Syntactic and Semantic Features for Authorship Attribution10.1016/j.asoc.2021.107815 ↗
92. Kim, Y., Jernite, Y., Sontag, D., and Rush, A.M. (2016, January 12–17). Character-aware neural language models. Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.10.1609/aaai.v30i1.10362 ↗
93. Training very deep networks
94. Ribeiro, T., Singh, S., and Guestrin, C. (2016, January 13–17). ‘Why should I trust you?’ Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.10.1145/2939672.2939778 ↗
95. A unified approach to interpreting model predictions
96. Mikolov, T., Sutskever, I., Chen, K., Corrado, G., and Dean, J. (2013, January 5–10). Distributed representations of words and phrases and their compo-sitionality. Proceedings of the 26th International Conference on Neural Information Processing Systems, Lake Tahoe, NV, USA.
97. Le, Q., and Mikolov, T. (2014, January 21–26). Distributed representations of sentences and documents. Proceedings of the International Conference on Machine Learning, Beijing, China.
98. Loshchilov, I., and Hutter, F. (2021, August 16). Fixing Weight Decay Regularization in Adam. Available online: https://openreview.net/pdf?id=rk6qdGgCZ.
99. Seroussi, Y., Zukerman, I., and Bohnert, F. (2010, January 20–14). Collaborative inference of sentiments from texts. Proceedings of the International Conference on User Modeling, Adaptation, and Personalization, Manoa, HI, USA.10.1007/978-3-642-13470-8_19 ↗
100. Glorot, X., and Bengio, Y. (2010, January 13–15). Understanding the difficulty of training deep feedforward neural networks. Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, Sardinia, Italy.