Upholding Academic Integrity amidst Advanced Language Models: Evaluating BiLSTM Networks with GloVe Embeddings for Detecting AI-Generated Scientific Abstracts
Source abstract
Abstract
The increasing fluency of advanced language models, such as GPT-3.5, GPT-4, and the recently introduced DeepSeek, challenges the ability to distinguish between human-authored and AI-generated academic writing. This situation is raising significant concerns regarding the integrity and authenticity of academic work. In light of the above, the current research evaluates the effectiveness of Bidirectional Long Short-Term Memory (BiLSTM) networks enhanced with pre-trained GloVe (Global Vectors for Word Representation) embeddings to detect AI-generated scientific abstracts drawn from the AI-GA (Artificial Intelligence Generated Abstracts) dataset. Two core BiLSTM variants were assessed: a single-layer approach and a dual-layer design, each tested under static or adaptive embeddings. The single-layer model achieved nearly 97% accuracy with trainable GloVe, occasionally surpassing the deeper model. Despite these gains, neither configuration fully matched the 98.7% benchmark set by an earlier LSTM-Word2Vec pipeline. Some runs were over-fitted when embeddings were fine-tuned, whereas static embeddings offered a slightly lower yet stable accuracy of around 96%. This lingering gap reinforces a key ethical and procedural concern: relying solely on automated tools, such as Turnitin’s AI-detection features, to penalize individuals’ risks and unjust outcomes. Misclassifications, whether legitimate work is misread as AI-generated or engineered text, evade detection, demonstrating that these classifiers should not stand as the sole arbiters of authenticity. A more comprehensive approach is warranted, one which weaves model outputs into a systematic process supported by expert judgment and institutional guidelines designed to protect originality.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Artificial Intelligence in Healthcare and Education · Academic integrity and plagiarism
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (72)
Show source-provided references
1. Language models are few-shot learners
2. OpenAI. ChatGPT: optimizing language models for dialogue [Internet]. [cited 2025 Feb 20]. Available from: https://openai.com/blog/chatgpt/.
3. Bi X, Chen D, Chen G, Chen S, Dai D, Deng C, et al. Deepseek LLM: scaling open-source language models with longtermism. arXiv:2401.02954. 2024.
4. Zhu Q, Guo D, Shao Z, Yang D, Wang P, Xu R, et al. DeepSeek-Coder-V2: breaking the barrier of closed-source models in code intelligence. arXiv: 2406.11931. 2024.
5. Jawahar G, Sagot B, Seddah D. What does BERT learn about the structure of language? In: Proceedings of the Annual Meeting of the Association for Computational Linguistics; 2019 Jul 28–Aug 2; Florence, Italy.10.18653/v1/P19-1356 ↗
6. Testing of detection tools for AI-generated text10.1007/s40979-023-00146-z ↗
7. Chatting and cheating: ensuring academic integrity in the era of ChatGPT10.1080/14703297.2023.2190148 ↗
8. ChatGPT in higher education—a synthesis of the literature and a future research agenda10.1007/s10639-024-12723-x ↗
9. Detection of fake generated scientific abstracts
10. Human-written vs AI-generated texts in orthopedic academic literature: comparative qualitative analysis10.2196/52164 ↗
11. Human versus artificial intelligence-generated arthroplasty literature: a single-blinded analysis of perceived communication, quality, and authorship source10.1002/rcs.2621 ↗
12. Evaluating human ability to distinguish between ChatGPT-generated and original scientific abstracts
13. Assessing the reproducibility of the structured abstracts generated by ChatGPT and Bard compared to human-written abstracts in the field of spine surgery: comparative analysis10.2196/52001 ↗
14. Evaluating science: a comparison of human and AI reviewers10.1017/jdm.2024.24 ↗
15. Comparisons of quality, correctness, and similarity between ChatGPT-generated and human-written abstracts for basic research: cross-sectional study10.2196/51229 ↗
16. AI vs. human—academic essay authenticity challenge
17. Leveraging multilingual proxy LLMs for machine-generated text detection in academic essays
18. Fine-tuned language models for detection of academic authenticity, results and thoughts
19. Guardians of academic integrity: multilingual detection of AI-generated essays
20. Fast and scalable method for detection of academic essay authenticity
21. The consistent lack of variance of psychological factors expressed by LLMs and spambots
22. SilverSpeak: evading AI-generated text detectors using homoglyphs
23. The potential and concerns of using AI in scientific research: chatGPT performance evaluation10.2196/47049 ↗
24. Does ChatGPT foster academic misconduct in the future?
25. Interplay of rationality and morality in using ChatGPT for academic misconduct10.1080/0144929X.2024.2325023 ↗
26. Addressing academic misconduct in the age of ChatGPT: strategies and solutions
27. Artificial intelligence can generate fraudulent but authentic-looking scientific medical articles: pandora’s box has been opened10.2196/46924 ↗
28. Evaluation and comparison of ophthalmic scientific abstracts and references by current artificial intelligence chatbots10.1001/jamaophthalmol.2023.3119 ↗
29. Reassessing AI in medicine: exploring the capabilities of AI in academic abstract synthesis10.2196/55920 ↗
30. Evaluating the coherence and diversity in AI-generated and paraphrased scientific abstracts: a fuzzy topic modeling approach
31. Detection and classification of ChatGPT generated contents using deep transformer models10.3389/frai.2025.1458707 ↗
32. AI-generated text using transformer-based approaches
33. Mobin MK, Islam MS. LuxVeri at GenAI detection task 3: cross-domain detection of AI-generated text using inverse perplexity-weighted ensemble of fine-tuned transformer models. arXiv:2501.11918v1. 2025.
34. Team Unibuc—NLP at GenAI detection task 1: Qwen it detect machine-generated text?
35. Advancing multilingual machine-generated text detection
36. A proposed model for distinguishing between human-based and ChatGPT content in scientific articles10.1109/ACCESS.2024.3448315 ↗
37. Detecting machine-generated academic essays in English and Arabic using ELECTRA and stylometry
38. Enhancing machine-generated text detection with semantic and probabilistic features
39. Pangram at GenAI detection task 3: an active learning approach to machine-generated text detection
40. Tracing the footprints of AI in radiology literature: a detailed analysis of journal abstracts10.1055/a-2224-9230 ↗
41. A study on distinguishing ChatGPT-generated and human-written orthopaedic abstracts by reviewers: decoding the discrepancies
42. Performance of a generative pre-trained transformer in generating scientific abstracts in dentistry: a comparative observational study10.1111/eje.13057 ↗
43. AI-generated text in otolaryngology publications: a comparative analysis before and after the release of ChatGPT10.1007/s00405-024-08834-3 ↗
44. Characterizing the increase in artificial intelligence content detection in oncology scientific abstracts from 2021 to 202310.1200/CCI.24.00077 ↗
45. Gotcha GPT: ensuring the integrity in academic writing10.1021/acs.jcim.4c01203 ↗
46. Detection of ChatGPT fake science with the xFakeSci learning algorithm10.1038/s41598-024-66784-6 ↗
47. Comparing scientific abstracts generated by ChatGPT to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers
48. Anagnostou P. Panagiotisanagnostou/AI-GA [Internet]. [cited 2025 Apr 29]. Available from: https://github.com/panagiotisanagnostou/AI-GA.
49. Bird · Natural language processing with Python: analyzing text with the natural language toolkit · 2009 · 504
50. Glove: global vectors for word representation
51. Efficient estimation of word representations in vector space
52. Bidirectional recurrent neural networks10.1109/78.650093 ↗
53. Long short-term memory10.1162/neco.1997.9.8.1735 ↗
54. Batch normalization: accelerating deep network training by reducing internal covariate shift
55. A system for massively parallel hyperparameter tuning
56. Meyes R, Lu M, Waubert de Puiseau C, Meisen T. Ablation studies in artificial neural networks. arXiv:1901.08644v2. 2019.
57. Goodfellow · Deep learning · 2016 · 1 · 800
58. Kingma D, Ba J. Adam: a method for stochastic optimization. arXiv:1412.6980. 2014.
59. Automatic early stopping using cross validation: quantifying the criteria10.1016/S0893-6080(98)00010-0 ↗
60. Cyclical learning rates for training neural networks
61. The regression analysis of binary sequences10.1111/j.2517-6161.1958.tb00292.x ↗
62. Naive (Bayes) at forty: the independence assumption in information retrieval
63. Random forests10.1023/A:1010933404324 ↗
64. Greedy function approximation: a gradient boosting machine10.1214/aos/1013203451 ↗
65. XGBoost: a scalable tree boosting system
66. Convolutional neural networks for sentence classification
67. Learning to forget: continual prediction with LSTM10.1162/089976600300015015 ↗
68. Note on the sampling error of the difference between correlated proportions or percentages10.1007/BF02295996 ↗
69. The probable error of a mean10.2307/2331554 ↗
70. Wilcox · Modern statistics for the social and behavioral sciences: a practical introduction · 2011 · 862
71. Efron · An introduction to the bootstrap · 1994 · 456
72. Ioffe S. Batch renormalization: towards reducing minibatch dependence in batch-normalized models. arXiv:1702.03275v2. 2017.