2026
doi: 10.12720/jait.17.7.1321-1330
Advancing Machine-generated Text Detection: A Comprehensive Evaluation of Transformer-based Models
- 1Department of Information Systems, SDU University, Kaskelen, Kazakhstan
- 2Department of General Education, New Uzbekistan University, Tashkent, Uzbekistan
- Manuscript receivedFebruary 24, 2025
- revisedMarch 31, 2026
- acceptedApril 14, 2026
- publishedJuly 23, 2026
Abstract
Improved fluency in large language models has intensified the need for accurate detection of machine-generated text. This study evaluates transformer-based models using an improved version of the Conference on Computational Linguistics 2025 (COLING 2025), Generative Artificial Intelligence (GenAI) Content Detection Task 1 dataset, which was carefully preprocessed to enhance label quality and balance. All models were trained under a unified protocol to ensure fair comparison and robust evaluation. Test set results show that Decoding-Enhanced Bert with Disentangled Attention (DeBERTa) achieves the highest macro F1−Score of 85.48%, surpassing the previously top-ranked Multi-Task Learning (MTL) system, which attains a macro F1 of 83.07%. These results highlight the effectiveness of advanced transformer architectures for distinguishing human-written and machine-generated text. Despite these gains, performance degradation under domain shift and highly paraphrased inputs remains a challenge.
Keywords
- text classification
- Artificial Intelligence (AI)
- AI-generated content
- transformers
- Decoding-Enhanced Bert with Disentangled Attention (DeBERTa)
- Artificial Intelligence (AI) detection
- Bidirectional Long Short-Term Memory (BiLSTM)
How to Cite
Batyr Sharimbayev and Shirali Kadyrov, "Advancing Machine-generated Text Detection: A Comprehensive Evaluation of Transformer-based Models," Journal of Advances in Information Technology, vol. 17, no. 7, pp. 1321-1330, 2026. https://doi.org/10.12720/jait.17.7.1321-1330
Copyright & License
Copyright © 2026 by the authors. This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).






