Abstract

The automated classification of mental health disorders from social media text holds significant promise for scalable early screening and clinical decision-making. However, progress in this domain remains constrained by the lack of clinically annotated corpora for training purposes. Existing datasets predominantly rely on crowdsourced, automated, or semi-supervised annotation strategies that lack the clinical credibility required for the reliable classification of disorders. We present International Classification of Diseases, 11th revision (MANAS-ICD11), a Reddit-based benchmark corpus of 3005 posts exclusively annotated by medical professionals, comprising one psychiatrist and two clinical psychologists, across five diagnostic categories aligned with the ICD-11. A novel three-task annotation design capturing disorder-specific keywords, clinical characteristics, and diagnostic labels achieved a strong Inter-annotator Agreement (IAA) with a Krippendorff’s alpha value of 0.80. Six Artificial Intelligence (AI) models spanning Machine Learning (ML), Deep Learning (DL), and Transfer Learning (TL) architectures were benchmarked, with the Support Vector Machine (SVM) achieving the strongest performance on the held-out test set at 0.83 accuracy and a weighted F1-Score of 0.81, outperforming all DL and transformer-based models, including MentalBERT. Cross-validation additionally showed that Logistic Regression (LR) was a statistically more consistent performer across folds. The dominance of Term Frequency-Inverse Document Frequency (TF-IDF)-based classifiers highlights the discriminative richness of ICD-11 aligned clinical lexical patterns in the corpus. MANAS-ICD11 provides a clinically grounded and reproducible foundation for automated mental health screening with direct applicability in clinical and public health settings.