Abstract—Accurate prediction of user personality from social media data represents a challenging problem in applied artificial intelligence due to the heterogeneity of multimodal data and multilingual user behavior. This paper proposes a scalable machine learning framework for multimodal data processing and analysis within real-world information technology environments, integrating textual and visual information for personality trait prediction in practical, IT-driven environments. The proposed system combines multilingual Bidirectional Encoder Representations from Transformers (mBERT) for deep semantic text representation and Distillation with No Labels v2 (DINOv2) for visual feature extraction, followed by a lightweight Transformer-based fusion architecture designed to capture cross-modal interactions while ensuring computational efficiency and deployment scalability. The framework is evaluated on an original dataset comprising 32,679 captions and 79,968 images collected from 1290 Instagram users, annotated using the NEO (Neuroticism, Extraversion, Openness to Experience)-FFI (Five Factor Inventory) personality model. To address variability in user behavior, a gender-aware modeling strategy is introduced, enabling more precise representation of heterogeneous user profiles. Experimental results, assessed using Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE), demonstrate that deep semantic representations significantly outperform surface-level features, while the proposed multimodal fusion model achieves state-of-the-art predictive performance, reaching an RMSE of 0.37 for Agreeableness and 0.52 for Conscientiousness. These results highlight the effectiveness of multimodal machine learning approaches for processing complex real-world data and demonstrate the potential of the proposed system for intelligent decision support, personalized services, and scalable Information Technology (IT)-based applications such as digital mental health and automated recruitment platforms within modern information technology infrastructures.
 
Keywords—multimodal data fusion, transformer-based models, machine learning, social media analytics, personality trait prediction, natural language processing, computer vision
 
Copyright © 2026 by the authors. This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).