Abstract

Recently, there has been a surge in the number of mental health support apps available to combat stress and improve emotional health. But most mobile applications are based on the processing of sensitive biometric information in the cloud and are self-reported, which compromises privacy, requires users to rely on network connectivity, and reduces user engagement. Moreover, few mobile development frameworks offer real-time emotion recognition on-device with multiple user inputs and personalized emotional support. The study proposed a novel mobile framework called the Feeling Diagnostic Application (FDA) for performing facial emotion recognition and sentiment analysis on text without sharing biometric information to third-party servers. The proposed framework combines a custom-trained Convolutional Neural Network (CNN), which is deployed using TensorFlow Lite for face emotion recognition; a rule-based sentiment analysis engine for text input; a high-risk emotion detector for facial expressions; and an adaptive recommendation engine for providing personalized multimedia content to the user based on the detected emotion. All the inferences are executed locally, thereby enabling offline operation and providing user privacy while minimizing security concerns with cloud-based inference. Experimental evaluation using 250 facial expression samples collected from 50 participants and 50 manually labeled text samples achieved 71.6% facial emotion recognition accuracy, 74.0% text sentiment analysis accuracy, 88.5% high-risk expression detection accuracy, and an average inference latency below 100 ms on a consumer-grade Android smartphone.