2025
doi: 10.12720/jait.16.4.545-555
A Comparative Study on Face/Body-Based Lie Detection Using Convolutional Neural Networks
El-Sayed M. El-Alfy1,2,3,*
, , , ,
- 1Information and Computer Science Department, King Fahd University of Petroleum and Minerals, Dhahran, Saudi Arabia
- 2Computer Engineering Department, King Fahd University of Petroleum and Minerals, Dhahran, Saudi Arabia
- 3IRC of Intelligent Secure Systems, King Fahd University of Petroleum and Minerals, Dhahran, Saudi Arabia
- 4SDAIA-KFUPM JRC for Artificial Intelligence, King Fahd University of Petroleum and Minerals, Dhahran, Saudi Arabia
- Manuscript receivedSeptember 24, 2024
- revisedOctober 30, 2024
- acceptedJanuary 28, 2025
- publishedApril 16, 2025
Abstract
Recognizing human emotional and physical states using physiological measurements has several potential interdisciplinary applications and use cases across multiple domains, including detecting deceptive behaviors, employee screening, safety and security, mental health assessment, stress monitoring, and identifying learning difficulties and attention disorder. This paper presents and evaluates a novel solution approach for lie detection using multi-frame video models and convolutional neural networks for video processing. Different architectures have been evaluated and compared for detecting facial and bodily clues to enhance the detection accuracy. The video stream is preprocessed by selecting key frames and applying human face and body extraction techniques as inputs to the feature learning and detection model. In an ablation study, we also evaluated various ways to combine the trained face and body models under four scenarios: combining outputs of the same model architecture for both face and body, combining different model architectures for faces, combining different model architectures for body, and combining all model architectures for face and body. Evaluations using the Real-Life Trials dataset demonstrate the effectiveness of MobileNetV2 trained on full-body images, outperforming other test methods with more than 94% accuracy while significantly reducing computational costs and model size. Moreover, combining facial and bodily cues enhances model accuracy compared to using each modality in isolation.
Keywords
- deception detection
- lie detection
- machine learning
- deep learning
- convolutional neural networks
- physiological measurements
How to Cite
El-Sayed M. El-Alfy, Abdelrahman S. Shbair, Omar A. Al-Oumi, Abdullatif M. Alsaad, and Sadam Al-Azani, "A Comparative Study on Face/Body-Based Lie Detection Using Convolutional Neural Networks," Journal of Advances in Information Technology, vol. 16, no. 4, pp. 545-555, 2025. https://doi.org/10.12720/jait.16.4.545-555
Copyright & License
Copyright © 2025 by the authors. This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).
