Abstract—Oral cancer deals with the cancerous lesions that appear in the mouth, lips, tongue and cheeks and poses a significant health challenge, and its detection in early stages is pivotal for better patient outcomes. Latest Deep Learning Techniques have provided a lot of opportunities for automatic detection and classification of oral cancers with the accuracy better than that of human experts by providing a non-invasive and cost-effective method to detect oral cancer at early-stages, thus enabling early treatment. Convolutional Neural Networks (CNNs) have shown immense potential in the field of medical image analysis, including the automatic detection of oral cancer from histopathological and photographic images. However, optimizing CNNs for this specific task remains intricate and time-consuming due to the numerous hyper parameters involved. The paper proposes a novel approach to optimize CNN hyper parameters that affects the performance and speed of CNN architecture for oral cancer detection using photographic images. The method involves applying Genetic Algorithm (GA) to efficiently tune the hyper parameter space and enhance the CNN’s performance. The experimental findings reveal that the GA-optimized CNN outperforms the base CNN model, achieving higher accuracy, sensitivity, and specificity. The proposed approach was able to achieve an accuracy of 95% after GA optimization on DenseNet201 model compared to the performance before optimization. Thus, the trained CNN model holds promising prospects for ensuing early oral cancer detection and subsequently improving patient prognosis and survival rates.
 
Keywords—deep learning techniques, oral cancer, Convolutional Neural Network (CNN), Genetic Algorithm (GA), medical image analysis


Copyright © 2025 by the authors. This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).