Optimasi MobileNetV2 dengan Fine-tuning dan Additional Layers untuk Pendeteksian Bahasa Isyarat Indonesia (BISINDO) secara Real-time dalam Aplikasi Video Call ElCue

Abstract

Advancements in communication technology, such as video calls, have facilitated long-distance interactions. However, individuals with hearing disabilities still face communication barriers due to the limited understanding of Indonesian Sign Language (BISINDO) among public. This study proposes a real-time BISINDO detection system in the ElCue video call application by optimizing MobileNetV2 through fine-tuning techniques and the incorporation of additional layers. The research methodology involves collecting BISINDO dataset consisting of 9 gesture classes, preprocessing through augmentation, edge detection, and normalization, followed by model training utilizing transfer learning and fine-tuning, wherein several final layers of MobileNetV2 are unfrozen to refine the model’s weights. Additionally, the model architecture is extended with Global Average Pooling, Dense Layers, and Dropout to enhance classification accuracy and stability. The model’s performance is evaluated based on accuracy and real-time inference capabilities using a camera. The results indicate that the optimized model achieved 97.1% accuracy on the test dataset and 88.9% in real-time testing, proving that the optimized MobileNetV2 can serve as an effective solution for real-time BISINDO detection.