Pembuatan Sistem untuk Pengumpulan Dataset dan Inferensi Bahasa Isyarat Indonesia (BISINDO) Berbasis Computer vision pada Aplikasi Video call ElCue

Abstract

Computer vision-based sign language recognition plays a crucial role in improving communication accessibility for the Deaf community. This study develops a system that automates the collection of the Indonesian Sign Language (BISINDO) dataset and performs real-time hand gesture inference using OpenCV and TensorFlow. The system enables the device's camera to capture users' hand gestures, process images using image processing techniques, and store them in a structured dataset for inference purposes. Experimental results show that the system successfully collected 9,448 images, with each BISINDO sign consisting of 1,050 images recorded under various lighting conditions and camera angles. Inference conducted on nine different signs achieved an average accuracy of 86.3%, with certain signs such as A, C, D, Terima Kasih, and Halo exceeding 90% accuracy. However, the I sign had the lowest accuracy at 66.6%, indicating challenges in recognition due to hand movement complexity, variations in camera angles, and lighting conditions affecting inference accuracy. Overall, the developed system has demonstrated good performance in automating BISINDO dataset collection and performing real-time inference. However, further improvements are required to enhance the system's reliability in recognizing signs under diverse real-world conditions. Future development may involve expanding dataset variations, applying data augmentation techniques, and optimizing the deep learning architecture used in the system.