Perancangan dan Implementasi Aplikasi Video Call ElCue Berbasis Framework Flutter terintegrasi Firebase dan Agora dengan Pendeteksian Bahasa Isyarat Indonesia (BISINDO) secara Real-time

Abstract

Communication is a fundamental aspect of human life, but for the Deaf community, social interaction is often hampered due to lack of accessibility in communication technology, especially in the use of video calls. This research aims to design and implement a Flutter-based video call application that can detect Indonesian Sign Language (BISINDO) in real-time and convert voice to text (speech-to-text), with Firebase integration for data management and Agora SDK for video communication. This application is expected to facilitate more inclusive communication between Deaf and non-Deaf people. A Convolutional Neural Network (CNN)-based deep learning model is used to detect hand gestures in BISINDO, while the Flutter speech-to-text plugin is applied for real-time voice transcription. The implementation of the application is done by integrating the technology into a cross-platform mobile platform. System testing was conducted through functionality testing. The test results show that the ElCue application can run well, providing smoother and more inclusive communication for Deaf users, with accurate BISINDO detection and speech-to-text.