Perancangan Sistem Penerjemah Isyarat Bahasa Indonesia (SIBI) Menggunakan Mediapipe dengan Metode Kombinasi Multilayer Perceptron dan Long Short Term Memory

Abstract

The Indonesian Sign Language System (SIBI) is a vital visual communication method, but the implementation of its detection technology is often hampered by the differences in the characteristics of statistical (single-pose) and dynamic (multi-pose) gestures. This study proposes and implements an integrated system to detect and translate both types of SIBI gestures simultaneously. The method used is a combination of deep learning models, namely Multilayer Perceptron (MLP) and Long Short-Term Memory (LSTM). MLP is designed to classify static gestures, namely single-pose gestures, and the initial gesture of a dynamic SIBI movement by processing MediaPipe data extraction from a single frame. When the initial dynamic gesture is validated by the MLP, the system will solve the data extraction from 20 or more consecutive frames before the data is processed by the LSTM to translate the dynamic gesture. The test results show that the MLP model achieves an accuracy of 0.982, while the LSTM model achieves an accuracy of 1.0, confirming the system's capability in providing a comprehensive and highly accurate SIBI translation solution. The output produced by this system is in the form of continuous text from a series of detected gestures.