Implementasi Hand Gesture Recognition Secara Real-Time Menggunakan Algoritma Convolutional Neural Network Untuk Menggerakkan Robot Strandbeest

Abstract

Technological developments in the field of artificial intelligence and robotics have enabled more natural human-machine interactions, one of which is through hand gesture recognition systems. This study aims to implement Convolutional Neural Network (CNN) in a hand gesture recognition system to control the movement instructions of a Strandbeest robot in real-time. In this study, the system was trained to recognize several types of hand gestures with the types of instructions being forward, backward, turning left, turning right, and stopping. A dataset of hand gesture images was collected and processed through a pre-processing stage consisting of hand landmarking, cropping, and resizing. Then, a model with CNN architecture will be trained using the dataset and produce a hand gesture recognition system. The results of hand gesture recognition as a type of instruction will be sent to the Strandbeest robot via Wi-Fi connectivity and an ESP32 microcontroller as the receiver. Then the microcontroller sends instructions to the L298N motor driver to move the Strandbeest robot according to the hand gestures recognized by the training system. The test results show that the CNN model is able to recognize hand gestures with an accuracy of 92% in real-time with an average response time of less than 100 milliseconds. This research proves that CNN can be implemented as a real-time hand gesture recognition algorithm to provide a robotics control system, especially the Strandbeest robot.