Real-Time Sign Language Translator

Abstract

Communication between hearing and speech-impaired individuals and the general population remains a significant challenge in modern society. Sign language serves as the primary communication medium for the deaf and mute community, yet the vast majority of people cannot understand it. This paper presents a Real-Time Sign Language Translator Glove designed to bridge this communication gap by converting hand gestures into text and audible speech output. The proposed system integrates flex sensors and an MPU6050 Inertial Measurement Unit (IMU) into a wearable smart glove, with data acquisition handled by an ESP32 microcontroller and gesture recognition executed on a Raspberry Pi Zero 2 W using Python-based algorithms and an offline text-to-speech engine. Sensor fusion combining flex and IMU data improves recognition accuracy for both static gestures such as alphabets and dynamic gestures such as common words and phrases. The system operates entirely offline, requires no internet connectivity, and is designed to be fully portable. Wireless communication via ESP32 Bluetooth enables dual-glove operation for richer gesture vocabularies. Simulation and expected implementation results confirm reliable gesture detection, real-time speech output, and practical suitability for deployment in hospitals, educational institutions, and public environments.