Real-Time Sign Language Recognition Using Advanced Computer Vision

Abstract

Sign language interpretation tools are effective tools for the deaf or hard of hearing to communicate. The proposed research provides an effective solution to such getting over communication and accessibility barriers in dynamic sign language recognition for people who are both hearing and speaking impaired. Leveraging advanced machine learning techniques, including Random Forest Classifiers, and the Google MediaPipe framework, we propose a robust system capable of interpreting hand gestures into text. Our methodology encompasses data collection under diverse conditions to ensure robustness, preprocessing with geometric feature engineering for enhanced accuracy, and model training on 26 English alphabets. MediaPipe Hand landmark detection provides 21 key points per hand, enabling precise spatial feature extraction, while data augmentation improves adaptability to real-world variations. The system achieved a high classification accuracy of 90.00%, attributed to the ensemble learning capabilities of the Random Forest model. Real-time inference is supported by a seamless integration of MediaPipe Hands for gesture detection and temporal smoothing for stability in dynamic scenarios. Deployed on the Streamlit platform, the user-friendly interface allows for data collection, training, live predictions, constructing words and sentences dynamically based on recognized gestures. This project demonstrates significant potential for applications in education, healthcare, and public services, enhancing accessibility for the hearing impaired. Demonstrating inclusivity through technology establishes a basis for future innovations, such as understanding complete phrases or continuous sign language, further narrowing communication gaps between auditory and non-auditory communities.