Bridging Silence: ISL Recognition and Speech Generation Using AI

Abstract

Indian Sign Language (ISL) is a structured visual-manual language used by more than 18 million deaf and speech-impaired individuals in India. However, limited technological support and accessible digital tools often create communication barriers between deaf and speech-impaired individuals and the general public. This paper presents SaigaiOli, a real-time web-based Indian Sign Language recognition system that translates static hand gestures into text and speech using deep learning and computer vision techniques. The proposed system recognizes 36 ISL gesture classes, including 26 alphabets (A-Z) and 10 numerical signs (0-9). A Kaggle-sourced dataset containing approximately 2,000 images per class was used for training, resulting in a total of about 72,000 images collected from signers of different age groups. Hand landmarks are extracted using the Media-Pipe Hands framework, producing 84 normalized features per gesture. These features are classified using a sequential deep neural network implemented with Tensor-Flow. The model achieved an overall accuracy of 96.75%, with macro-averaged precision, recall, and F1-score of 0.97. The system is deployed as a React.js web application, enabling real-time gesture recognition through a webcam, text accumulation, text-to-speech output, and an integrated ISL learning module. The proposed system promotes accessible communication and supports greater inclusion for deaf and speech-impaired individuals.