Enhancing Communication Methods for Individuals with Physical Disabilities

Abstract

Communication is a basic human need. However, people with physical disabilities, especially those who are deaf, mute, or both, face significant challenges in sharing and understanding information. Current options like sign language, lip-reading, and text-based communication either lack universal application or need expensive hardware, like sensor-equipped gloves. Additionally, many existing systems only work one way and do not support Indian Sign Language (ISL), which is essential in India. This paper introduces a real-time communication platform powered by AI to address these issues. The system uses Convolutional Neural Networks (CNNs) to recognize gestures. It also uses Natural Language Processing (NLP) for understanding spoken and written text, Automatic Speech Recognition (ASR) for voice input, and text-to-speech technology. It provides five communication methods: sign-to-text/speech, speech-to-ISL animation, speech-to-text, text-to-speech, and sign-to-text for mute users. The platform is built with open-source tools like MediaPipe, OpenCV, and Google TTS. It only requires a standard webcam and microphone, making it affordable and accessible. The system aims to enhance inclusivity in education, healthcare, workplaces, and public services by allowing real-time, two-way, and cost-effective communication.