Study and Overview of Multilingual Voice & Text to Sign Language Translator

Abstract

This paper introduces a new approach to improving communication for deaf and hard-of-hearing individuals by developing a multilingual voice and text–to–sign language translator. Unlike existing tools that often lack support for multiple languages, struggle with real-time accuracy, or only handle text or voice separately, our solution combines both voice and text inputs in one system. The framework uses advanced AI technologies, including speech recognition (ASR), natural language processing (NLP) for context-aware understanding, and deep learning to generate realistic sign language. It is designed to support different sign languages (such as ASL and ISL) through specialized datasets, while addressing common challenges like limited vocabularies, robotic or unnatural avatars, and weak handling of continuous signing. The results show that this integrated AI model can create a more natural and inclusive communication experience, with the potential to promote social inclusion, expand educational opportunities, and bring economic benefits.