Real Time Voice Phishing Detection System with ML&NLP

Abstract

Voice phishing (vishing) has emerged as a critical cybersecurity threat, exploiting human trust through deceptive voice communications to extract sensitive information. With the rapid growth of voice-based services and telecommunication technologies, traditional rule-based detection mechanisms are increasingly inadequate against evolving social engineering attacks. This paper presents an intelligent voice phishing detection system that leverages Natural Language Processing (NLP) and Machine Learning (ML) techniques to automatically identify fraudulent voice interactions. The proposed system converts voice recordings into textual transcripts using speech-to-text processing, followed by comprehensive NLP-based feature extraction, including lexical, syntactic, and semantic features. These features are then analyzed using supervised machine learning classifiers such as Support Vector Machines, Random Forest, and Logistic Regression to distinguish between legitimate and phishing calls. Experimental evaluation on labeled voice datasets demonstrates that the proposed approach achieves high detection accuracy and robustness against diverse phishing strategies. The system offers a scalable and real-time solution for enhancing call security, making it suitable for deployment in telecommunication networks and voice-enabled platforms.