Dual-Prompt Text–Image Matching Framework Using CLIP for Real-Time Authenticity Detection

Abstract

The increasing reliance on digital media for real-time information sharing has intensified the spread of misleading incident reports and visually manipulated content, creating challenges for timely and reliable incident verification. Conventional fact-checking approaches rely on static or pre- curated datasets, which limits their ability to verify emerging or previously unseen incidents in real time. This paper proposes a novel Dual-Prompt Text–Image Incident Verification System (DP-TIVS) that authenticates incident reports through multimodal analysis. DP-TIVS operates in two complementary modes: Text-to-Image verification, where textual incident descriptions are semantically matched with internet-retrieved images, and Image-to-Text verification, where uploaded images are automatically captioned and cross-validated against online visual content. The system integrates Named Entity Recognition using spaCy, real-time image retrieval via the Bing Image Search API, and vision–language alignment using CLIP and BLIP-2 models to compute cross-modal similarity scores. Experimental evaluation shows that DP-TIVS achieves an average authenticity detection accuracy of 89.3%, outperforming baseline methods by 15.7%, and generates structured verification reports containing similarity metrics, supporting evidence, and actionable insights, effectively addressing challenges such as multimodal semantic alignment and scalability for real-world deployment.