An Efficient Deep Learning Framework for Photorealistic Fake Visual Media Detection

Abstract

This paper discusses recent advancements made in the area of generative learning that enable the creation of photorealistic fake visual media (often known as deepfakes) from artificially created data. Due to the benefits associated with these recent technologies, as well as their ability to create plausible, deceptive digital images and videos, they introduce significant challenges related to the authentication of both real and fake digital media content. The use of technological tools such as Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs) for synthesizing deepfakes imposes another set of challenges when conducting forensic analysis to determine whether a digital asset is authentic or not.  In this paper, we present a system based on deep learning that will allow the identification of manipulated visual media content (fake visual content) in both still image files and video file sequences. The proposed solution leverages the ResNet50 architecture as the underlying classifier. The initial step in the image analysis pipeline is a pre-trained face detection algorithm that will allow us to detect and extract facial regions from the input content so that we can focus our analysis on portions of the image or video content that are most likely to have been manipulated by an attacker. The facial images extracted from the input data will be processed through the ResNet50 network to learn representative spatial features of the manipulated images that represent visual characteristics such as unusual texture patterns, inconsistent lighting, and structural distortions created during the synthesis process. When evaluating the texture, lighting, and structural characteristics of video clips, multiple frames will be independently evaluated for authenticity, and the final decision regarding the authenticity of a video will reflect the aggregate of all frame evaluations, The proposed methodology has been tested and evaluated on both authentic and artificially created datasets of images and video files. The results of the experimental investigation are presented