IDNet: An End-to-End Identity Document Fraud Detection System Using Transfer Learning and Bayesian Hyperparameter Optimization

Abstract

Identity document fraud poses a significant and growing threat to financial institutions, government agencies, and digital identity verification platforms worldwide. Traditional manual inspection methods are slow, error-prone, and unable to scale to the volume of documents processed daily. This paper presents IDNet, an end-to-end identity card fraud detection system that leverages a Convolutional Neural Network (CNN) pipeline built with PyTorch to automatically detect visual fraud indicators in identity documents. The system employs a transfer learning approach using a ResNet50 architecture pretrained on ImageNet, fine-tuned for binary classification of fraudulent versus non-fraudulent documents. The pipeline encompasses automated data preparation from base64-encoded CSV sources, Bayesian hyperparameter optimization using Optuna, model training with class-balanced loss weighting and aggressive data augmentation, and deployment via the Hugging Face Hub for scalable inference. The methodology addresses two critical fraud patterns—inpainting-and-rewrite and crop-and-replace tampering—using stratified train/test/out-of-sample splits to ensure rigorous evaluation. Optimized hyperparameters were obtained through a structured search across learning rate, weight decay, batch size, and learning rate scheduler configurations. The system targets classification accuracy exceeding 95% with inference latency below 2 seconds per document. The full project scope further integrates PostgresML for in-database inference querying, enabling seamless deployment within existing data infrastructure. This work demonstrates that transfer learning combined with systematic hyperparameter optimization provides a practical, reproducible, and deployable solution for automated identity document fraud detection.