Efficient and Scalable Cluster-Based Multi-Model Federated Learning Framework with Knowledge Distillation for Privacy-Preserving Healthcare Systems

Abstract

Federated Learning (FL) enables collaborative model training across distributed healthcare institutions without requiring the exchange of sensitive patient data. However, existing FL systems face challenges such as high communication overhead, data heterogeneity, and limited scalability. Additionally, the reliance on a single global model restricts the ability to effectively process multi-modal healthcare data. This paper proposes a cluster-based multi-model federated learning framework integrated with knowledge distillation. Clients are grouped based on data similarity, and specialized models are assigned to each cluster. Model outputs are fused at the central server, and knowledge distillation is used to compress multiple models into a lightweight global model. The framework is designed within an edge-cloud architecture to support scalable and practical deployment in real-world healthcare systems.