Abstract

In recent years, Low-Rank Adaptation (LoRA) has emerged as a highly effective parameter-efficient fine-tuning technique for Transformer-based models. This paper extends the applicability of LoRA to traditional Deep Neural Networks (DNNs) by integrating it into a Federated Learning (FL) framework. The principal aim of this study is to facilitate data privacy protection while simultaneously minimizing weight transfer overhead in heterogeneous environments. To address the challenges of Non-Independent Identically Distribution (Non-IID) data, this study proposes the Adaptive LoRA FL framework, which incorporates feature mapping and category unification mechanisms to harmonize highly heterogeneous datasets. By integrating LoRA fine-tuning with feature mapping and category unification mechanisms, it effectively integrates highly heterogeneous datasets (e.g., CIC-IDS2017, Edge-IIoTset, and TON-IoT datasets). Furthermore, the performance of Adaptive LoRA FL framework is evaluated against state-of-the-art FL methods and demonstrate its generalizability across diverse application domains. Experimental results demonstrate that compared to full-weight aggregation, the Adaptive LoRA FL framework significantly reduces client communication costs while maintaining improved global accuracy. By integrating an early stopping mechanism, the training process terminates once convergence is reached, effectively minimizing redundant computation and communication overhead.