FedALA: Adaptive Local Aggregation for Personalized Federated Learning

A key challenge in federated learning (FL) is the statistical heterogeneity that impairs the generalization of the global model on each client. To address this, we propose a method Federated learning with Adaptive Local Aggregation (FedALA) by capturing the desired information in the global model for client models in personalized FL. The key component of FedALA is an Adaptive Local Aggregation (ALA) module, which can adaptively aggregate the downloaded global model and local model towards the local objective on each client to initialize the local model before training in each iteration. To evaluate the effectiveness of FedALA, we conduct extensive experiments with five benchmark datasets in computer vision and natural language processing domains. FedALA outperforms eleven state-of-the-art baselines by up to 3.27% in test accuracy. Furthermore, we also apply ALA module to other federated learning methods and achieve up to 24.19% improvement in test accuracy. Code is available at https://github.com/TsingZ0/FedALA.

Authors

• Jianqing Zhang Shanghai Jiao Tong University
• Yang Hua School of Artifcial Intelligence and Computer Science, Jiangnan University, Wuxi, Jiangsu, P.R. China Sino-UK Joint Laboratory on Artificial Intelligence, Ministry of Science and Technology, China International Joint Laboratory on Artificial Intelligence, Ministry of Education, China
• Hao Wang
• Tao Song
• Zhengui Xue Shanghai Jiao Tong University
• Ruhui Ma Shanghai Jiao Tong University
• Haibing Guan Shanghai Jiao Tong University

Keywords

• ML: Distributed Machine Learning & Federated Learning