Summary of Robust Model Aggregation For Heterogeneous Federated Learning: Analysis and Optimizations, by Yumeng Shao et al.
Robust Model Aggregation for Heterogeneous Federated Learning: Analysis and Optimizations
by Yumeng Shao, Jun Li, Long Shi, Kang Wei, Ming Ding, Qianmu Li, Zengxiang Li, Wen Chen, Shi Jin
First submitted to arxiv on: 11 May 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Distributed, Parallel, and Cluster Computing (cs.DC)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed time-driven synchronous federated learning (T-SFL) framework aims to address performance degradation issues in heterogeneous systems. By aggregating models from different clients at regular time intervals, T-SFL seeks to recover the loss incurred by asynchronous aggregation in conventional asynchronous FL (AFL) and semi-asynchronous FL frameworks. To evaluate T-SFL’s performance, an upper bound on the global loss function is developed, and optimization of aggregation weights is explored to minimize this bound. A discriminative model selection (DMS) algorithm is also introduced to ensure accurate aggregation weights by removing local models from clients with low iteration counts. Experimental results on popular datasets such as MNIST, Cifar-10, Fashion-MNIST, and SVHN demonstrate that T-SFL with DMS can reduce latency by 50% while achieving an average 3% improvement in learning accuracy over state-of-the-art AFL algorithms. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary T-SFL is a new way to do federated learning. Right now, when we do learning on different devices, it’s hard because the data and computers are different. This makes the learning slow and not very good. The T-SFL idea is to group all the learning together at regular times, like clocking in every hour. This helps make the learning better and faster. To make sure this works well, we developed a way to choose which devices get to contribute more or less to the overall learning. We tested this on different kinds of data and it worked really well! |
Keywords
» Artificial intelligence » Discriminative model » Federated learning » Loss function » Optimization
