Summary of Selective Knowledge Sharing For Personalized Federated Learning Under Capacity Heterogeneity, by Zheng Wang et al.
Selective Knowledge Sharing for Personalized Federated Learning Under Capacity Heterogeneity
by Zheng Wang, Zheng Wang, Zhaopeng Peng, Zihui Wang, Cheng Wang
First submitted to arxiv on: 31 May 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Distributed, Parallel, and Cluster Computing (cs.DC)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed Federated Learning (FL) framework, Pa3dFL, addresses the challenges of personalizing capacity-heterogeneous models by introducing a novel approach that decouples and selectively shares knowledge among models. The framework decomposes each layer into general and personal parameters, allowing for uniform sizes across clients and aggregating them through direct averaging. Learnable embeddings are used to generate size-varying personal parameters for clients, which are then aggregated through self-attention modules. Experimental results on three datasets show that Pa3dFL outperforms baseline methods across various heterogeneity settings, with competitive communication and computation efficiency. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Pa3dFL is a new way of learning from different devices and data together. Right now, this process isn’t very good at using the power of small devices. The problem is that these devices have different abilities to learn and store information. Pa3dFL helps by separating the important knowledge into two parts: general and personal. This allows for a fair comparison between devices with different capacities. The framework uses something called embeddings, which are like a set of coordinates that help devices find the right balance. By using these embeddings, Pa3dFL can learn from each device’s unique abilities and strengths. The results show that this new approach works better than current methods and is more efficient. |
Keywords
» Artificial intelligence » Federated learning » Self attention
