Summary of Fusefl: One-shot Federated Learning Through the Lens Of Causality with Progressive Model Fusion, by Zhenheng Tang et al.
FuseFL: One-Shot Federated Learning through the Lens of Causality with Progressive Model Fusion
by Zhenheng Tang, Yonggang Zhang, Peijie Dong, Yiu-ming Cheung, Amelie Chi Zhou, Bo Han, Xiaowen Chu
First submitted to arxiv on: 27 Oct 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Distributed, Parallel, and Cluster Computing (cs.DC); Networking and Internet Architecture (cs.NI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper proposes a novel federated learning approach called FuseFL, which aims to improve the performance of One-shot Federated Learning (OFL) while reducing communication costs. The authors identify the “isolation problem” in OFL, where local models fit to spurious correlations due to data heterogeneity, leading to decreased performance compared to traditional FL methods. They introduce a causal perspective and observe that augmenting intermediate features from other clients can alleviate this issue. FuseFL uses a bottom-up manner for feature augmentation, decomposing neural networks into blocks and progressively training and fusing each block without additional communication costs. The approach outperforms existing OFL and ensemble FL methods by a significant margin, while supporting high scalability, heterogeneous model training, and low memory costs. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary FuseFL is a new way to learn from lots of devices or computers (called clients) at the same time. This helps make sure everyone gets accurate results without sharing all their data. The problem with this approach is that each client’s information might not be reliable. To fix this, FuseFL takes small parts of what each client learned and combines them into better information. This makes it more accurate and efficient than other ways to do this. |
Keywords
» Artificial intelligence » Federated learning » One shot
