Summary of Dfrd: Data-free Robustness Distillation For Heterogeneous Federated Learning, by Kangyang Luo et al.
DFRD: Data-Free Robustness Distillation for Heterogeneous Federated Learning
by Kangyang Luo, Shuai Wang, Yexuan Fu, Xiang Li, Yunshi Lan, Ming Gao
First submitted to arxiv on: 24 Sep 2023
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This research paper proposes a new Federated Learning (FL) method called DFRD, which tackles the challenges of learning robust global models in heterogeneous FL scenarios. By resorting to data-free knowledge distillation, DFRD uses a conditional generator on the server to approximate the training space of local models uploaded by clients. The method also incorporates techniques like exponential moving averages and dynamic weighting/sampling to maintain model fidelity and accuracy. Experimental results on various image classification tasks demonstrate that DFRD outperforms state-of-the-art baselines. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Federated Learning helps computers learn together without sharing private data. But it’s hard to get all the computers to agree on a good answer when their data is different. Researchers came up with a new way to do this called DFRD. It uses a special kind of generator that tries to understand how each computer’s data works, and then shares that knowledge with other computers. To keep things consistent, it also keeps track of changes in the generator over time. The team tested DFRD on lots of image classification tasks and found that it did better than previous methods. |
Keywords
* Artificial intelligence * Federated learning * Image classification * Knowledge distillation
