Summary of On the Byzantine-resilience Of Distillation-based Federated Learning, by Christophe Roux et al.
On the Byzantine-Resilience of Distillation-Based Federated Learning
by Christophe Roux, Max Zimmer, Sebastian Pokutta
First submitted to arxiv on: 19 Feb 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Distributed, Parallel, and Cluster Computing (cs.DC)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Federated Learning (FL) using Knowledge Distillation (KD) has gained attention for its favorable properties regarding privacy, non-i.i.d. data, and communication cost. In this work, researchers investigate the performance of such approaches in the Byzantine setting, where a subset of clients acts adversarially to disrupt the learning process. The study finds that KD-based FL algorithms are resilient but can be influenced by Byzantine clients. Two new Byzantine attacks are introduced, which break existing Byzantine-resilient methods. A novel defence mechanism is proposed to enhance the resilience of KD-based FL algorithms. Finally, a general framework for obfuscating attacks is provided, making them harder to detect and improving their effectiveness. This research contributes to the analysis of Byzantine FL by developing new attacks and defence mechanisms, advancing the robustness of KD-based FL algorithms. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Imagine you’re trying to learn something online with a group of friends, but some of your friends are being naughty and trying to spoil it for everyone else. This is kind of like what happens in “Byzantine” learning, where some parts of the system act unfairly to mess things up. Researchers looked at how some special learning methods, called Federated Learning (FL) with Knowledge Distillation (KD), can handle this problem. They found that these methods are actually pretty good at resisting the naughty behavior, but they can still be influenced by it. The researchers also came up with new ways to make things go wrong and new ways to defend against them. This helps us understand how to make online learning better and more secure. |
Keywords
* Artificial intelligence * Attention * Federated learning * Knowledge distillation * Online learning
