Summary of Overcoming the Challenges Of Batch Normalization in Federated Learning, by Rachid Guerraoui et al.
Overcoming the Challenges of Batch Normalization in Federated Learning
by Rachid Guerraoui, Rafael Pinot, Geovani Rizk, John Stephan, François Taiani
First submitted to arxiv on: 23 May 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary In this paper, researchers introduce Federated BatchNorm (FBN), a novel approach to address challenges faced by batch normalization in federated learning. Specifically, they propose a scheme that ensures consistency with centralized execution, preserving data distribution and providing accurate running statistics. FBN reduces the external covariate shift and matches evaluation performance of centralized settings. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper introduces Federated BatchNorm (FBN), a new way to help deep neural networks work better in federated learning. Currently, batch normalization doesn’t work well when training models on different devices or environments. The researchers came up with FBN to solve this problem. It makes sure that the data distribution remains consistent and accurate statistics are kept, which helps reduce errors and potential attacks. |
Keywords
» Artificial intelligence » Batch normalization » Federated learning
