Summary of Federated Bayesian Deep Learning: the Application Of Statistical Aggregation Methods to Bayesian Models, by John Fischer et al.
Federated Bayesian Deep Learning: The Application of Statistical Aggregation Methods to Bayesian Models
by John Fischer, Marko Orescanin, Justin Loomis, Patrick McClure
First submitted to arxiv on: 22 Mar 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Machine Learning (stat.ML)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary In this paper, researchers propose novel aggregation strategies for Bayesian deep learning (DL) models in federated learning (FL) settings. They demonstrate that existing aggregation methods for deterministic DL models are not suitable for Bayesian models due to the probabilistic nature of their weights and biases. The authors analyze six different aggregation strategies on the CIFAR-10 dataset using a fully variational ResNet-20 architecture, highlighting the importance of selecting an appropriate aggregation strategy in Bayesian FL systems. Additionally, they explore a lightweight alternative approach that applies traditional federated averaging to approximate Bayesian Monte Carlo dropout models. The paper shows that the chosen aggregation strategy significantly impacts accuracy, calibration, uncertainty quantification, training stability, and client compute requirements. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Federated learning helps keep data private by using many computers to train one model. This makes it useful for tasks like recognizing objects in pictures or predicting weather patterns. Some models can tell how sure they are of their answers, which is helpful for important decisions. However, these models have trouble working together because of the way they were trained. The researchers in this paper tried different ways to combine the models’ ideas without sharing all their data. They found that choosing the right method makes a big difference in how well the model works and how sure it is of its answers. |
Keywords
* Artificial intelligence * Deep learning * Dropout * Federated learning * Resnet
