Summary of Stochastic Approximation Approach to Federated Machine Learning, by Srihari P V and Bharath Bhikkaji
Stochastic Approximation Approach to Federated Machine Learning
by Srihari P V, Bharath Bhikkaji
First submitted to arxiv on: 20 Feb 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper explores Federated learning (FL) in a Stochastic Approximation (SA) framework, allowing for collaborative neural network training across multiple clients without centralizing their data. FL involves each client training a model on their respective data and sending weights to a server for aggregation, which is then used by clients to re-initialize their models. SA uses approximate sample gradients and tapering step size to locate the cost function minimizer. In this paper, clients use stochastic approximation iterates to update neural network weights. The results show that aggregated weights track an autonomous ODE. Numerical simulations compare the proposed algorithm with standard algorithms like FedAvg and FedProx, demonstrating its robustness and reliable estimates of weights, especially when client data are not identically distributed. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Federated learning lets many devices work together to train AI models without sharing their own data. This paper shows how this process can be improved using a special algorithm called stochastic approximation. Imagine you’re trying to find the minimum value of a big hill by taking small steps and adjusting your direction slightly each time. That’s basically what these algorithms do, but with many devices working together instead of just one person. The results show that this approach is more reliable and works well even when the data from different devices are very different. |
Keywords
* Artificial intelligence * Federated learning * Neural network
