Summary of Towards Hyper-parameter-free Federated Learning, by Geetika et al.
Towards Hyper-parameter-free Federated Learning
by Geetika, Drishya Uniyal, Bapi Chatterjee
First submitted to arxiv on: 30 Aug 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Optimization and Control (math.OC)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed automated scaling techniques in federated learning (FL) for global model updates outperform vanilla federated averaging (FedAvg), but existing methods require additional tunable hyperparameters on the server. The authors introduce two algorithms that eliminate the need for these hyperparameters, achieving competitive convergence rates and good empirical performance. The first algorithm uses a descent-ensuring step-size regime at clients to ensure linear convergence for strongly convex federated objectives. The second algorithm replaces the objective function value at the server with the average of objective values from sampled clients, facilitating computation of the scaling factor. Empirical results demonstrate that these methods perform similarly or better than popular FL algorithms for both convex and non-convex problems. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Federated learning is a way to train models on many devices without sharing all their data. The problem is that the updates from each device can be very different, making it hard to combine them correctly. Some methods try to fix this by adjusting the amount of update sent from each device, but these adjustments require careful tuning. This paper proposes two new algorithms that automatically adjust these updates, allowing for better performance without needing manual tweaking. The results show that these algorithms work well for a variety of problems and can be used with different types of models. |
Keywords
» Artificial intelligence » Federated learning » Objective function
