Summary of Communication and Energy Efficient Federated Learning Using Zero-order Optimization Technique, by Elissa Mhanna and Mohamad Assaad
Communication and Energy Efficient Federated Learning using Zero-Order Optimization Technique
by Elissa Mhanna, Mohamad Assaad
First submitted to arxiv on: 24 Sep 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Distributed, Parallel, and Cluster Computing (cs.DC)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper proposes a zero-order optimization method for federated learning (FL) that addresses the communication bottleneck issue in the upload direction. The approach requires devices to upload a single quantized scalar per iteration instead of the entire gradient vector, significantly reducing energy consumption and communication overhead. The authors provide theoretical convergence proof and an upper bound on the convergence rate in non-convex settings. They also discuss implementation scenarios, including quantization and packet dropping due to wireless errors. Compared to standard gradient-based FL methods, this approach shows superior performance. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about a new way for devices to work together to train a model without sharing their private data. The problem is that it takes a lot of energy and communication resources to send all the information needed for training. To solve this, researchers came up with an idea where each device only sends a small number (one value per step) instead of the whole set of numbers. They proved that this works and even showed how it’s better than other methods. This could help make it easier for devices to work together without using too much energy or bandwidth. |
Keywords
» Artificial intelligence » Federated learning » Optimization » Quantization
