Summary of Flens: Federated Learning with Enhanced Nesterov-newton Sketch, by Sunny Gupta et al.
FLeNS: Federated Learning with Enhanced Nesterov-Newton Sketch
by Sunny Gupta, Mohit Jindal, Pankhi Kashyap, Pranav Jeevan, Amit Sethi
First submitted to arxiv on: 23 Sep 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Computer Vision and Pattern Recognition (cs.CV); Distributed, Parallel, and Cluster Computing (cs.DC); Optimization and Control (math.OC)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary In this paper, researchers introduce Federated Learning with Enhanced Nesterov-Newton Sketch (FLeNS), a novel method that balances communication efficiency and rapid convergence in federated learning. FLeNS combines the acceleration capabilities of Nesterov’s method with the dimensionality reduction benefits of Hessian sketching to approximate centralized Newton’s method without relying on exact Hessians, reducing communication overhead. The paper theoretically analyzes FLeNS’s super-linear convergence rates and provides rigorous guarantees for its performance. Experimental evaluation validates these findings, showcasing FLeNS’s state-of-the-art performance with reduced communication requirements in privacy-sensitive and edge-computing scenarios. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Federated learning is a way to train machine learning models on many devices without sharing their data. But it can be slow because each device only sends small updates. Researchers developed a new method called FLeNS that makes training faster by reducing the amount of information that needs to be shared between devices. This is important for privacy and edge computing, where devices are far from the cloud. The paper shows that FLeNS works well in practice and can train models even faster than before. |
Keywords
» Artificial intelligence » Dimensionality reduction » Federated learning » Machine learning
