Summary of Lw-fedssl: Resource-efficient Layer-wise Federated Self-supervised Learning, by Ye Lin Tun et al.
LW-FedSSL: Resource-efficient Layer-wise Federated Self-supervised Learning
by Ye Lin Tun, Chu Myaet Thwal, Huy Q. Le, Minh N. H. Nguyen, Choong Seon Hong
First submitted to arxiv on: 22 Jan 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper proposes Layer-Wise Federated Self-Supervised Learning (LW-FedSSL), an approach to train complex models like Transformers on edge devices while reducing computational and communication costs. By breaking down training into stages, each focusing on a specific model layer, LW-FedSSL minimizes resource requirements. This results in up to 3.34x reduced memory usage, 4.20x fewer computations (GFLOPs), and 5.07x lower communication costs compared to end-to-end training. The paper also explores Prog-FedSSL, a progressive training strategy offering similar efficiency improvements. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This study develops ways for edge devices to learn from large-scale models like Transformers without using up too many resources. It does this by dividing the learning process into smaller steps that focus on different parts of the model. This makes it more efficient and reduces the need for computing power and data transfer between devices. The approach is useful because it can make complex AI models work on devices with limited resources. |
Keywords
* Artificial intelligence * Self supervised
