Summary of Global Update Tracking: a Decentralized Learning Algorithm For Heterogeneous Data, by Sai Aparna Aketi et al.
Global Update Tracking: A Decentralized Learning Algorithm for Heterogeneous Data
by Sai Aparna Aketi, Abolfazl Hashemi, Kaushik Roy
First submitted to arxiv on: 8 May 2023
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Multiagent Systems (cs.MA)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper proposes Global Update Tracking (GUT), a novel tracking-based method to mitigate the impact of heterogeneous data distribution across devices in decentralized deep learning. The algorithm aims to improve model performance without introducing additional communication overhead. The authors demonstrate its effectiveness on various Computer Vision datasets and architectures, achieving state-of-the-art performance with a 1-6% improvement in test accuracy compared to existing techniques. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Decentralized learning lets computers train models together without a central server. But when the data is different at each location, model performance can suffer. This paper develops a new way to train models that works well even when the data is very different. It’s called Global Update Tracking (GUT) and it doesn’t require extra communication between devices. The authors test GUT on various computer vision tasks and show that it performs better than other methods. |
Keywords
* Artificial intelligence * Deep learning * Tracking
