Summary of Parametric Feature Transfer: One-shot Federated Learning with Foundation Models, by Mahdi Beitollahi et al.
Parametric Feature Transfer: One-shot Federated Learning with Foundation Models
by Mahdi Beitollahi, Alex Bie, Sobhan Hemati, Leo Maxime Brunswic, Xu Li, Xi Chen, Guojun Zhang
First submitted to arxiv on: 2 Feb 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper presents a new approach to one-shot federated learning (FL) called FedPFT, which enhances both the accuracy and communication efficiency of FL models. The methodology involves transferring parametric feature models extracted from foundation models to each client, which are then used to generate synthetic features for training a classifier head. Experimental results on eight datasets show that FedPFT improves the communication-accuracy frontier in both centralized and decentralized FL scenarios, with gains of up to 20.6%. Additionally, FedPFT adheres to the data minimization principle of FL and is amenable to formal privacy guarantees via differential privacy. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Imagine a group of people working together on a big project. Each person has some information that they don’t want to share with others. This paper introduces a new way for these people to work together while keeping their information private. It’s called FedPFT, and it helps them make better decisions by using a special kind of model called a foundation model. The approach is more efficient than previous methods and can even help protect the privacy of each person’s information. |
Keywords
* Artificial intelligence * Federated learning * One shot
