Summary of Pre-trained Model Recommendation For Downstream Fine-tuning, by Jiameng Bai et al.
Pre-Trained Model Recommendation for Downstream Fine-tuning
by Jiameng Bai, Sai Wu, Jie Song, Junbo Zhao, Gang Chen
First submitted to arxiv on: 11 Mar 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper presents a pragmatic framework called Fennec for model selection in transfer learning. It addresses existing limitations by considering nuanced relationships between models and tasks, allowing it to rank off-the-shelf pre-trained models and select the most suitable one for a new target task. The key insight is to map all models and historical tasks into a transfer-related subspace, where the distance between model vectors and task vectors represents the magnitude of transferability. This approach circumvents the computational burden of extensive forward passes by using a large vision model as a proxy to infer a new task’s representation in the transfer space. The paper also investigates the impact of inherent inductive bias on transfer results and proposes a novel method called archi2vec to encode intricate structures of models. The transfer score is computed through straightforward vector arithmetic with a time complexity of O(1). The paper validates its effectiveness on two benchmarks and releases a comprehensive benchmark, making it publicly available. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper helps us choose the best pre-trained model for a new task by understanding how well different models can adapt to that task. It does this by looking at all the existing tasks and models together in a special space that shows how transferable each model is. This saves a lot of computer time because it doesn’t need to test each model on every task. The paper also looks at how the internal workings of each model affect its ability to adapt, and proposes a new way to understand these internal structures. By releasing a benchmark with many models and tasks, this research can help other scientists compare their own work. |
Keywords
* Artificial intelligence * Transfer learning * Transferability
