Summary of Back to the Basics on Predicting Transfer Performance, by Levy Chaves et al.
Back to the Basics on Predicting Transfer Performance
by Levy Chaves, Eduardo Valle, Alceu Bissoto, Sandra Avila
First submitted to arxiv on: 30 May 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Computer Vision and Pattern Recognition (cs.CV)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary In the rapidly expanding realm of deep learning, choosing the most effective pre-trained models from an increasingly vast array of options is a pressing concern. Recent proliferation of transferability scorers has led to a new challenge: assessing their performance. This work addresses this issue by proposing robust benchmark guidelines for evaluating these scores and a novel technique for combining multiple scorers, which consistently enhances their results. A comprehensive evaluation across 11 datasets, including generalist, fine-grained, and medical imaging datasets, reveals that few scorers match the predictive power of the straightforward raw metric based on ImageNet, while all predictors exhibit limitations on medical datasets. Our findings underscore the potential benefits of integrating diverse information sources for reliably predicting transferability across various domains. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Imagine trying to pick the best tool from a huge collection of pre-trained models in deep learning. This is getting harder because there are so many options now! To make it easier, we’re proposing new ways to evaluate these tools and combine them to get better results. We tested 13 different evaluation methods on 11 different datasets, including some focused on medical images. Our results show that most of these methods aren’t as good as a simple method called “ImageNet”. But they all struggle when trying to predict how well models will work on medical images. By combining different information sources, we might be able to get more accurate predictions about which model is best for the job. |
Keywords
» Artificial intelligence » Deep learning » Transferability
