Summary of Towards Understanding the Effect Of Pretraining Label Granularity, by Guan Zhe Hong et al.
Towards Understanding the Effect of Pretraining Label Granularity
by Guan Zhe Hong, Yin Cui, Ariel Fuxman, Stanley H. Chan, Enming Luo
First submitted to arxiv on: 29 Mar 2023
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper investigates how different granularities of pre-training labels impact the generalization performance of deep neural networks in image classification tasks. Specifically, it examines the “fine-to-coarse” transfer learning setting where the pre-training label space is more fine-grained than the target problem’s. The authors empirically demonstrate that pre-training on the leaf labels of ImageNet21k yields better transfer results on ImageNet1k compared to coarser granularity levels. They also provide theoretical explanations, showing that fine-grained pre-training enables networks to learn both common and rare features, leading to improved accuracy on challenging test samples. The study further highlights the importance of label hierarchies in pre-training datasets and alignment between pre-training and target labels. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This research paper looks at how different levels of detail in training labels affect how well artificial intelligence (AI) models can be used to identify images. They found that when the training labels are more detailed, AI models perform better on a test image set. The researchers also explained why this is happening and showed that when AI models learn both common and rare features, they do better at identifying tricky test images. To make transfer learning work well in practice, it’s important to have a meaningful label hierarchy in the training data and for the training and testing labels to match up. |
Keywords
* Artificial intelligence * Alignment * Generalization * Image classification * Transfer learning
