Summary of For Better or For Worse? Learning Minimum Variance Features with Label Augmentation, by Muthu Chidambaram and Rong Ge
For Better or For Worse? Learning Minimum Variance Features With Label Augmentation
by Muthu Chidambaram, Rong Ge
First submitted to arxiv on: 10 Feb 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Computer Vision and Pattern Recognition (cs.CV)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This research paper explores the role of label augmentation in deep learning models trained using data augmentation techniques like label smoothing and Mixup. The authors prove that linear models trained with label augmentation learn only minimum-variance features, while standard training can learn higher-variance features. They also show that the losses of label smoothing and Mixup are lower-bounded by a function of the model output variance. Empirical experiments demonstrate that label smoothing and Mixup can be both beneficial (learning low-variance hidden representations) and detrimental (susceptible to spurious correlations). |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about how we train deep learning models using special techniques called data augmentation. It’s like adding new information to the pictures or words we use to teach the model, so it becomes better at recognizing things. The researchers found that when we modify not just the picture or word but also its correct answer, this helps the model learn simple things first and then more complex ones. But sometimes this can be a problem if the data is not good quality. They did some experiments to see how well these techniques work and what they can learn from them. |
Keywords
* Artificial intelligence * Data augmentation * Deep learning
