Summary of The Group Robustness Is in the Details: Revisiting Finetuning Under Spurious Correlations, by Tyler Labonte et al.
The Group Robustness is in the Details: Revisiting Finetuning under Spurious Correlations
by Tyler LaBonte, John C. Hill, Xinchen Zhang, Vidya Muthukumar, Abhishek Kumar
First submitted to arxiv on: 19 Jul 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper investigates the performance of finetuned machine learning models on minority groups, finding that they often rely too heavily on spurious correlations. To address this issue, the authors conduct comprehensive experiments across four benchmarks in vision and language tasks. They first show that commonly used class-balancing techniques can actually decrease worst-group accuracy (WGA) with training epochs, suggesting that these approaches may not be effective. The authors then propose a mixture method that can outperform both techniques in certain scenarios. Additionally, they find that scaling pre-trained models is generally beneficial for WGA, but only when combined with appropriate class-balancing. The paper also identifies spectral imbalance in finetuning features as a potential source of group disparities. Overall, the results highlight more nuanced interactions between modern finetuned models and group robustness than previously understood. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper looks at how well machine learning models work for different groups of people. Right now, many models are good at recognizing patterns that aren’t actually important, which can lead to poor performance on certain groups. The authors did lots of experiments on four big datasets in vision and language tasks to see what happens when they fine-tune their models. They found that some common ways to balance the data don’t work well, and that scaling pre-trained models can help, but only if you do it the right way. The results show that these modern models are more complex than we thought, and that there’s still a lot to learn about making them fairer. |
Keywords
» Artificial intelligence » Machine learning
