Summary of Are Compressed Language Models Less Subgroup Robust?, by Leonidas Gee et al.
Are Compressed Language Models Less Subgroup Robust?
by Leonidas Gee, Andrea Zugarini, Novi Quadrianto
First submitted to arxiv on: 26 Mar 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Computation and Language (cs.CL)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A novel study examines the impact of 18 different model compression methods and settings on the subgroup robustness of BERT language models. The researchers investigate how these techniques affect the performance of minority subgroups defined by labels and attributes in a dataset. The findings suggest that worst-group performance is not solely dependent on model size, but also influenced by the compression method used. Interestingly, the study reveals that model compression does not always worsen the performance on minority subgroups. This analysis contributes to the ongoing research into the subgroup robustness of model compression. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This study looks at how making language models smaller can affect their performance for different groups of people defined by labels and attributes in a dataset. The researchers try out 18 different ways to make models smaller and see how they affect the worst-performing group. They find that the size of the model isn’t the only thing that matters – it also depends on which method is used. Also, surprisingly, making models smaller doesn’t always make them worse for minority groups. |
Keywords
* Artificial intelligence * Bert * Model compression
