Summary of How Does Promoting the Minority Fraction Affect Generalization? a Theoretical Study Of the One-hidden-layer Neural Network on Group Imbalance, by Hongkang Li et al.
How does promoting the minority fraction affect generalization? A theoretical study of the one-hidden-layer neural network on group imbalance
by Hongkang Li, Shuai Zhang, Yihua Zhang, Meng Wang, Sijia Liu, Pin-Yu Chen
First submitted to arxiv on: 12 Mar 2024
Categories
- Main: Machine Learning (stat.ML)
- Secondary: Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper presents a theoretical analysis of empirical risk minimization (ERM) for individual groups, addressing the issue of group imbalance where high average accuracy is achieved at the expense of low accuracy in minority groups. The authors formulate the problem using the Gaussian Mixture Model and quantify the impact on sample complexity, convergence rate, and testing performance. Although centered on binary classification with one-hidden-layer neural networks, the paper provides a first-of-its-kind analysis of group-level generalization. Key findings include that moderate variance and zero-mean conditions lead to desirable learning performance, while increasing minority group fraction in training data does not necessarily improve minority group accuracy. These results are validated on synthetic and empirical datasets like CelebA and CIFAR-10. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper helps fix a big problem with how computers learn from data. Right now, when we train a computer to do something, it often gets really good at doing what the most common type of data looks like, but not so great at recognizing the less common types. This is called “group imbalance.” The authors used a special model to understand why this happens and how to make computers better at learning from all kinds of data. They found that when there’s just the right amount of variation in the data, the computer can learn really quickly and accurately. But they also discovered that even if we use more data that is minority-like, it doesn’t necessarily get better at recognizing those types of data. |
Keywords
* Artificial intelligence * Classification * Generalization * Mixture model
