Summary of Improving Fairness and Mitigating Madness in Generative Models, by Paul Mayer et al.
Improving Fairness and Mitigating MADness in Generative Models
by Paul Mayer, Lorenzo Luzi, Ali Siahkoohi, Don H. Johnson, Richard G. Baraniuk
First submitted to arxiv on: 22 May 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Machine Learning (stat.ML)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Generative models unfairly penalize minority classes, suffer from model autophagy disorder (MADness), and learn biased estimates. Our theoretical and empirical results show that training with intentionally designed hypernetworks leads to fairer generation of minority class datapoints, more stable self-consumed (i.e., MAD) models, and less statistically biased parameter estimation. Additionally, we introduce a regularization term that penalizes discrepancies between estimated weights when trained on real data versus synthetic data. To facilitate training existing deep generative models within our framework, we offer a scalable implementation of hypernetworks. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Generative models have a problem: they unfairly treat minority classes and get stuck in a loop (model autophagy disorder). They also learn incorrect estimates about the world. Our research shows that by using special networks called hypernetworks, we can make generative models fairer, more stable, and less biased. We even created a way to regulate these models so they don’t become too self-absorbed. This helps existing deep learning models be more fair and accurate. |
Keywords
» Artificial intelligence » Deep learning » Regularization » Synthetic data
