Summary of A Structured Regression Approach For Evaluating Model Performance Across Intersectional Subgroups, by Christine Herlihy et al.
A structured regression approach for evaluating model performance across intersectional subgroups
by Christine Herlihy, Kimberly Truong, Alexandra Chouldechova, Miroslav Dudik
First submitted to arxiv on: 26 Jan 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Computers and Society (cs.CY); Applications (stat.AP); Machine Learning (stat.ML)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper proposes a novel structured regression approach to disaggregated evaluation in AI fairness assessment, which enables reliable system performance estimates even for very small intersectional subgroups. By introducing this method, the authors aim to improve the standard stratification-based approach, which is limited by sample size issues when considering multiple subgroups. The proposed approach includes inference strategies for constructing confidence intervals and goodness-of-fit testing to identify fairness-related harms experienced by intersectional groups. Experimental results on publicly available datasets and semi-synthetic data demonstrate the superiority of this method over the standard approach. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary AI researchers want to make sure that artificial intelligence (AI) systems are fair to everyone, even when people have different characteristics like gender or race. One way they do this is by checking how well the AI system works for different groups of people. The problem is that some groups might be too small to get a good understanding of how well the AI system works for them. This paper presents a new way to solve this problem, which gives more accurate results even when dealing with very small groups. It also shows how they can identify the reasons why an AI system might not be fair. |
Keywords
* Artificial intelligence * Inference * Regression * Synthetic data
