Summary of Improving Llm Group Fairness on Tabular Data Via In-context Learning, by Valeriia Cherepanova et al.
Improving LLM Group Fairness on Tabular Data via In-Context Learning
by Valeriia Cherepanova, Chia-Jung Lee, Nil-Jana Akpinar, Riccardo Fogliato, Martin Andres Bertran, Michael Kearns, James Zou
First submitted to arxiv on: 5 Dec 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Large language models (LLMs) have been shown to be effective on tabular prediction tasks in low-data regimes, leveraging internal knowledge and learning from instructions. However, LLMs can fail to generate predictions that satisfy group fairness, producing inequitable outcomes across groups. Conventional debiasing approaches for natural language tasks do not directly translate to mitigating group unfairness in tabular settings. This work investigates four empirical approaches to improve group fairness of LLM predictions on tabular datasets: fair prompt optimization, soft prompt tuning, strategic selection of few-shot examples, and self-refining predictions via chain-of-thought reasoning. Experiments on four tabular datasets using open-source and proprietary LLMs show the effectiveness of these methods in enhancing demographic parity while maintaining high overall performance. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Imagine trying to predict what will happen based on some data. Large language models are really good at doing this, but sometimes they don’t do it fairly for everyone. This paper looks at four ways to make sure the predictions are fair and good. They tested these methods using different types of data and large language models. The results show that these methods work well and can be used to make sure predictions are fair. |
Keywords
» Artificial intelligence » Few shot » Optimization » Prompt
