Summary of Pride and Prejudice: Llm Amplifies Self-bias in Self-refinement, by Wenda Xu et al.
Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinement
by Wenda Xu, Guanglei Zhu, Xuandong Zhao, Liangming Pan, Lei Li, William Yang Wang
First submitted to arxiv on: 18 Feb 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper investigates the self-bias of large language models (LLMs) in evaluating their own output. It formally defines LLM’s self-bias as the tendency to favor its own generation using two statistics. The study analyzes six LLMs on translation, constrained text generation, and mathematical reasoning tasks, finding that self-bias is prevalent across multiple languages and tasks. Although the self-refine pipeline improves model outputs’ fluency and understandability, it amplifies self-bias. To mitigate biases, larger models and external feedback with accurate assessment can reduce bias in the self-refine pipeline, leading to actual performance improvement. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper explores how large language models are biased towards their own output. The researchers show that these models tend to favor their own generation over other options. They studied six different models on various tasks and found that this bias is common across many languages and types of tasks. The team also discovered that while making the models more accurate can make them better at generating text, it also makes them more biased. To solve this problem, the researchers suggest using larger models and getting feedback from humans to reduce this bias. |
Keywords
» Artificial intelligence » Text generation » Translation
