Summary of Mitigating Gender Bias in Contextual Word Embeddings, by Navya Yarrabelly et al.
Mitigating Gender Bias in Contextual Word Embeddings
by Navya Yarrabelly, Vinay Damodaran, Feng-Guang Su
First submitted to arxiv on: 18 Nov 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper tackles the issue of bias in word embeddings, particularly in contextual models used for natural language processing tasks. The authors propose a novel objective function for masked-language modeling that significantly reduces gender bias while maintaining performance on downstream tasks. To evaluate this debiasing approach, they introduce new metrics aligned with their motivations. Additionally, they investigate and compare the sources of bias between static and contextual embeddings. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about making sure language models don’t have biases towards certain groups, like men or women. The current language models can learn these biases from the data they were trained on. To fix this, the authors created a new way to train the models that reduces these biases without affecting their performance. They also came up with new ways to measure how biased the models are and tested it all out. |
Keywords
» Artificial intelligence » Natural language processing » Objective function
