Summary of Queerbench: Quantifying Discrimination in Language Models Toward Queer Identities, by Mae Sosto et al.
QueerBench: Quantifying Discrimination in Language Models Toward Queer Identities
by Mae Sosto, Alberto Barrón-Cedeño
First submitted to arxiv on: 18 Jun 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI); Computers and Society (cs.CY)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This research paper investigates the potential harm caused by sentence completions generated by English large language models (LLMs) regarding LGBTQIA+ individuals. The study employs a new assessment framework, QueerBench, which utilizes a template-based approach and Masked Language Modeling (MLM) task to evaluate the discriminatory behavior of LLMs. The results show that LLMs exhibit more frequent discriminatory behavior towards LGBTQIA+ individuals, with a 7.2% difference gap in the QueerBench score of harmfulness. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper looks into how big language models can hurt people who are part of the LGBTQIA+ community. It wants to know if these models make sentences that might be harmful or biased towards this group. To do this, the researchers created a special way to test and measure the harm caused by these language models. They found out that these models tend to make more harmful sentences for people who are part of the LGBTQIA+ community. |
