Summary of Besstie: a Benchmark For Sentiment and Sarcasm Classification For Varieties Of English, by Dipankar Srirag and Aditya Joshi and Jordan Painter and Diptesh Kanojia
BESSTIE: A Benchmark for Sentiment and Sarcasm Classification for Varieties of English
by Dipankar Srirag, Aditya Joshi, Jordan Painter, Diptesh Kanojia
First submitted to arxiv on: 6 Dec 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper introduces BESSTIE, a benchmark for sentiment and sarcasm classification in three English language varieties: Australian (en-AU), Indian (en-IN), and British (en-UK). To create this benchmark, the authors collected datasets from Google Place reviews and Reddit comments, filtered by location and topic, and annotated them manually. They fine-tuned nine large language models (LLMs) on these datasets and evaluated their performance. The results show that the models perform better in inner-circle varieties (en-AU and en-UK) but struggle with cross-variety generalisation, particularly for Indian English (en-IN). This highlights the need for language variety-specific datasets like BESSTIE to develop more equitable LLMs. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper creates a special dataset called BESSTIE that helps computers understand how people feel when they write in different types of English. The authors collected information from websites and asked native speakers to label it with words like “happy” or “sad”. They then tested computer models on this data and found out that the models are better at understanding certain types of English than others. This is important because computers need to be able to understand all kinds of writing to be useful. |
Keywords
» Artificial intelligence » Classification
