Summary of From Human Judgements to Predictive Models: Unravelling Acceptability in Code-mixed Sentences, by Prashant Kodali et al.
From Human Judgements to Predictive Models: Unravelling Acceptability in Code-Mixed Sentences
by Prashant Kodali, Anmol Goel, Likhith Asapu, Vamshi Krishna Bonagiri, Anirudh Govil, Monojit Choudhury, Manish Shrivastava, Ponnurangam Kumaraguru
First submitted to arxiv on: 9 May 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper presents a novel approach to analyzing and generating code-mixed sentences by explicitly modeling their naturalness and acceptability. The current methods rely on training corpora that reflect the distribution of acceptable code-mixed sentences, but this can be limiting. To address this issue, the authors construct Cline, a large dataset containing human acceptability judgments for English-Hindi code-mixed text. This dataset consists of 16,642 sentences sourced from two sources: synthetically generated code-mixed text and online social media samples. The analysis reveals that popular code-mixing metrics have low correlation with human acceptability judgments, highlighting the need for this new dataset. The authors then demonstrate the effectiveness of their approach using Multilayer Perceptron (MLP) models trained solely on code-mixing metrics, as well as fine-tuned pre-trained Multilingual Large Language Models (MLLMs). The results show that MLLMs outperform MLP models and even surpass ChatGPT’s zero- and few-shot capabilities. Additionally, the authors demonstrate the potential for zero-shot transfer from English-Hindi to English-Telugu acceptability judgments using their model checkpoints. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about a new way to understand and generate code-mixed sentences, which are when people mix two languages in one sentence. Right now, computers don’t do this very well because they just copy what humans have done before without thinking if it sounds natural or not. To fix this, the authors created a big dataset called Cline that has lots of examples of English-Hindi code-mixed sentences with human judges saying whether they sound good or bad. This helps computers learn to make their own judgments about what sounds natural and what doesn’t. The results show that using special types of artificial intelligence models can do better than other methods, even beating a popular language model called ChatGPT. This is important because it could help computers generate more realistic code-mixed text in the future. |
Keywords
» Artificial intelligence » Few shot » Language model » Zero shot
