Summary of Concse: Unified Contrastive Learning and Augmentation For Code-switched Embeddings, by Jangyeong Jeon et al.
ConCSE: Unified Contrastive Learning and Augmentation for Code-Switched Embeddings
by Jangyeong Jeon, Sangyeon Cho, Minuk Ma, Junyoung Kim
First submitted to arxiv on: 28 Aug 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper delves into Code-Switching (CS), where two languages, English and Korean, intertwine within a single utterance. The authors highlight the need for research in this area due to the unique grammatical differences between the two languages. A novel Koglish dataset is introduced, tailored for CS scenarios, to address these complexities. The study demonstrates the importance of CS datasets in various tasks, such as language modeling and natural language inference. Foundation multilingual language models trained on monolingual versus CS datasets exhibit differential outcomes. SimCSE, a model showing strengths in monolingual sentence embedding, is found to have limitations in CS scenarios. A novel Koglish-NLI dataset is constructed using a CS augmentation-based approach to verify this. The proposed ConCSE method, combining contrastive learning and augmentation, enhances the semantics of CS sentences. Experimental results validate ConCSE with an average performance enhancement of 1.77% on the Koglish-STS tasks. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Code-Switching happens when people mix two languages in one sentence. This paper looks at how English and Korean can be mixed together. Researchers found that current theories about Code-Switching don’t fully explain how these two languages work together. They created a new dataset, called Koglish, to help with this challenge. The study shows that language models trained on just one language or both languages perform differently. A model good at understanding English sentences might struggle when mixing English and Korean. The researchers also introduced a new way to train language models for Code-Switching scenarios, which improves their performance. |
Keywords
» Artificial intelligence » Embedding » Inference » Semantics
