Summary of Tuning Llms with Contrastive Alignment Instructions For Machine Translation in Unseen, Low-resource Languages, by Zhuoyuan Mao and Yen Yu
Tuning LLMs with Contrastive Alignment Instructions for Machine Translation in Unseen, Low-resource Languages
by Zhuoyuan Mao, Yen Yu
First submitted to arxiv on: 11 Jan 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper introduces a novel approach to address two challenges in machine translation on large language models. The first challenge is expanding supported languages to previously unseen ones, while the second relates to the lack of data in low-resource languages. To tackle these challenges, the authors propose contrastive alignment instructions (AlignInstruct) that emphasize cross-lingual supervision via a discriminator built using statistical word alignments. The authors fine-tune BLOOMZ models (1b1, 3b, and 7b1) in up to 24 unseen languages and demonstrate that LLMs can effectively translate unseen languages using MT instructions (MTInstruct). They also show that AlignInstruct leads to consistent improvements in translation quality across multiple translation directions involving English. Additionally, the authors find that discriminator-based instructions outperform their generative counterparts as cross-lingual instructions. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper helps us make machines better at translating languages they don’t know well. There are two big challenges: teaching machines new languages and helping them translate languages with very little information available. The researchers created a new way to give machines guidance called contrastive alignment instructions (AlignInstruct). This method uses statistical word alignments to help the machine learn from similar words in different languages. They tested this approach by fine-tuning special models, called BLOOMZ, on 24 languages they had never seen before. The results showed that machines can translate these new languages well and that their new approach works better than other methods. |
Keywords
» Artificial intelligence » Alignment » Fine tuning » Translation
