Summary of Mexa: Multilingual Evaluation Of English-centric Llms Via Cross-lingual Alignment, by Amir Hossein Kargaran et al.
MEXA: Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignment
by Amir Hossein Kargaran, Ali Modarressi, Nafiseh Nikeghbal, Jana Diesner, François Yvon, Hinrich Schütze
First submitted to arxiv on: 8 Oct 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper introduces MEXA, a method for evaluating the multilingual capabilities of pre-trained English-centric large language models (LLMs) using parallel sentences. The authors leverage the fact that these models use English as an intermediate language in their layers and compute the alignment between English and non-English languages to estimate model performance in other languages. They conduct studies on various parallel datasets, models from different families, and established downstream tasks. The results show that MEXA achieves a statistically significant average Pearson correlation of 0.90 with three downstream tasks across nine models and two parallel datasets, indicating its reliability as an estimation method. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary MEXA is a new way to measure how well English-centric language models work in other languages. These models are like super smart dictionaries that can understand and generate text in many languages. But we don’t really know if they’re good at speaking different languages until now. The authors of this paper created MEXA, which uses special pairs of sentences in the same language to see how well the model can translate between them. They tested it on lots of different models and languages and found that it’s very accurate! This is important because it helps us understand what these models are really capable of. |
Keywords
» Artificial intelligence » Alignment
