Summary of M-rewardbench: Evaluating Reward Models in Multilingual Settings, by Srishti Gureja et al.
M-RewardBench: Evaluating Reward Models in Multilingual Settings
by Srishti Gureja, Lester James V. Miranda, Shayekh Bin Islam, Rishabh Maheshwary, Drishti Sharma, Gusti Winata, Nathan Lambert, Sebastian Ruder, Sara Hooker, Marzieh Fadaee
First submitted to arxiv on: 20 Oct 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper investigates the performance of reward models (RMs) in multilingual settings, building on their success in driving state-of-the-art language model results. RMs are typically trained and evaluated in English, leaving their capabilities in non-English languages understudied. The authors construct a benchmark, M-RewardBench, comprising 2.87k preference instances across 23 typologically diverse languages, testing chat, safety, reasoning, and translation capabilities. They rigorously evaluate various RMs on this benchmark, revealing significant performance gaps between English and non-English languages. Notably, RM preferences can change substantially from one language to another, influenced by factors like translation quality and high-resource languages. The study also presents the M-RewardBench dataset and codebase for further research. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Reward models are special tools that help improve how well language models understand us. They’re usually trained on English data, but what if we want to use them in other languages? This study looks at how reward models perform when used in many different languages. The researchers created a special test set with lots of examples to see which types of tasks RMs are good for (like chat, safety, and translation). They found that RMs do better in some languages than others, and that their performance changes depending on the quality of translations from English to those languages. |
Keywords
» Artificial intelligence » Language model » Translation
