Summary of Upsample or Upweight? Balanced Training on Heavily Imbalanced Datasets, by Tianjian Li et al.
Upsample or Upweight? Balanced Training on Heavily Imbalanced Datasets
by Tianjian Li, Haoran Xu, Weiting Tan, Kenton Murray, Daniel Khashabi
First submitted to arxiv on: 6 Oct 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Machine Learning (cs.LG); Machine Learning (stat.ML)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This research paper investigates the effectiveness of two common strategies for addressing the imbalance between high- and low-resource languages in multilingual settings. The strategies, Temperature Sampling and Scalarization, aim to mitigate data scarcity by upsampling or upweighting low-resource data. While these methods are often used interchangeably, their equivalence has not been thoroughly studied, leading to a lack of understanding about which approach is more effective. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper focuses on improving language models in multilingual settings where high-resource languages have much more data than low-resource ones. The authors look into two ways to deal with this imbalance: upsampling or upweighting the limited data from low-resource languages. They want to see if these methods are really equivalent, as they’re often used together without proof. |
Keywords
* Artificial intelligence * Temperature
