Summary of Sftmix: Elevating Language Model Instruction Tuning with Mixup Recipe, by Yuxin Xiao et al.
SFTMix: Elevating Language Model Instruction Tuning with Mixup Recipe
by Yuxin Xiao, Shujian Zhang, Wenxuan Zhou, Marzyeh Ghassemi, Sanqiang Zhao
First submitted to arxiv on: 7 Oct 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Large language models (LLMs) are trained on instruction-response pairs using next-token prediction (NTP), a process called instruction tuning. Researchers have been trying to improve this process by creating higher-quality datasets, but this often requires filtering data with proprietary LLMs or human annotation. A new approach, SFTMix, has been proposed that goes beyond the conventional NTP paradigm without relying on curated datasets. SFTMix uses Mixup-based regularization to identify examples with varying confidence levels and bridge the gap between confident and unconfident data. This allows for better learning on both types of examples, reducing overfitting in confident ones and enhancing generalization in unconfident ones. The effectiveness of SFTMix is demonstrated in instruction-following and healthcare-specific tasks, with consistent improvements across different LLM families and datasets. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Large language models are trained to follow instructions. To make them better at this, researchers have been working on a process called instruction tuning. They’ve been trying to create better training data, but this often requires using special computers or having people manually label the data. A new idea is to mix up different examples of training data and use that to help the model learn. This approach, called SFTMix, helps the model avoid getting too good at one type of example and not being able to generalize to others. It also makes the model better at learning from examples it’s not very sure about. The new approach works well in two different types of tasks: following instructions and working with healthcare data. |
Keywords
» Artificial intelligence » Generalization » Instruction tuning » Overfitting » Regularization » Token
