Summary of Instruction Tuning with Loss Over Instructions, by Zhengyan Shi et al.
Instruction Tuning With Loss Over Instructions
by Zhengyan Shi, Adam X. Yang, Bin Wu, Laurence Aitchison, Emine Yilmaz, Aldo Lipani
First submitted to arxiv on: 23 May 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed Instruction Modelling (IM) method trains language models by applying a loss function to the instruction and prompt part rather than solely to the output part. This approach is shown to effectively improve model performance on various NLP tasks and open-ended generation benchmarks, with some cases seeing over 100% improvement. Factors influencing IM’s effectiveness include the ratio of instruction length to output length in training data and the number of training examples. IM is particularly beneficial when trained on datasets with lengthy instructions paired with brief outputs or under the Superficial Alignment Hypothesis (SAH) where a small amount of training examples are used for instruction tuning. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper helps language models learn to follow instructions better. They train the models by focusing on what the instructions say, rather than just looking at what the output is. This makes the models do a lot better on certain tasks and tests. The researchers found that this method works best when there are long instructions paired with short outputs, or when they’re using a small amount of training examples. |
Keywords
» Artificial intelligence » Alignment » Instruction tuning » Loss function » Nlp » Prompt
