Summary of On the Loss Of Context-awareness in General Instruction Fine-tuning, by Yihan Wang et al.
On the Loss of Context-awareness in General Instruction Fine-tuning
by Yihan Wang, Andrew Bai, Nanyun Peng, Cho-Jui Hsieh
First submitted to arxiv on: 5 Nov 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
Summary difficulty | Written by | Summary |
---|---|---|
High | Paper authors | High Difficulty Summary Read the original abstract here |
Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This research paper investigates the impact of supervised fine-tuning (SFT) on the context awareness of large language models (LLMs). Specifically, it examines how SFT affects the ability of LLMs to extract and understand information from user-provided context and respond accordingly. The study finds that SFT can lead to a loss of context awareness in open-source models, particularly when the chat template is applied to input prompts. The researchers identify a bias towards different roles learned during conversational instruction fine-tuning as the cause of this performance decline. To mitigate this issue, they propose a metric to identify context-dependent examples from general instruction fine-tuning datasets and apply conditional instruction fine-tuning with a context-dependency indicator. Experimental results on four downstream tasks and three pre-trained LLMs demonstrate that this approach effectively preserves context awareness without compromising general instruction-following capabilities. |
Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper looks at how big language models work after they’ve been trained to follow instructions. It finds that when these models are taught to respond to specific prompts, it can make them worse at understanding the bigger picture and using information from their training data. The researchers think this is because the models start to focus on different roles or jobs they’ve learned about during training. To fix this problem, they suggest a way to identify situations where the model should use its original context-aware abilities and propose a new method to help the model keep these skills while still learning from new instructions. |
Keywords
* Artificial intelligence * Fine tuning * Supervised