Summary of Refusal Tokens: a Simple Way to Calibrate Refusals in Large Language Models, by Neel Jain et al.
Refusal Tokens: A Simple Way to Calibrate Refusals in Large Language Models
by Neel Jain, Aditya Shrivastava, Chenyang Zhu, Daben Liu, Alfy Samuel, Ashwinee Panda, Anoop Kumar, Micah Goldblum, Tom Goldstein
First submitted to arxiv on: 9 Dec 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Computation and Language (cs.CL)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A novel approach to engineering language models that can refuse to answer certain questions or follow specific instructions is proposed in this paper. The authors focus on enabling models to output refusal messages for various categories of user queries, such as ill-posed questions or requests that require information beyond the model’s knowledge horizon. To achieve this, they introduce “refusal tokens” that are prepended to the model’s responses during training and can be controlled during inference to steer the model’s refusal behavior. This allows for controlling a single model’s refusal rates without requiring further fine-tuning. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Language models are being developed to refuse certain questions or follow specific instructions. The goal is to make them output “refusal messages” when asked something that doesn’t make sense or requires information they don’t know. Right now, this involves training multiple models with different levels of refusal and adjusting the results for each user’s preferences. This can be time-consuming and require creating new models. To simplify things, the authors propose using “refusal tokens” during training to make the model more likely or less likely to refuse a question based on its category. |
Keywords
» Artificial intelligence » Fine tuning » Inference
