Summary of Pku-saferlhf: Towards Multi-level Safety Alignment For Llms with Human Preference, by Jiaming Ji et al.
PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preference
by Jiaming Ji, Donghai Hong, Borong Zhang, Boyuan Chen, Josef Dai, Boren Zheng, Tianyi Qiu, Boxun Li, Yaodong Yang
First submitted to arxiv on: 20 Jun 2024
Categories
- Main: Artificial Intelligence (cs.AI)
- Secondary: Computation and Language (cs.CL)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper introduces the PKU-SafeRLHF dataset, designed to promote research on safety alignment in large language models (LLMs). The dataset includes 44.6k refined prompts and 265k question-answer pairs with safety meta-labels for 19 harm categories and three severity levels. The dataset is annotated by Llama-family models and provides distinct perspectives on helpfulness and harmlessness for question-answering pairs. The authors also collected 166.8k preference data, including dual-preference and single-preference data. This dataset will be useful for the community in developing safe deployment strategies for LLMs. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper creates a big database to help make language models safer. Language models are like super smart computers that can answer questions. But sometimes they might give wrong answers or even hurt people’s feelings. The authors of this paper want to make sure these models don’t do that, so they created a special dataset with lots of examples and labels to help train the models to be safe. |
Keywords
» Artificial intelligence » Alignment » Llama » Question answering
