Summary of Representation Noising: a Defence Mechanism Against Harmful Finetuning, by Domenic Rosati et al.
Representation Noising: A Defence Mechanism Against Harmful Finetuning
by Domenic Rosati, Jan Wehner, Kai Williams, Łukasz Bartoszcze, David Atanasov, Robie Gonzales, Subhabrata Majumdar, Carsten Maple, Hassan Sajjad, Frank Rudzicz
First submitted to arxiv on: 23 May 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper proposes a defense mechanism against fine-tuning attacks on large language models (LLMs). The authors recognize that even closed models are vulnerable to weight stealing and fine-tuning APIs, which can be used for harmful purposes. To address this issue, they introduce Representation Noising (RepNoise), a method that removes information about harmful representations, making it difficult for attackers to recover them during fine-tuning. RepNoise generalizes across different subsets of harm and does not degrade the overall capability of LLMs. The authors demonstrate the efficacy of their defense through empirical evidence, highlighting its “depth” in removing information across all layers of the model. They also identify areas where RepNoise remains ineffective, providing insights for future research. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about keeping large language models safe from being used for bad things. Even if the models are not fully released, they can still be fine-tuned to do harm. The authors want to fix this problem by creating a way to hide information that could be used for evil purposes. They call it Representation Noising (RepNoise) and show that it works even when attackers have access to the model’s weights. RepNoise is good at hiding harmful representations, but there are still some things it can’t do. The authors think this is important research because it helps keep our language models safe. |
Keywords
» Artificial intelligence » Fine tuning
