Summary of Securing Multi-turn Conversational Language Models From Distributed Backdoor Triggers, by Terry Tong et al.
Securing Multi-turn Conversational Language Models From Distributed Backdoor Triggers
by Terry Tong, Jiashu Xu, Qin Liu, Muhao Chen
First submitted to arxiv on: 4 Jul 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Medium Difficulty summary: Large language models (LLMs) have become increasingly capable of handling longer context lengths, enabling them to understand nuances in text and engage in multi-turn dialogues. However, our paper reveals a vulnerability that leverages the LLM’s strengths to harm users: the backdoor attack. We demonstrate that LLMs can capture combinational backdoor representations, which only activate when specific trigger utterances are presented together. Empirically verifying this representation’s invariance to trigger position, we show that inserting a single extra token into 5% of the data can achieve an Attack Success Rate (ASR) of over 99%. Our results demonstrate generalizability with any trigger, making it challenging to defend against backdoors. We analyze the distributed backdoor’s impact on defending large input and output spaces, and propose a decoding time defense – decayed contrastive decoding – that scales linearly with response sequence length and reduces the backdoor’s effectiveness. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Low Difficulty summary: This paper talks about language models that can have long conversations. It seems like they’re getting smarter! But researchers found a way to trick them into doing something bad. They call it a “backdoor” attack. It works by saying specific words together, which makes the model do what you want it to do without you even asking nicely. The good news is that there’s a new way to defend against these attacks called “decayed contrastive decoding”. It helps keep the backdoors from being activated. |
Keywords
» Artificial intelligence » Token
