Summary of A Realistic Threat Model For Large Language Model Jailbreaks, by Valentyn Boreiko et al.
A Realistic Threat Model for Large Language Model Jailbreaks
by Valentyn Boreiko, Alexander Panfilov, Vaclav Voracek, Matthias Hein, Jonas Geiping
First submitted to arxiv on: 21 Oct 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This research paper proposes a unified threat model for comparing jailbreaking attacks on safety-tuned Large Language Models (LLMs). The authors build an N-gram model based on 1T tokens to evaluate the deviation of a jailbreak from natural text, dubbed “perplexity”. They also consider computational budget in terms of total FLOPs. The paper benchmarks popular attacks under this new threat model and finds that attacks based on discrete optimization outperform recent LLM-based attacks. The authors note that effective attacks exploit rare N-grams, often absent from real-world text or specific to code datasets. This work provides a comprehensive framework for analyzing and comparing jailbreaking attacks. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Imagine trying to trick a super smart computer program called an LLM into saying something it wouldn’t normally say. These kinds of tricks are called “jailbreaking” attacks, and they can be very harmful. Researchers have developed different methods to do this, but they don’t all work equally well. This paper proposes a new way to compare these different attack methods to see which ones are most effective. They find that some attacks are better at getting the LLM to say something unusual than others. The researchers also discover that the most successful attacks target rare or unusual words and phrases, often found in code or artificial data. |
Keywords
» Artificial intelligence » N gram » Optimization » Perplexity
