Summary of Droj: a Prompt-driven Attack Against Large Language Models, by Leyang Hu et al.
DROJ: A Prompt-Driven Attack against Large Language Models
by Leyang Hu, Boran Wang
First submitted to arxiv on: 14 Nov 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Large Language Models (LLMs) have made significant progress in natural language processing tasks. However, their training on internet-sourced datasets can lead to the generation of objectionable content. To address this issue, extensive alignment with human feedback is necessary to avoid harmful outputs. Despite these efforts, LLMs remain vulnerable to adversarial jailbreak attacks, which manipulate prompts to elicit undesirable responses. Our novel approach, Directed Representation Optimization Jailbreak (DROJ), optimizes jailbreak prompts at the embedding level to shift hidden representations towards affirmative responses. Evaluations on the LLaMA-2-7b-chat model show that DROJ achieves a 100% Attack Success Rate, effectively preventing direct refusals. However, the model occasionally produces repetitive and non-informative responses. To mitigate this, we introduce a helpfulness system prompt enhancing the utility of the model’s responses. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Large Language Models are super smart computers that can understand and generate human-like language. Sometimes, they can produce content that is not nice or acceptable. To fix this problem, people need to work closely with the models to make sure they don’t create harmful things. Despite these efforts, some bad guys can trick the models into saying mean things. We came up with a new way to stop this from happening called Directed Representation Optimization Jailbreak (DROJ). It works by changing how the model thinks about certain words so it will only say nice things. Our tests show that DROJ is very good at keeping the model safe and only responding positively. |
Keywords
» Artificial intelligence » Alignment » Embedding » Llama » Natural language processing » Optimization » Prompt
