Summary of The Butterfly Effect Of Altering Prompts: How Small Changes and Jailbreaks Affect Large Language Model Performance, by Abel Salinas and Fred Morstatter
The Butterfly Effect of Altering Prompts: How Small Changes and Jailbreaks Affect Large Language Model Performance
by Abel Salinas, Fred Morstatter
First submitted to arxiv on: 8 Jan 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Medium Difficulty summary: Large Language Models (LLMs) are widely used for labeling data across various domains and tasks. Practitioners prompt LLMs to answer questions through a series of decisions, including word choice, output format, and sensitive topic addressing. This study investigates whether variations in prompting construction affect the ultimate decision made by the LLM. The researchers used text classification tasks with different prompt variations and found that even slight perturbations, such as adding spaces or using specific formats like XML, can significantly alter the labeled data. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Low Difficulty summary: Researchers are studying how people use Large Language Models (LLMs) to label information. They’re looking at how the way you ask a question affects what the LLM answers. The scientists tried asking different types of questions and found that even small changes, like adding a space or using special formats, can change the answer. This is important because it helps us understand how we can use these language models to get accurate information. |
Keywords
» Artificial intelligence » Prompt » Prompting » Text classification
