Summary of Sequence-level Leakage Risk Of Training Data in Large Language Models, by Trishita Tiwari and G. Edward Suh
Sequence-Level Leakage Risk of Training Data in Large Language Models
by Trishita Tiwari, G. Edward Suh
First submitted to arxiv on: 15 Dec 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This research paper analyzes sequence-level probabilities to quantify the risk of training data extraction from Large Language Models (LLMs). By studying decoding schemes, model sizes, prefix lengths, partial sequence leakages, and token positions, the authors uncover new insights that were not possible in previous works. The study is performed on two pre-trained models, Llama and OPT, trained on Common Crawl and The Pile respectively. The findings show that Extraction Rate underestimates the threat of leakage by as much as 2.14X. Larger models and longer prefixes can extract more data, but this is not true for individual sequences. Partial leakage in decoding schemes like top-k and top-p is not easier than leaking verbatim training data. Extracting later tokens is up to 10.12X easier than extracting earlier tokens. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This study looks at how well Large Language Models remember their training data by analyzing the probability of extracting individual sequences. The researchers use two different models, Llama and OPT, and find some surprising things about how they work. They show that most metrics used to measure leakage are wrong, and that bigger models don’t always mean you can extract more data. They also found that it’s easier to get training data by looking at the end of a sequence rather than the start. |
Keywords
» Artificial intelligence » Llama » Probability » Token
