Summary of Recall: Membership Inference Via Relative Conditional Log-likelihoods, by Roy Xie et al.
ReCaLL: Membership Inference via Relative Conditional Log-Likelihoods
by Roy Xie, Junlin Wang, Ruomin Huang, Minxing Zhang, Rong Ge, Jian Pei, Neil Zhenqiang Gong, Bhuwan Dhingra
First submitted to arxiv on: 23 Jun 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed ReCaLL (Relative Conditional Log-Likelihood) model is a novel membership inference attack designed to detect pretraining data used in large language models. By leveraging their conditional language modeling capabilities, ReCaLL examines the relative change in log-likelihoods when prefixing target data points with non-member context. The model achieves state-of-the-art performance on the WikiMIA dataset and can be further improved using an ensemble approach. Moreover, the study provides insights into how large language models leverage membership information for effective inference at both sequence and token levels. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Large language models are getting bigger! But that means they’re learning from a lot of data, some of which might not be public or fair to use. Scientists want to make sure this data is transparent so we can trust the results. A new way to detect this hidden data uses something called membership inference attacks. This attack, called ReCaLL, looks at how well a model does when it’s given information from either its own training data or other sources. The researchers tested ReCaLL and found it worked really well! They even showed that adding more models together made it even better. |
Keywords
* Artificial intelligence * Inference * Log likelihood * Pretraining * Recall * Token
