Summary of Under the Surface: Tracking the Artifactuality Of Llm-generated Data, by Debarati Das et al.
Under the Surface: Tracking the Artifactuality of LLM-Generated Data
by Debarati Das, Karin De Langis, Anna Martin-Boyle, Jaehyung Kim, Minhwa Lee, Zae Myung Kim, Shirley Anugrah Hayati, Risako Owan, Bin Hu, Ritik Parkar, Ryan Koo, Jonginn Park, Aahan Tyagi, Libby Ferland, Sanjali Roy, Vincent Liu, Dongyeop Kang
First submitted to arxiv on: 26 Jan 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper explores the growing use of large language models (LLMs) to generate artificial data, which is increasingly used in training cycles. This study aggregates various types of LLM-generated text data, from task labels to free-form text, and evaluates their quality and implications compared to human data across existing benchmarks. Despite matching human performance in some tasks, the study reveals significant hidden disparities, especially in complex tasks where LLMs lack nuanced understanding. The paper highlights the need for ethical practices in data creation and the importance of addressing biases and artifacts produced in LLM-generated content. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This study looks at how big computers can make fake information. These computers are getting better at making things like labels, prompts, and even whole conversations. But this fake information is being used to teach other computers, and that’s a problem. The computer-made info might look good at first, but it’s missing the special touches that humans add. This study shows how different these two kinds of information are, and why we need to be careful when using these big computers. |
