Summary of Craft Your Dataset: Task-specific Synthetic Dataset Generation Through Corpus Retrieval and Augmentation, by Ingo Ziegler et al.
CRAFT Your Dataset: Task-Specific Synthetic Dataset Generation Through Corpus Retrieval and Augmentation
by Ingo Ziegler, Abdullatif Köksal, Desmond Elliott, Hinrich Schütze
First submitted to arxiv on: 3 Sep 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A novel method called Corpus Retrieval and Augmentation for Fine-Tuning (CRAFT) is proposed to generate synthetic datasets for specialized tasks. This approach leverages few-shot examples, large-scale public web-crawled corpora, and instruction-tuned large language models (LLMs) to augment task-specific training datasets. CRAFT is demonstrated to efficiently generate large-scale datasets for four diverse tasks: biology question-answering, medicine QA, commonsense QA, and summarization. The results show that CRAFT-based models outperform or achieve comparable performance to general LLMs for QA tasks, while CRAFT-based summarization models outperform human-curated data by 46 preference points. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary CRAFT is a new way to make training datasets for specific jobs. It starts with just a few examples of what the job looks like and then uses big collections of text from the internet to find more examples that are similar. Then, it asks special language models to change these texts into the right format for training. CRAFT can be used to make large datasets for different tasks, such as answering biology questions or summarizing text. The results show that models trained with CRAFT do just as well or even better than general language models. |
Keywords
» Artificial intelligence » Few shot » Fine tuning » Question answering » Summarization
