Summary of Riff: Learning to Rephrase Inputs For Few-shot Fine-tuning Of Language Models, by Saeed Najafi and Alona Fyshe
RIFF: Learning to Rephrase Inputs for Few-shot Fine-tuning of Language Models
by Saeed Najafi, Alona Fyshe
First submitted to arxiv on: 4 Mar 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The study investigates the impact of modifying input text prompts on fine-tuning pre-trained language models (PLMs) for downstream text processing tasks. Researchers explored the use of parameter-efficient fine-tuning methods, such as LoRA, and introduced a few-shot paraphrase model to most effectively rewrite input text. Experiments on six few-shot text classification datasets showed that enriching data with paraphrases at train and test time improved performance beyond what could be achieved with parameter-efficient fine-tuning alone. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This study looks into how changing the words in the original prompt affects how well pre-trained language models do when used for a different task. They tried some new methods to make this work more efficiently, like adjusting just a few important model parts. The results showed that making the data better by adding similar sentences during training and testing helped even more. |
Keywords
* Artificial intelligence * Few shot * Fine tuning * Lora * Parameter efficient * Prompt * Text classification
