Summary of Enhancing Low-resource Llms Classification with Peft and Synthetic Data, by Parth Patwa et al.
Enhancing Low-Resource LLMs Classification with PEFT and Synthetic Data
by Parth Patwa, Simone Filice, Zhiyu Chen, Giuseppe Castellucci, Oleg Rokhlenko, Shervin Malmasi
First submitted to arxiv on: 3 Apr 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Large Language Models operating in 0-shot or few-shot settings achieve competitive results in Text Classification tasks. The In-Context Learning (ICL) method typically achieves better accuracy than the 0-shot setting, but at the cost of efficiency due to longer input prompts. This paper proposes a strategy to make LLMs as efficient as 0-shot text classifiers while achieving comparable or better accuracy than ICL. The solution targets the low-resource setting where only four examples per class are available. A single LLM and few-shot real data are used, followed by generation, filtering, and Parameter-Efficient Fine-Tuning steps to create a robust and efficient classifier. Experimental results demonstrate competitive results on multiple text classification datasets using Large Language Models. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Large language models can quickly learn about texts without needing much training or examples. This paper shows how to make these models work just as well as others that need more time and data, while being more efficient. The solution uses a single model and only a few real-life examples of each type of text to create a reliable and fast classifier. |
Keywords
* Artificial intelligence * Few shot * Fine tuning * Parameter efficient * Text classification
