Summary of Advancing Single- and Multi-task Text Classification Through Large Language Model Fine-tuning, by Hang Zhao et al.
Advancing Single- and Multi-task Text Classification through Large Language Model Fine-tuning
by Hang Zhao, Qile P. Chen, Yijing Barry Zhang, Gang Yang
First submitted to arxiv on: 11 Dec 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper investigates the performance of encoder-only models (e.g., BERT, RoBERTa) and large language models (LLMs) in text classification tasks. The study employs a diverse range of models and methods, varying in size and architecture, including both fine-tuned and pre-trained approaches. The authors assess the performances of LLMs on two datasets: 20 Newsgroups (20NG) and MASSIVE. They also explore the multi-task capabilities of both model types by combining multiple classification tasks into a single model using data from both datasets. The results show that fully fine-tuned Llama3-70B models outperform RoBERTa-large and other decoder LLMs across various classification tasks and datasets. Additionally, the consolidated multi-task fine-tuned LLMs matched the performance of dual-model setups in both tasks across both datasets. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper compares two types of language models: BERT-like models and big language models. They want to see which one works better for classifying text. The researchers tested these models on different data sets, including news articles and large texts. They also tried combining multiple tasks into one model to see how well it would work. Surprisingly, the bigger language models worked better than the smaller ones in many cases. This study shows that using big language models can be a good way to classify text. |
Keywords
» Artificial intelligence » Bert » Classification » Decoder » Encoder » Multi task » Text classification
