Summary of Evaluating Large Language Models For Health-related Text Classification Tasks with Public Social Media Data, by Yuting Guo et al.
Evaluating Large Language Models for Health-Related Text Classification Tasks with Public Social Media Data
by Yuting Guo, Anthony Ovadje, Mohammed Ali Al-Garadi, Abeed Sarker
First submitted to arxiv on: 27 Mar 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper investigates the performance of large language models (LLMs) on social media-based health-related natural language processing tasks, which have historically been challenging. The study compares a supervised classic machine learning model based on Support Vector Machines (SVMs), three pretrained language models (RoBERTa, BERTweet, and SocBERT), and two LLM-based classifiers (GPT3.5 and GPT4) across six text classification tasks. The researchers propose three approaches to leverage LLMs for text classification: using them as zero-shot classifiers, annotators, or with few-shot examples for data augmentation. The results show that employing data augmentation using GPT-4 with relatively small human-annotated data achieves superior results compared to training with human-annotated data alone. Supervised learners outperform GPT-4 and GPT-3.5 in zero-shot settings. The study suggests that leveraging LLMs for data augmentation can develop smaller, more effective domain-specific NLP models. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper looks at how well large language models (LLMs) do on health-related social media posts. Right now, there aren’t many studies on this topic. The researchers compared different types of LLMs and a traditional machine learning model to see which one works best. They found that using the LLMs for data augmentation gets better results than just using human-annotated data. This study shows that LLMs can be helpful in developing smaller, more effective models for specific tasks. |
Keywords
* Artificial intelligence * Data augmentation * Few shot * Gpt * Machine learning * Natural language processing * Nlp * Supervised * Text classification * Zero shot
