Summary of Human-llm Hybrid Text Answer Aggregation For Crowd Annotations, by Jiyi Li
Human-LLM Hybrid Text Answer Aggregation for Crowd Annotations
by Jiyi Li
First submitted to arxiv on: 22 Oct 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Human-Computer Interaction (cs.HC); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper investigates the role of Large Language Models (LLMs) in aggregating text answers from crowd annotators. Recent studies have focused on individual worker performance, but this work looks at the scenario where LLMs are used as aggregators to combine text answers. The authors propose a hybrid method combining human and LLM contributions using a Creator-Aggregator Multi-Stage (CAMS) framework. Experiments were conducted on public datasets, showing the effectiveness of the approach in improving answer aggregation quality. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about how computers can help people make sense of lots of answers given by many individuals. When a lot of people give different answers to the same question, it’s hard to figure out which one is right. Recently, really smart computer models called Large Language Models (LLMs) have been shown to be good at helping with this problem. This paper looks at how LLMs can help when they are used to combine lots of short text answers from many people into a single answer. The authors came up with a new way to do this using both human and computer input, and tested it on real datasets. They found that their approach worked well and could help make the process of finding the best answer faster and more accurate. |
