Summary of Smurfcat at Semeval-2024 Task 6: Leveraging Synthetic Data For Hallucination Detection, by Elisei Rykov et al.
SmurfCat at SemEval-2024 Task 6: Leveraging Synthetic Data for Hallucination Detection
by Elisei Rykov, Yana Shishkina, Kseniia Petrushina, Kseniia Titova, Sergey Petrakov, Alexander Panchenko
First submitted to arxiv on: 9 Apr 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper presents novel systems for the SemEval-2024 hallucination detection task, employing various strategies to compare model predictions with reference standards. These strategies include baselines, refining pre-trained encoders through supervised learning, and ensemble approaches utilizing high-performing models. The authors introduce three distinct methods that exhibit strong performance metrics, including generating additional training samples from unlabelled data. A comparative analysis of these approaches is provided, showcasing the premier method’s effectiveness with a commendable 9th place in the model-agnostic track and 17th place in the model-aware track. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper creates systems to detect hallucinations for SemEval-2024. It tries different ways to compare computer models’ predictions to real answers. The authors come up with three new methods that do well, including making extra training examples from unmarked data. They also compare these methods and show which one works best. |
Keywords
» Artificial intelligence » Hallucination » Supervised
