Loading Now

Summary of Real Sampling: Boosting Factuality and Diversity Of Open-ended Generation Via Asymptotic Entropy, by Haw-shiuan Chang et al.


REAL Sampling: Boosting Factuality and Diversity of Open-Ended Generation via Asymptotic Entropy

by Haw-Shiuan Chang, Nanyun Peng, Mohit Bansal, Anil Ramakrishna, Tagyoung Chung

First submitted to arxiv on: 11 Jun 2024

Categories

  • Main: Computation and Language (cs.CL)
  • Secondary: Machine Learning (cs.LG)

     Abstract of paper      PDF of paper


GrooveSquid.com Paper Summaries

GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!

Summary difficulty Written by Summary
High Paper authors High Difficulty Summary
Read the original abstract here
Medium GrooveSquid.com (original content) Medium Difficulty Summary
A novel decoding method called REAL (Residual Entropy from Asymptotic Line) sampling is proposed to balance factuality and diversity in large language models (LLMs). REAL sampling predicts an adaptive threshold of p based on the likelihood of LLMs hallucinating, adjusting the threshold to ensure improved factuality and diversity. A Token-level Hallucination Forecasting (THF) model is developed to predict asymptotic entropy without supervision, allowing for a more accurate assessment of LLM uncertainty. In the FactualityPrompts benchmark, REAL sampling demonstrates significant improvements in both factuality and diversity compared to nucleus sampling, as measured by retrieval-based metrics and human evaluation.
Low GrooveSquid.com (original content) Low Difficulty Summary
Large language models (LLMs) are super smart computers that can understand and generate human-like text. But sometimes they make mistakes by making things up that aren’t true. This paper is about a new way to help LLMs be more accurate while also being more creative. It’s like finding the right balance between being correct and being interesting. The researchers developed a special method called REAL sampling that can predict when an LLM is likely to make something up, and then adjust its behavior to avoid mistakes. They tested this method on a big dataset and showed that it can significantly improve the accuracy and diversity of LLMs.

Keywords

» Artificial intelligence  » Hallucination  » Likelihood  » Token