Summary of On the Benefits Of Fine-grained Loss Truncation: a Case Study on Factuality in Summarization, by Lorenzo Jaime Yu Flores et al.
On the Benefits of Fine-Grained Loss Truncation: A Case Study on Factuality in Summarization
by Lorenzo Jaime Yu Flores, Arman Cohan
First submitted to arxiv on: 9 Mar 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper tackles the problem of hallucination in AI models developed for text summarization and simplification tasks. Hallucination occurs when a model generates untruthful information due to training on unaligned data. One approach, Loss Truncation (LT), aims to address this issue by modifying the standard log loss to remove noisy examples during training. However, the authors find that LT alone is not sufficient in reducing hallucinated entities and propose refining its performance by studying the behavior of underlying losses between factual and non-factual examples. They leverage these insights to develop fine-grained NLL loss and data cleaning strategies, achieving improved hallucination reduction on certain datasets. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper explores how AI models can sometimes make up false information. This happens when they’re trained on wrong or incomplete data. The researchers looked at a technique called Loss Truncation to fix this issue. They found that even with this approach, some models still made up false info. So, they studied what makes true and fake information different and developed new strategies to help AI models be more accurate. These improvements can help reduce the amount of false information generated by AI models. |
Keywords
» Artificial intelligence » Hallucination » Summarization
